Water area image data annotation method, device, equipment and storage medium

Abstract

The embodiments of the present invention disclose a method, device, equipment and storage medium for annotating water image data. The method for annotating water image data includes: determining candidate annotation results in the water image to be annotated based on the characteristics of the target category to be annotated; wherein the candidate annotation results include the target category and / or target attribute information of the candidate annotation; receiving a user's judgment instruction on the candidate annotation results, and determining the final annotation result according to the judgment instruction; wherein the final annotation result includes the target category and / or target attribute information of the final annotation; and determining the annotation result of the water image to be annotated according to the final annotation result. The embodiments of the present invention can improve the efficiency of water image data annotation by determining the candidate annotation results, and no longer need to manually annotate the water images to be annotated one by one, and then improve the accuracy of water image data annotation by receiving the user's judgment instruction on the candidate annotation results.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of Internet technology, and in particular to a method, device, equipment and storage medium for annotating water area image data. Background Art

[0002] The existing technology is mainly aimed at image annotation of land vehicles. In this field, the shape, method and classification of image annotation are similar. The image annotation and data collection links of various research organizations or individuals in this field are relatively unified and have certain standards, so the annotation data of each organization or individual can be used universally.

[0003] However, the labeling classification for water areas is unclear, and it relies on manual labeling of single pictures, resulting in low labeling efficiency. Summary of the invention

[0004] Embodiments of the present invention provide a water area image data annotation method, device, equipment and storage medium to improve the efficiency of water area image data annotation.

[0005] In a first aspect, an embodiment of the present invention provides a water area image data labeling method, comprising:

[0006] Based on the characteristics of the target category to be labeled, determining candidate labeling results in the water area image to be labeled; wherein the candidate labeling results include the target category and / or target attribute information of the candidate labeling;

[0007] Receiving a user's judgment instruction on the candidate annotation results, and determining a final annotation result according to the judgment instruction; wherein the final annotation result includes the final annotated target category and / or target attribute information;

[0008] According to the final labeling result, a labeling result of the water area image to be labeled is determined.

[0009] Optionally, the target category and target attribute information to be marked are determined by presetting, and the target attribute information is associated with the target category.

[0010] Optionally, the target category to be marked includes at least one of the following: a ship, a drivable water area, a coastline, an underwater obstacle, an irrelevant object, and the ship itself;

[0011] Optionally, the target attribute information of the ship includes at least one of the following: boundary information, occlusion ratio and truncation ratio, and the attribute information of the drivable waters, coastline, underwater obstacles, irrelevant objects and the ship itself includes at least boundary information.

[0012] Optionally, the occlusion ratio refers to the ratio of the portion of the ship that is occluded by other objects in the water area image;

[0013] The truncation ratio refers to the ratio of the portion of the ship that is truncated by the edge of the water area image.

[0014] Optionally, the drivable waters include the coastline.

[0015] Optionally, the user's judgment instruction on the candidate annotation result includes at least one of the following:

[0016] A user's instruction to determine the target category and / or target attribute information of the candidate annotation;

[0017] A user's negative instruction on the target category and / or target attribute information of the candidate annotation;

[0018] The user's instruction to modify the target category and / or target attribute information of the candidate annotation.

[0019] Optionally, receiving a user's judgment instruction on the candidate annotation result, and determining a final annotation result according to the judgment instruction includes:

[0020] If a user's instruction to determine the candidate labeled target category and / or target attribute information is received, determining the candidate labeled target category and / or target attribute information associated with the instruction to determine as the final labeled target category and / or target attribute information;

[0021] If the user issues a negative instruction for the target category and / or target attribute information of the candidate annotation, deleting the target category and / or target attribute information of the candidate annotation associated with the negative instruction;

[0022] If the user has a modification instruction for the target category and / or target attribute information of the candidate annotation, the associated target category and / or target attribute information of the candidate annotation is modified according to the modification instruction, and the modified target category and / or target attribute information of the candidate annotation is determined as the final labeled target category and / or target attribute information.

[0023] Optionally, based on the characteristics of the target category to be labeled, determining candidate labeling results in the water area image to be labeled includes:

[0024] Based on the characteristics of the target category to be labeled, the candidate labeling results in the water area image to be labeled are determined by image recognition technology.

[0025] Optionally, the image recognition technology includes edge extraction.

[0026] Optionally, if the target category of any object to be labeled in the water image does not belong to any of ships, drivable waters, coastlines, irrelevant objects and the ship itself, the target category of the object to be labeled is determined to be an underwater obstacle to avoid omission of the object to be labeled in the water image.

[0027] Optionally, receiving a user's judgment instruction on the candidate annotation result includes:

[0028] A judgment instruction of the user on the candidate annotation result according to a preset annotation standard is received.

[0029] In a second aspect, an embodiment of the present invention further provides a water area image data annotation device, comprising:

[0030] A candidate result determination module is used to determine candidate annotation results in the water area image to be annotated based on the characteristics of the target category to be annotated; wherein the candidate annotation results include the target category and / or target attribute information of the candidate annotation;

[0031] A final result determination module is used to receive a user's judgment instruction on the candidate annotation results, and determine the final annotation result according to the judgment instruction; wherein the final annotation result includes the final annotation target category and / or target attribute information;

[0032] The water area image annotation module is used to determine the annotation result of the water area image to be annotated according to the final annotation result.

[0033] Optionally, the target category and target attribute information to be marked are determined by presetting, and the target attribute information is associated with the target category.

[0034] Optionally, the target category to be marked includes at least one of the following: a ship, a drivable water area, a coastline, an underwater obstacle, an irrelevant object, and the ship itself;

[0035] Optionally, the target attribute information of the ship includes at least one of the following: boundary information, occlusion ratio and truncation ratio, and the attribute information of the drivable waters, coastline, underwater obstacles, irrelevant objects and the ship itself includes at least boundary information.

[0036] Optionally, the occlusion ratio refers to the ratio of the portion of the ship that is occluded by other objects in the water area image;

[0037] The truncation ratio refers to the ratio of the portion of the ship that is truncated by the edge of the water area image.

[0038] Optionally, the drivable waters include the coastline.

[0039] Optionally, the user's judgment instruction on the candidate annotation result includes at least one of the following:

[0040] A user's instruction to determine the target category and / or target attribute information of the candidate annotation;

[0041] A user's negative instruction on the target category and / or target attribute information of the candidate annotation;

[0042] The user's instruction to modify the target category and / or target attribute information of the candidate annotation.

[0043] Optionally, the final result determination module is specifically used to:

[0044] If a user's instruction to determine the candidate labeled target category and / or target attribute information is received, determining the candidate labeled target category and / or target attribute information associated with the instruction to determine as the final labeled target category and / or target attribute information;

[0045] If the user issues a negative instruction for the target category and / or target attribute information of the candidate annotation, deleting the target category and / or target attribute information of the candidate annotation associated with the negative instruction;

[0046] If the user has a modification instruction for the target category and / or target attribute information of the candidate annotation, the associated target category and / or target attribute information of the candidate annotation is modified according to the modification instruction, and the modified target category and / or target attribute information of the candidate annotation is determined as the final labeled target category and / or target attribute information.

[0047] Optionally, the candidate result determination module is specifically used to:

[0048] Based on the characteristics of the target category to be labeled, the candidate labeling results in the water area image to be labeled are determined by image recognition technology.

[0049] Optionally, the image recognition technology includes edge extraction.

[0050] Optionally, if the target category of any object to be labeled in the water image does not belong to any of ships, drivable waters, coastlines, irrelevant objects and the ship itself, the target category of the object to be labeled is determined to be an underwater obstacle to avoid omission of the object to be labeled in the water image.

[0051] Optionally, the final result determination module is specifically used to:

[0052] A judgment instruction of the user on the candidate annotation result according to a preset annotation standard is received.

[0053] In a third aspect, an embodiment of the present invention further provides a device, including:

[0054] one or more processors;

[0055] a storage device for storing one or more programs,

[0056] When the one or more programs are executed by the one or more processors, the one or more processors implement the water area image data labeling method as described in any embodiment of the present invention.

[0057] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the water area image data labeling method as described in any embodiment of the present invention.

[0058] The embodiment of the present invention is based on the characteristics of the target category to be labeled. A candidate labeling result is obtained from the water image to be labeled, and then the candidate labeling result is corrected according to the user's judgment instruction on the candidate labeling result to obtain the final labeling result, and finally the result of labeling the water image to be labeled is obtained according to the final labeling result. The embodiment of the present invention can improve the efficiency of water image data labeling by determining the candidate labeling result, and no longer needs to manually label the water images to be labeled one by one, and then improve the accuracy of water image data labeling by receiving the user's judgment instruction on the candidate labeling result. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 is a flow chart of a method for labeling water area image data in Embodiment 1 of the present invention;

[0060] Figure 2A is a flow chart of a method for labeling water area image data in Embodiment 2 of the present invention;

[0061] Figure 2B is a schematic diagram of an example of labeling ships in a water area image in the second embodiment of the present invention;

[0062] Figure 2C is a schematic diagram of an example of marking a drivable water area in a water area image in the second embodiment of the present invention;

[0063] Figure 2D is a schematic diagram of an example of labeling a coastline in a water area image in the second embodiment of the present invention;

[0064] Figure 2E is a schematic diagram of an example of marking underwater obstacles in a water area image in the second embodiment of the present invention;

[0065] Figure 2F is a schematic diagram of an example of labeling irrelevant objects in a water area image in the second embodiment of the present invention;

[0066] Figure 2G is a schematic diagram of an example of labeling a vessel in a water area image in the second embodiment of the present invention;

[0067] Figure 2H This is an example diagram of marking drivable water areas by bypassing objects in the second embodiment of the present invention;

[0068] Fig.2I This is an example diagram of an example of marking drivable waters without bypassing objects in the second embodiment of the present invention;

[0069] Figure 2J is a schematic diagram of a marking example of a ship in the second embodiment of the present invention;

[0070] Figure 2K This is a schematic diagram of an example of reuse of drivable water area and coastline annotation in the second embodiment of the present invention;

[0071] Figure 2L This is a schematic diagram of an example of marking the coastline of a ship at the shore in the second embodiment of the present invention;

[0072] Figure 3 is a schematic diagram of the structure of a water area image data labeling device in Embodiment 3 of the present invention;

[0073] Figure 4 It is a schematic diagram of the structure of the device in the fourth embodiment of the present invention. DETAILED DESCRIPTION

[0074] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0075] Embodiment 1

[0076] Figure 1 is a flow chart of the method for labeling water area image data in the first embodiment of the present invention. This embodiment is applicable to the case of labeling water area images. The method can be executed by a water area image data labeling device, which can be implemented in software and / or hardware and can be configured in a device, for example, a background server or other device with communication and computing capabilities. Figure 1 As shown, the method specifically includes:

[0077] Step 101: Determine candidate annotation results in the water area image to be annotated based on the characteristics of the target category to be annotated; wherein the candidate annotation results include the target category and / or target attribute information of the candidate annotation.

[0078] With the rise of artificial intelligence, the most critical technology is deep learning. As the basis of deep learning, neural network is an input-oriented algorithm, and the accuracy of its results depends on data close to "infinite" magnitude. Therefore, excluding those complex intermediate links, the most important thing about deep learning is that it requires a large amount of data for training. Before data training, a large amount of data must be labeled as a leading experience for machine learning. Therefore, in the field of unmanned ship driving, it is necessary to let the machine replace the driver to identify various targets in the water, and then make correct judgments based on the type and distance of the target. It is necessary to teach the machine to recognize various complex scenes in the water, so a large number of water images need to be labeled. The machine can automatically recognize the target by learning the features in a large number of images.

[0079] In the prior art, water images are manually annotated. However, since learning of neural networks requires a large amount of annotated data, simply manually annotating will result in low efficiency and it will be difficult to unify the standards.

[0080] The features of the target category to be marked can be determined by analyzing the features of the water image. For example, the objects included in the water image are analyzed in advance to determine the features of the objects that affect the navigation of the ship and determine the category to which they belong.

[0081] Specifically, by determining the features of the target category to be labeled in the water image, the image part that meets the conditions is determined from the water image to be labeled as a candidate labeling result according to the features, and the candidate labeling result includes the target category matched by the features of the image part and the attribute information of the target category corresponding to the image part. The attribute information is used to describe the unique properties of the target category in the image to be labeled, such as the boundary information of the target category in the image, etc. The integrity of the target category can be determined through the boundary information, and other features can be described.

[0082] In the embodiment of the present invention, optionally, step 101 includes:

[0083] Based on the characteristics of the target category to be labeled, the candidate labeling results in the water area image to be labeled are determined by image recognition technology.

[0084] The image recognition technology refers to a technology that can identify the features of different target categories. For example, the image recognition technology includes but is not limited to template matching or edge extraction. Specifically, by determining the features of the target category to be labeled in the water image, the water image to be labeled is identified by image recognition technologies such as template matching or edge extraction, and the candidate labeling result is obtained by identifying the edge features after template matching or edge extraction.

[0085] Step 102: Receive a user's judgment instruction on the candidate annotation results, and determine a final annotation result according to the judgment instruction; wherein the final annotation result includes the final annotated target category and / or target attribute information.

[0086] After obtaining the candidate annotation results of the water area image to be annotated, the candidate annotation results are sent to the user, who judges the candidate annotation results to obtain the judgment results, generates and sends the judgment instructions according to the judgment results. After receiving the user's judgment instructions on the candidate annotation results, the user's judgment results on the candidate annotation results are obtained according to the judgment instructions, and the target categories and / or target attribute information of the candidate annotations in the candidate annotation results are determined to obtain the target categories and / or target attribute information of the final annotation. Among them, the user's judgment instructions on the candidate annotation results include memory judgment of the candidate annotation results of the entire water area image to be annotated, or judgment of the target categories of each candidate annotation in the candidate annotation results.

[0087] In an optional embodiment, the user's judgment instruction on the candidate annotation result includes at least one of the following:

[0088] A user's instruction to determine the target category and / or target attribute information of the candidate annotation;

[0089] A user's negative instruction on the target category and / or target attribute information of the candidate annotation;

[0090] The user's instruction to modify the candidate labeled target category and / or target attribute information.

[0091] The user's judgment instruction on the candidate annotation result is generated by the judgment result on the candidate annotation result. The user's judgment result on the candidate annotation result includes the determination, negation and modification result of the target category and / or target attribute information of the candidate annotation.

[0092] In an optional embodiment, receiving a user's judgment instruction on the candidate annotation result, and determining a final annotation result according to the judgment instruction includes:

[0093] If a user's instruction to determine the candidate labeled target category and / or target attribute information is received, determining the candidate labeled target category and / or target attribute information associated with the instruction to determine as the final labeled target category and / or target attribute information;

[0094] If the user issues a negative instruction for the target category and / or target attribute information of the candidate annotation, deleting the target category and / or target attribute information of the candidate annotation associated with the negative instruction;

[0095] If the user has a modification instruction for the target category and / or target attribute information of the candidate annotation, the associated target category and / or target attribute information of the candidate annotation is modified according to the modification instruction, and the modified target category and / or target attribute information of the candidate annotation is determined as the final labeled target category and / or target attribute information.

[0096] The target categories and corresponding target attribute information of the candidate annotation results of the water area image to be annotated are sequentially displayed to the user for judgment. The user can confirm, deny or modify the annotation results of any category. Exemplarily, the candidate annotation results of the water area image to be annotated are candidate target category 1, candidate target category 2 and candidate target category 3, as well as candidate target attribute information 1 of candidate target category 1, candidate target attribute information 2 of candidate target category 2 and candidate target attribute information 3 of candidate target category 3. The user judges the candidate annotation results of candidate target category 1, candidate target category 2 and candidate target category 3 respectively. For example, the judgment on the candidate annotation result of candidate target category 1 is affirmative, indicating that the recognition result of candidate target category 1 is correct, and candidate target category 1 is determined as the final target category 1; the judgment on the candidate annotation result of candidate target category 2 is negative, indicating that the recognition result of candidate target category 2 is wrong and inconvenient to modify, and candidate target category 2 is deleted from the final target category; the judgment on the candidate annotation result of candidate target category 3 is modified, indicating that the recognition result of candidate target category 3 is partially wrong, and the user's modification of candidate target category 3 is received, and the recognition result of candidate target category 3 is re-determined according to the modified content, such as modifying the category name of candidate target category 3 or the content in the corresponding target attribute information, and determining the modified content as the final target category 3.

[0097] By receiving the user's judgment instructions on the candidate annotation results, the accuracy of the final annotation results can be further guaranteed and the accuracy of data annotation can be improved.

[0098] Step 103: Determine the labeling result of the water area image to be labeled according to the final labeling result.

[0099] The annotation result of the water area image to be annotated is determined according to the final target category in the final annotation result determined by the user's judgment instruction on the candidate annotation result and the associated target attribute information. Exemplarily, the target annotation result is saved in a JSON format file, and the annotation result of the water area image to be annotated is obtained by combining the image file of the corresponding water area image to be annotated. For example, the JSON format file includes the location information, category information, and attribute information of the target category in the water area image to be annotated.

[0100] The embodiment of the present invention is based on the characteristics of the target category to be labeled. A candidate labeling result is obtained from the water image to be labeled, and then the candidate labeling result is corrected according to the user's judgment instruction on the candidate labeling result to obtain the final labeling result, and finally the result of labeling the water image to be labeled is obtained according to the final labeling result. The embodiment of the present invention can improve the efficiency of water image data labeling by determining the candidate labeling result, and no longer needs to manually label the water images to be labeled one by one, and then improve the accuracy of water image data labeling by receiving the user's judgment instruction on the candidate labeling result.

[0101] Embodiment 2

[0102] Figure 2A is a flow chart of a method for labeling water area image data in the second embodiment of the present invention. The second embodiment is further optimized on the basis of the first embodiment. Figure 2A As shown, the method includes:

[0103] Step 201, based on the characteristics of the target category to be labeled, determine the candidate labeling results in the water area image to be labeled; wherein the candidate labeling results include the target category and / or target attribute information of the candidate labeling, the target category and target attribute information to be labeled are determined by pre-setting, and the target attribute information is associated with the target category.

[0104] The existing technology is mainly aimed at land-based automobile image annotation. In this field, the shape, method, and classification of image annotation are similar. The image annotation and data collection of various research organizations or individuals in this field are relatively unified and have certain standards, so the data of various organizations or individuals can be used universally. However, there are relatively few annotations for water areas, the classification is unclear, and there is no unified or industry-wide standard.

[0105] Therefore, the target category and target attribute information to be marked are determined through pre-setting, that is, the objects in the water image that affect the navigation of ships are analyzed and uniformly classified, and a set of universal target category determination methods are established, so that the results of marking by various organizations according to the marking method of the embodiment of the present invention can be universal, thereby improving the marking efficiency of water images.

[0106] Specifically, by analyzing the unique properties in the water image, the objects in the water image that affect the navigation of the ship are pre-classified to obtain the classification results of the target category, and the associated target attribute information is set according to the properties of each category. For example, if the water image includes ships, the ships are set as the target category to be marked, and the drawing shape used for the boundary of the ship or the type of the ship is determined as the target attribute information of the ship.

[0107] After the target category to be labeled is pre-set, the features of the target category to be labeled are determined according to the features of the shape and position of the target category, so as to be recognized on the water area image to be labeled.

[0108] In an optional embodiment, the target category to be marked includes at least one of the following: a ship, a drivable water area, a coastline, an underwater obstacle, an irrelevant object, and the ship itself;

[0109] Correspondingly, the target attribute information of the ship includes at least one of the following: boundary information, occlusion ratio and truncation ratio, and the attribute information of the drivable waters, coastline, underwater obstacles, irrelevant objects and the ship itself includes at least boundary information.

[0110] According to the nature of the water image, the target categories to be labeled are pre-set to include ships, drivable waters, coastlines, underwater obstacles, irrelevant objects and the ship itself. Among them, ships represent other ships located on the water image, such as Figure 2B The figure shows a schematic diagram of the annotation of ships in a water image. The classification code of the ship can be represented by boat, and the white line in the figure is the annotation result; the navigable water area indicates the area where the ship can navigate in the water image, such as Figure 2C The figure shows a schematic diagram of the annotation example of the drivable water area in the water image. The classification code of the drivable water area can be represented by water_area. The white line in the figure is the annotation result. The coastline is used to represent the part where the drivable area is connected to the land or the sky, such as Figure 2D The figure shows a schematic diagram of an example of the annotation of the coastline in a water image. The classification code of the coastline can be represented by shoreline, and the white line in the figure is the annotation result; the underwater obstacle refers to the obstacle that affects the navigation of the ship in the navigable water area, such as Figure 2E The figure shows a schematic diagram of an example of marking underwater obstacles in a water image. The classification code of underwater obstacles can be represented by obstacle_in_water. The white line in the figure is the marking result. Irrelevant objects refer to objects that are located in the navigable waters but have no impact on the navigation of the ship or unrecognizable objects, such as Figure 2F The figure shows a schematic diagram of an example of labeling irrelevant objects in a water image. The classification code of irrelevant objects can be represented by boat_dont_care. The white line in the figure is the labeling result. The own ship refers to the ship where the device that takes pictures of the water image is located, such as Figure 2G The figure shows a schematic diagram of an example of labeling a vessel in a water image. The classification code of the vessel can be represented by my_boat, and the white line in the figure is the labeling result. All categories of water images can be represented by the above target categories, which is conducive to establishing a unified labeling standard.

[0111] The target attribute information of each target category is pre-set according to the characteristics of the target category. The target attributes of a ship include boundary information, occlusion ratio and truncation ratio. The boundary information is used to determine the position information and size information of the ship, and the occlusion ratio and truncation ratio are used to indicate the degree of influence of the ship on driving.

[0112] In the embodiment of the present invention, optionally, the occlusion ratio refers to the ratio of the portion of the ship that is occluded by other objects in the water image; the truncation ratio refers to the ratio of the portion of the ship that is truncate by the edge of the water image.

[0113] Occlusion means that part of the target ship is blocked by other objects in the image. In this case, the proportion of the blocked part of the target ship should be estimated, and the values ​​are 0.1, 0.2, 0.4, 0.6, 0.8, etc., and the overall size of the target object (including the visible part and the blocked part) is marked. Truncation means that the target ship is at the edge of the water image, that is, the ship is not completely in the image. In this case, only the visible part is marked as not to exceed the image, and the proportion of the truncated part is estimated, and the values ​​are 0.1, 0.2, 0.4, 0.6, 0.8, etc.

[0114] In the embodiment of the present invention, the drivable waters may include a coastline. Since the coastline is located at the edge of the drivable waters, the coastline may be determined based on the drivable waters, which may improve the accuracy of the coastline determination. Exemplarily, after the drivable waters are determined, a coastline portion is determined at the edge of the drivable waters based on the characteristics of the coastline.

[0115] The characteristics of the target category to be labeled are determined according to the preset target category and target attribute information. For example, the drawing shape of the coastline boundary information is a broken line, while the drawing shape of the boundary information of other target categories is a polygon. The coastline is used to mark the drivable water area and the part where the sea and the sky meet. If it is blocked, the coastline or the part where the sea and the sky meet can be estimated. The part where the sea and the sky meet connects the coastlines on both sides, because a water area image has only one coastline category. Identification is performed based on the distance characteristics of the object. For example, an object that is unclear due to a distance exceeding a certain threshold is determined as an irrelevant object, etc.

[0116] The accuracy of data labeling can be improved according to the pre-set characteristics of the target category to be labeled, and a unified labeling standard can be formed.

[0117] Step 202: receiving a user's judgment instruction on the candidate annotation results, and determining a final annotation result according to the judgment instruction; wherein the final annotation result includes the final annotated target category and / or target attribute information.

[0118] The user's judgment on the candidate annotation results may be determined based on the characteristics of the above-mentioned target category, or may also be determined based on a pre-set judgment criterion.

[0119] In an optional embodiment, receiving a user's judgment instruction on the candidate annotation result includes:

[0120] A judgment instruction of the user on the candidate annotation result according to a preset annotation standard is received.

[0121] The pre-set annotation standard is used to establish a unified standard for data annotation of water area images. For example, the technical points of water area image annotation are pre-set. When a ship or a water obstacle is located at the edge of the drivable water area, the entire outline of the ship or the water obstacle is marked. When marking the drivable water area, if there is an overlap between the drivable water area and the object, the annotator can bypass it or not. Figure 2H and 2I The following is a schematic diagram of an example of an object being labeled at the edge of a drivable area. Figure 2H The following is an example of marking drivable waters by bypassing objects. Fig.2I The following is an example of marking the drivable water area without bypassing the object. When marking a ship, the marking accuracy is improved for the part of the ship that is in contact with the water area, and the marking accuracy can be reduced for the part of the ship above the water, but the entire ship needs to be included, such as Figure 2J The following is a schematic diagram of a ship annotation example. When multiple ships are gathered together and it is difficult to distinguish a single ship, they are annotated according to the distance. When the distance exceeds a certain threshold, it is set as an irrelevant object. Otherwise, the single ship is distinguished and annotated. After the user determines the annotation result of the navigable waters, two points can be directly added to the polygon line of the navigable waters (or two existing points can be selected) to directly determine the boundary polyline of the coastline. Figure 2K The figure shows an example of reuse of navigable waters and coastline annotations. The polygonal area ABCD in the figure is the navigable waters, and the line between AB is directly determined to be the coastline. In addition, for many ships docked at the shore, it is not clear to determine the coastline, so the line formed outside the ship can be directly regarded as the coastline. Figure 2L Shown is a schematic diagram of an example of coastline marking when a ship is at the shore.

[0122] In a feasible embodiment, if the target category of any object to be marked in the water image does not belong to any of ships, navigable waters, coastlines, irrelevant objects and the ship itself, the target category of the object to be marked is determined to be an underwater obstacle to avoid missing the object to be marked in the water image. Exemplarily, for objects such as seabirds in the water image, when it can be clearly distinguished that these objects in the water are seabirds, there is no need to mark them as obstacles, because when the ship approaches the seabirds, the seabirds will fly away. However, if the objects are not clear and it is impossible to determine whether they are seabirds, they are marked as underwater obstacles. Any object that needs to be marked in the water area cannot be missed. When the distance is relatively far and many ships are close together, it is impossible to distinguish the dividing line between ships, and a polygon can be used to include several ships. The submerged part of the ship needs to be estimated and marked, and the occlusion ratio is estimated and marked.

[0123] The embodiment of the present invention achieves consistency of annotation standards between candidate annotation results and final annotation results by presetting the target categories and target attribute information to be annotated for water images, thereby improving the accuracy of annotation results. The embodiment of the present invention provides technical solutions for the methods of establishing training data sets and classification, and improves the versatility of establishing data sets for water images.

[0124] Step 203: Determine the labeling result of the water area image to be labeled according to the final labeling result.

[0125] The embodiment of the present invention is based on the characteristics of the target category to be labeled. A candidate labeling result is obtained from the water image to be labeled, and then the candidate labeling result is corrected according to the user's judgment instruction on the candidate labeling result to obtain the final labeling result, and finally the result of labeling the water image to be labeled is obtained according to the final labeling result. The embodiment of the present invention can improve the efficiency of water image data labeling by determining the candidate labeling result, and no longer needs to manually label the water images to be labeled one by one, and then improve the accuracy of water image data labeling by receiving the user's judgment instruction on the candidate labeling result.

[0126] Embodiment 3

[0127] Figure 3 Schematic diagram of the structure of the water area image data labeling device in the third embodiment of the present invention. This embodiment is applicable to the case of data labeling of water area images. Figure 3 As shown, the device comprises:

[0128] The candidate result determination module 310 is used to determine the candidate annotation results in the water area image to be annotated based on the characteristics of the target category to be annotated; wherein the candidate annotation results include the target category and / or target attribute information of the candidate annotation;

[0129] The final result determination module 320 is used to receive a user's judgment instruction on the candidate annotation results, and determine the final annotation result according to the judgment instruction; wherein the final annotation result includes the final annotation target category and / or target attribute information;

[0130] The water area image labeling module 330 is used to determine the labeling result of the water area image to be labeled according to the final labeling result.

[0131] The embodiment of the present invention is based on the characteristics of the target category to be labeled. A candidate labeling result is obtained from the water image to be labeled, and then the candidate labeling result is corrected according to the user's judgment instruction on the candidate labeling result to obtain the final labeling result, and finally the result of labeling the water image to be labeled is obtained according to the final labeling result. The embodiment of the present invention can improve the efficiency of water image data labeling by determining the candidate labeling result, and no longer needs to manually label the water images to be labeled one by one, and then improve the accuracy of water image data labeling by receiving the user's judgment instruction on the candidate labeling result.

[0132] Optionally, the target category and target attribute information to be marked are determined by presetting, and the target attribute information is associated with the target category.

[0133] Optionally, the target category to be marked includes at least one of the following: a ship, a drivable water area, a coastline, an underwater obstacle, an irrelevant object, and the ship itself;

[0134] Optionally, the target attribute information of the ship includes at least one of the following: boundary information, occlusion ratio and truncation ratio, and the attribute information of the drivable waters, coastline, underwater obstacles, irrelevant objects and the ship itself includes at least boundary information.

[0135] Optionally, the occlusion ratio refers to the ratio of the portion of the ship that is occluded by other objects in the water area image;

[0136] The truncation ratio refers to the ratio of the portion of the ship that is truncated by the edge of the water area image.

[0137] Optionally, the drivable waters include the coastline.

[0138] Optionally, the user's judgment instruction on the candidate annotation result includes at least one of the following:

[0139] A user's instruction to determine the target category and / or target attribute information of the candidate annotation;

[0140] A user's negative instruction on the target category and / or target attribute information of the candidate annotation;

[0141] The user's instruction to modify the target category and / or target attribute information of the candidate annotation.

[0142] Optionally, the final result determination module is specifically used to:

[0143] If a user's instruction to determine the candidate labeled target category and / or target attribute information is received, determining the candidate labeled target category and / or target attribute information associated with the instruction to determine as the final labeled target category and / or target attribute information;

[0144] If the user issues a negative instruction for the target category and / or target attribute information of the candidate annotation, deleting the target category and / or target attribute information of the candidate annotation associated with the negative instruction;

[0145] If the user has a modification instruction for the target category and / or target attribute information of the candidate annotation, the associated target category and / or target attribute information of the candidate annotation is modified according to the modification instruction, and the modified target category and / or target attribute information of the candidate annotation is determined as the final labeled target category and / or target attribute information.

[0146] Optionally, the candidate result determination module is specifically used to:

[0147] Based on the characteristics of the target category to be labeled, the candidate labeling results in the water area image to be labeled are determined by image recognition technology.

[0148] Optionally, the image recognition technology includes edge extraction.

[0149] Optionally, if the target category of any object to be labeled in the water image does not belong to any of ships, drivable waters, coastlines, irrelevant objects and the ship itself, the target category of the object to be labeled is determined to be an underwater obstacle to avoid omission of the object to be labeled in the water image.

[0150] Optionally, the final result determination module is specifically used to:

[0151] A judgment instruction of the user on the candidate annotation result according to a preset annotation standard is received.

[0152] The water area image data labeling device provided in the embodiment of the present invention can execute the water area image data labeling method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the water area image data labeling method.

[0153] Embodiment 4

[0154] Figure 4 It is a structural schematic diagram of a device provided in Embodiment 4 of the present invention. Figure 4A block diagram of an exemplary device 12 suitable for use in implementing embodiments of the present invention is shown. Figure 4 The device 12 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0155] like Figure 4 As shown, device 12 is in the form of a general purpose computing device. Components of device 12 may include, but are not limited to, one or more processors or processing units 16, system memory 28, and a bus 18 that connects various system components (including system memory 28 and processing units 16).

[0156] Bus 18 represents one or more of several types of bus structures, including a storage device bus or storage device controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. By way of example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0157] Device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by device 12, including volatile and non-volatile media, removable and non-removable media.

[0158] System storage 28 may include computer system readable media in the form of volatile storage devices, such as random access memory (RAM) 30 and / or cache memory 32. Device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 4 not shown, usually called a "hard drive"). Although Figure 4 Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The storage device 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.

[0159] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in the storage device 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 42 generally perform the functions and / or methods of the embodiments described herein.

[0160] The device 12 may also communicate with one or more external devices 14 (e.g., a keyboard, a pointing device, a display 24, etc.), one or more devices that enable a user to interact with the device 12, and / or any device that enables the device 12 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.). Such communication may be performed through an input / output (I / O) interface 22. Furthermore, the device 12 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 20. Figure 4 As shown, the network adapter 20 communicates with the other modules of the device 12 via the bus 18. It should be understood that although Figure 4 Not shown, other hardware and / or software modules may be used in conjunction with device 12, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0161] The processing unit 16 executes various functional applications and data processing by running the programs stored in the system storage device 28, for example, implementing the water area image data annotation method provided in the embodiment of the present invention, including:

[0162] Based on the characteristics of the target category to be labeled, determining candidate labeling results in the water area image to be labeled; wherein the candidate labeling results include the target category and / or target attribute information of the candidate labeling;

[0163] Receiving a user's judgment instruction on the candidate annotation results, and determining a final annotation result according to the judgment instruction; wherein the final annotation result includes the final annotated target category and / or target attribute information;

[0164] According to the final labeling result, a labeling result of the water area image to be labeled is determined.

[0165] Embodiment 5

[0166] Embodiment 5 of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method for labeling water area image data provided in the embodiment of the present invention is implemented, including:

[0167] Based on the characteristics of the target category to be labeled, determining candidate labeling results in the water area image to be labeled; wherein the candidate labeling results include the target category and / or target attribute information of the candidate labeling;

[0168] Receiving a user's judgment instruction on the candidate annotation results, and determining a final annotation result according to the judgment instruction; wherein the final annotation result includes the final annotated target category and / or target attribute information;

[0169] According to the final labeling result, a labeling result of the water area image to be labeled is determined.

[0170] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, a device or a device or used in combination with it.

[0171] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0172] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0173] Computer program code for performing the operation of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0174] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A method for labeling water area image data, characterized in that: include: Based on the characteristics of the target category to be labeled, determine the candidate labeling results in the water area image to be labeled; wherein the candidate labeling results include the target category and / or target attribute information of the candidate labeling; the target category and target attribute information to be labeled are determined by presetting, and the target attribute information is associated with the target category; the target category to be labeled includes at least one of the following: ships, navigable waters, coastlines, underwater obstacles, irrelevant objects and the ship itself; the target attribute information of the ship includes at least one of the following: boundary information, occlusion ratio and truncation ratio, and the attribute information of the navigable waters, coastlines, underwater obstacles, irrelevant objects and the ship itself includes at least boundary information; The ship refers to a vessel other than the own ship located on the water image; the navigable water area refers to an area on the water image where the ship can navigate; the coastline is used to represent the part of the navigable area that is connected to the land or the sky; the underwater obstacle refers to an obstacle located in the navigable water area that affects the navigation of the ship; the irrelevant object refers to an object located in the navigable water area that does not affect the navigation of the ship or an unrecognizable object; the own ship refers to the ship where the device for taking a photo of the water image is located; receiving a user's judgment instruction on the candidate annotation result, and determining a final annotation result according to the judgment instruction; wherein the final annotation result includes the final annotated target category and / or target attribute information; Determining a labeling result of the water area image to be labeled according to the final labeling result; The step of determining candidate annotation results in the water area image to be annotated based on the features of the target category to be annotated includes: Based on the characteristics of the target category to be labeled, determining candidate labeling results in the water area image to be labeled by image recognition technology; wherein the image recognition technology includes edge extraction; The receiving of a user's judgment instruction on the candidate annotation result includes: A user's judgment instruction on the candidate annotation result according to a preset annotation standard is received; wherein the preset annotation standard is used to establish a unified standard for data annotation of water area images.

2. The method according to claim 1, characterized in that The occlusion ratio refers to the ratio of the portion of the ship that is occluded by other objects in the water area image; The truncation ratio refers to the ratio of the portion of the ship that is truncated by the edge of the water area image.

3. The method according to claim 1, characterized in that The drivable waters include the coastline.

4. The method according to claim 1, characterized in that: The user's judgment instruction on the candidate annotation result includes at least one of the following: A user's instruction to determine the target category and / or target attribute information of the candidate annotation; A user's negative instruction on the target category and / or target attribute information of the candidate annotation; The user's instruction to modify the target category and / or target attribute information of the candidate annotation.

5. The method according to claim 4, characterized in that Receiving a user's judgment instruction on the candidate annotation result, and determining a final annotation result according to the judgment instruction, including: If a user's instruction to determine the candidate labeled target category and / or target attribute information is received, determining the candidate labeled target category and / or target attribute information associated with the instruction to determine as the final labeled target category and / or target attribute information; If the user issues a negative instruction for the target category and / or target attribute information of the candidate annotation, deleting the target category and / or target attribute information of the candidate annotation associated with the negative instruction; If the user has a modification instruction for the target category and / or target attribute information of the candidate annotation, the associated target category and / or target attribute information of the candidate annotation is modified according to the modification instruction, and the modified target category and / or target attribute information of the candidate annotation is determined as the final labeled target category and / or target attribute information.

6. The method according to claim 1, characterized in that If the target category of any object to be labeled in the water image does not belong to any of ships, drivable waters, coastlines, irrelevant objects and the ship itself, the target category of the object to be labeled is determined to be an underwater obstacle to avoid missing the object to be labeled in the water image.

7. A water area image data annotation device, characterized in that: include: A candidate result determination module is used to determine the candidate annotation results in the water area image to be annotated based on the characteristics of the target category to be annotated; wherein the candidate annotation results include the target category and / or target attribute information of the candidate annotation; the target category and target attribute information to be annotated are determined by presetting, and the target attribute information is associated with the target category; the target category to be annotated includes at least one of the following: ship, navigable water area, coastline, underwater obstacle, irrelevant object and own ship; the target attribute information of the ship includes at least one of the following: boundary information, occlusion ratio and truncation ratio, and the attribute information of the navigable water area, coastline, underwater obstacle, irrelevant object and own ship includes at least boundary information; the ship represents a ship other than the own ship located on the water area image; the navigable water area represents an area where the ship can travel on the water area image; the coastline is used to represent the part of the navigable area connected to the land or the sky; the underwater obstacle represents an obstacle located in the navigable water area that affects the travel of the ship; the irrelevant object represents an object located in the navigable water area that does not affect the travel of the ship or an unrecognizable object; the own ship refers to the ship where the device for taking pictures of the water area image is located; A final result determination module is used to receive a user's judgment instruction on the candidate annotation results, and determine the final annotation result according to the judgment instruction; wherein the final annotation result includes the final annotation target category and / or target attribute information; A water area image annotation module, used to determine the annotation result of the water area image to be annotated according to the final annotation result; The candidate result determination module is specifically used to: determine the candidate annotation results in the water area image to be annotated by image recognition technology based on the characteristics of the target category to be annotated; wherein the image recognition technology includes edge extraction; The final result determination module is specifically used to: receive a user's judgment instruction on the candidate annotation result according to a preset annotation standard; wherein the preset annotation standard is used to establish a unified standard for data annotation of water area images.

8. A device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the water area image data labeling method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the water area image data labeling method as described in any one of claims 1-6 is implemented.

Images