Picture labeling method and device, equipment and storage medium

By using different styles of canvas area labeling and displaying the overlay results between objects in the picture, the extraction inaccuracy problem caused by complex object location is solved, and the accuracy of selection and modification of target objects is improved and the user experience is improved.

CN118915947APending Publication Date: 2024-11-08BEIJING CHENGSHI WANGLIN INFORMATION TECH CO LTD
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411187135.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-27
Publication Date
2024-11-08

Smart Images

  • Figure CN118915947A_ABST
    Figure CN118915947A_ABST
Patent Text Reader

Abstract

The embodiment of the invention provides a picture labeling method and device, equipment and a storage medium, and the method comprises the steps: determining an object selected by a user according to a labeling style of an object selection operation in a canvas region to which a trigger position in a first canvas belongs; and if the user selects the second object, displaying a target area corresponding to a second canvas area in the target picture in a processing result area of the target page, marking the second canvas area in a third canvas area of the second canvas, and displaying the target picture and the canvas in an operation area of the target page in an overlapping manner. Wherein the second canvas region is marked with the difference between the picture regions where the second object and the first object are located in the target picture, and the picture region where the first object is located is included in the picture region where the second object is located. Different objects can be distinguished by different labeling styles, so that the object selected by the user can be accurately determined according to the labeling style of the canvas area to which the triggering position belongs, and the object extraction accuracy is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of human-computer interaction technology, and in particular to a picture annotation method, device, equipment and storage medium. Background Art

[0002] With the rapid development of Internet technology, the interaction between users and browser pages has become more frequent and in-depth. For example, a browser page may display a picture containing multiple objects. When a user wants to modify a target object in the picture, he or she may first trigger a selection operation on the target object in the picture.

[0003] However, in practice, a picture may contain multiple objects with complex positional relationships, and the complex positional relationships between objects will affect the accurate selection of the target object. Based on the above description, how to improve the accuracy of extracting the target object in the picture becomes a technical problem that needs to be solved urgently. Summary of the invention

[0004] In view of this, embodiments of the present invention provide a method, apparatus, device and storage medium for labeling an image, so as to improve the accuracy of extracting a target object in an image.

[0005] In a first aspect, an embodiment of the present invention provides a method for labeling an image, comprising:

[0006] In response to an object selection operation triggered by a user using an interactive tool, a trigger position of the object selection operation in a first canvas is obtained, wherein the first canvas includes a first canvas area marked with a picture area where a first object in a target picture is located, and a second canvas area marked with a difference between picture areas where a second object in the target picture and the first object are located, the picture area where the first object is located is included in the picture area where the second object is located, and the first canvas area and the second canvas area have different marking styles;

[0007] Determining the object selected by the user according to the marking style of the canvas area to which the trigger position belongs;

[0008] If the user selects the second object, displaying the target area in the target image corresponding to the second canvas area in the processing result area of ​​the target page;

[0009] If the user selects the second object, the second canvas area is marked in the third canvas area of ​​the second canvas, and the target image, the first canvas and the second canvas are superimposed and displayed in the operation area of ​​the target page.

[0010] In a second aspect, an embodiment of the present invention provides a picture annotation device, including:

[0011] an acquisition module, configured to, in response to an object selection operation triggered by a user using an interactive tool, acquire a trigger position of the object selection operation in a first canvas, wherein the first canvas includes a first canvas area marked with a picture area where a first object in a target picture is located, and a second canvas area marked with a difference between picture areas where a second object in the target picture and the first object are located, the picture area where the first object is located is included in the picture area where the second object is located, and the first canvas area and the second canvas area have different annotation styles;

[0012] A determination module, used to determine the object selected by the user according to the annotation style of the canvas area to which the trigger position belongs;

[0013] a display module, configured to display a target area in the target image corresponding to the second canvas area in a processing result area of ​​the target page if the user selects the second object;

[0014] The marking module is used for marking the second canvas area in the third canvas area of ​​the second canvas if the user selects the second object, and the target image, the first canvas and the second canvas are superimposed and displayed in the operation area of ​​the target page.

[0015] In a third aspect, an embodiment of the present invention provides an electronic device, including a processor and a memory, wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions, when executed by the processor, implement the image annotation method in the first aspect. The electronic device may also include a communication interface for communicating with other devices or communication systems.

[0016] In a fourth aspect, an embodiment of the present invention provides a non-temporary machine-readable storage medium having executable code stored thereon. When the executable code is executed by a processor of an electronic device, the processor can at least implement the image annotation method as described in the first aspect above.

[0017] In the picture annotation method provided by the embodiment of the present invention, in response to the object selection operation triggered by the user using the interactive tool, the trigger position of the object selection operation in the first canvas is obtained. Among them, the first canvas includes a first canvas area annotated with the picture area where the first object in the target picture is located, and a second canvas area annotated with the difference between the picture areas where the second object and the first object are located in the target picture, the picture area where the first object is located is included in the picture area where the second object is located, and the annotation styles of the first canvas area and the second canvas area are different. Then, according to the annotation style of the canvas area to which the trigger position belongs, the object selected by the user is determined. If the user selects the second object, the target area corresponding to the second canvas area in the target picture is displayed in the processing result area of ​​the target page. If the user selects the second object, the second canvas area is annotated in the third canvas area of ​​the second canvas, and the target picture, the first canvas and the second canvas are superimposed and displayed in the operation area of ​​the target page.

[0018] It can be seen that, in the case where the image area where the first object in the target image is located overlaps with the image area where the second object is located, the solution can determine the annotation style of the canvas area to which the trigger position belongs according to the trigger position of the object selection operation in the first canvas. Since the annotation styles of different canvas areas in the first canvas are different, and different annotation styles can be used to distinguish different objects in the target image, therefore, according to the determined annotation style of the canvas area to which the trigger position belongs, the target object selected by the user can be accurately determined, that is, the accuracy of target object extraction in the target image can be improved.

[0019] Furthermore, by displaying the overlay result between the target image and the canvas in the operation area of ​​the target page, and displaying the target area in the target image corresponding to the second canvas area in the processing result area of ​​the target page, the user can more intuitively see the result of the triggering operation, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1 A schematic diagram of a selected area provided by an embodiment of the present invention;

[0022] Figure 2 A schematic diagram of another selected area provided by an embodiment of the present invention;

[0023] Figure 3A flowchart of a method for labeling an image provided by an embodiment of the present invention;

[0024] Figure 4 A schematic diagram of the structure of a target page provided by an embodiment of the present invention;

[0025] Figure 5 A flowchart of another image annotation method provided by an embodiment of the present invention;

[0026] Figure 6 A schematic diagram of a first canvas provided by an embodiment of the present invention;

[0027] Figure 7 A schematic diagram of a second canvas provided by an embodiment of the present invention;

[0028] Figure 8 A flow chart of a method for determining a target area provided by an embodiment of the present invention;

[0029] Fig. 9 A schematic diagram of a moving process of a marking result provided by an embodiment of the present invention;

[0030] Fig.10 A schematic diagram of a scaling process of a labeling result provided by an embodiment of the present invention;

[0031] Fig.11 A schematic diagram of a picture annotation process in an artificial intelligence picture modification scenario provided by an embodiment of the present invention;

[0032] Fig.12 A schematic diagram of the structure of a picture annotation device provided by an embodiment of the present invention;

[0033] Fig.13 For Fig.12 A schematic diagram of the structure of an electronic device corresponding to the image annotation device provided in the illustrated embodiment. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0035] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings, and "multiple" generally includes at least two, but does not exclude the inclusion of at least one.

[0036] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0037] As used herein, the words "if" and "if" may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to identifying", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is identified" may be interpreted as "when determining" or "in response to determining" or "when identifying (stated condition or event)" or "in response to identifying (stated condition or event)", depending on the context.

[0038] It should be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a product or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such product or system. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the product or system including the elements.

[0039] It should also be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the present invention are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0040] Before describing in detail the image annotation methods provided by the embodiments of the present invention, an application scenario of image annotation may be schematically described:

[0041] As described in the background technology, a picture containing multiple objects may be displayed in a page on the browser side. For example, suppose the picture includes multiple objects, such as a face, a tree, a bird, etc. Among them, the bird may be located on the tree, that is, the picture area where the bird is located is included in the picture area where the tree is located. Then, when the user wants to modify the tree in the picture, he can first trigger a selection operation on the tree.

[0042] However, since the image area where the bird is located is included in the image area where the tree is located, when the user triggers a selection operation on the tree in the image, the selection result of the tree in the image can be as follows: Figure 1 As shown in the figure, the shaded part is the image area where the tree selected by the user is located. In this image area, not only the tree but also the bird on the tree is included. Therefore, for extracting the image area where the tree is located, the image area where the bird on the tree is located will become interference, thereby reducing the accuracy of extracting the tree in the image.

[0043] In order to improve the above problems, the image annotation methods provided by the following embodiments of the present invention can be used to obtain the following Figure 2 The selected result of the tree in the picture shown is that the shaded part is the picture area where the tree is located after removing the picture area where the bird located on the tree is located.

[0044] Based on the above description, some embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the case where there is no conflict between the embodiments, the following embodiments and the features in the embodiments can be combined with each other. In addition, the step sequence in the following method embodiments is only an example and not a strict limitation.

[0045] Figure 3 The flowchart of a method for annotating a picture provided by an embodiment of the present invention is shown in FIG. The method for annotating a picture provided by an embodiment of the present invention can be executed by an annotation device. It is understood that the annotation device can be implemented as software or a combination of software and hardware. Figure 3 As shown, the method may include the following steps:

[0046] S301, in response to an object selection operation triggered by a user using an interactive tool, obtaining a trigger position of the object selection operation in a first canvas, the first canvas including a first canvas area marked with a picture area where a first object in a target picture is located, and a second canvas area marked with a difference between a second object in the target picture and a picture area where the first object is located, the picture area where the first object is located is included in the picture area where the second object is located, and the marking styles of the first canvas area and the second canvas area are different.

[0047] S302: Determine the object selected by the user according to the marking style of the canvas area to which the trigger position belongs.

[0048] When the user has a need to modify an object in the target image, the user can use an interactive tool to trigger a selection operation on the object in the target image so that the annotation device can extract the object from the target image.

[0049] Specifically, the annotation device can obtain a trigger position of the object selection operation in the first canvas in response to the object selection operation triggered by the user using the interactive tool, and then determine the object selected by the user according to the annotation style of the canvas area to which the trigger position belongs.

[0050] Optionally, the target image may include a first object and a second object. Assume that the first object coincides with the second object, that is, the image area where the first object is located can be included in the image area where the second object is located. Then the annotation device can mark the image area where the first object is located on the first canvas to obtain a first canvas area marked with the image area where the first object is located in the target image. Since the image area where the first object is located coincides with the image area where the second object is located, that is, the actual image area where the second object is located is the difference between the image areas where the second object and the first object are respectively located, the annotation device can also mark the difference between the image areas where the second object and the first object are respectively located on the first canvas to obtain a second canvas area marked with the difference between the image areas where the second object and the first object are respectively located in the target image. Among them, the annotation styles of the first canvas area and the second canvas area are different.

[0051] Following the above Figure 2 For example, the first object may be a bird on a tree, and the second object may be a tree. Accordingly, the first canvas area may display the marking traces of the picture area where the bird is located on the first canvas, and the second canvas area may display the marking traces of the picture area where the tree is located after the bird is removed on the first canvas.

[0052] Regarding the determination of the trigger position of the object selection operation in the first canvas, optionally, the annotation device may first obtain the trigger position of the object selection operation in the target page. The size of the first canvas may be smaller than the size of the target page. Then, the annotation device may calculate the offset between the vertex coordinates of the target vertices in the same direction of the target page and the first canvas. Optionally, the target vertex may be the upper left corner of the target page and the first canvas. Afterwards, the annotation device may subtract the trigger position of the object selection operation in the target page from the offset to obtain the trigger position of the object selection operation in the first canvas.

[0053] Optionally, the interactive tool may be a tool capable of realizing interaction, such as a mouse, a stylus pen, a finger, etc. Optionally, the size of the first canvas may be the same as the size of the target image, or the size of the first canvas may be different from the size of the target image.

[0054] S303: If the user selects the second object, the target area corresponding to the second canvas area in the target image is displayed in the processing result area of ​​the target page.

[0055] S304: If the user selects the second object, the second canvas area is marked in the third canvas area of ​​the second canvas, and the target image, the first canvas and the second canvas are superimposed and displayed in the operation area of ​​the target page.

[0056] In order to enable users to more intuitively see the results of their triggering operations, optionally, Figure 4 As shown, the target page can be set with an operation area and a processing result area. The operation area is used to display the superposition result between the target image and the canvas. The processing result area is used to display the target area of ​​the target image corresponding to the object selection operation.

[0057] In an optional case, if the user selects the second object, the annotation device can display the target area corresponding to the second canvas area in the target image in the processing result area of ​​the target page. Figure 2 In the example of the embodiment shown, the target area can be Figure 2 The shadow area in the image is the area of ​​the tree without the bird. After obtaining the target area in the target image, the annotation device can display the target area in Figure 4 The processing result area of ​​the target page is shown. Figure 4 The shaded area in the figure is the target area.

[0058] In order to further improve the user experience and let the user know the scope of the object selection operation triggered by the user, if the user selects the second object, then optionally, the marking device can also mark the second canvas area in the third canvas area of ​​the second canvas. Figure 4 The operation area of ​​the target page shown may display the superposition result between the target image, the first canvas and the second canvas.

[0059] The size of the first canvas may be the same as the size of the second canvas. The marking traces displayed on the third canvas area are different from the marking traces displayed on the first canvas area and the second canvas area. For details, please refer to the description in the following embodiment.

[0060] Optionally, the transparency of the first canvas and the transparency of the second canvas may be different. Specifically, the transparency of the first canvas may be set to 0, and the transparency of the second canvas may be set to 1.

[0061] In the picture annotation method provided by the embodiment of the present invention, in response to the object selection operation triggered by the user using the interactive tool, the trigger position of the object selection operation in the first canvas is obtained. Wherein, the first canvas includes a first canvas area annotated with the picture area where the first object in the target picture is located, and a second canvas area annotated with the difference between the picture areas where the second object and the first object are located in the target picture, the picture area where the first object is located is included in the picture area where the second object is located, and the annotation styles of the first canvas area and the second canvas area are different. Then, according to the annotation style of the canvas area to which the trigger position belongs, the object selected by the user is determined. If the user selects the second object, the target picture area corresponding to the second canvas area in the target picture is displayed in the processing result area of ​​the target page. If the user selects the second object, the second canvas area is annotated in the third canvas area of ​​the second canvas. Wherein, the target picture, the first canvas and the second canvas are superimposed and displayed in the operation area of ​​the target page.

[0062] It can be seen that, in the case where the image area where the first object in the target image is located overlaps with the image area where the second object is located, the solution can determine the annotation style of the canvas area to which the trigger position belongs according to the trigger position of the object selection operation in the first canvas. Since the annotation styles of different canvas areas in the first canvas are different, and different annotation styles can be used to distinguish different objects in the target image, therefore, according to the determined annotation style of the canvas area to which the trigger position belongs, the target object selected by the user can be accurately determined, that is, the accuracy of target object extraction in the target image can be improved.

[0063] Furthermore, by displaying the overlay result between the target image and the canvas in the operation area of ​​the target page, and displaying the target image area in the target image corresponding to the second canvas area in the processing result area of ​​the target page, the user can more intuitively see the result of the triggering operation, thereby improving the user experience.

[0064] In addition, Figure 3 The illustrated embodiment shows one situation in steps S303 to S305. Another situation may be: if the user selects the first object, the annotation device may display the target area in the target image corresponding to the first canvas area in the processing result area of ​​the target page. Continuing with the above example, in this case, the target area is the image area where the bird on the tree is located. And if the user selects the first object, the annotation device may also mark the image area where the first object is located in the fourth canvas area of ​​the second canvas. Afterwards, the annotation device may overlay the target image, the first canvas, and the second canvas and display them in the processing result area. Figure 4 in the operation area of ​​the target page shown.

[0065] In practice, in order to ensure the display effect of the target image in the target page, the size of the target image can be adjusted by the size of the original image. Optionally, the size of the original image can be larger than the size of the target image, or smaller than the size of the target image.

[0066] For determining the size of the target image, optionally, the annotation device may first obtain the aspect ratio of the operation area and the aspect ratio of the original image. Then, the annotation device may adjust the size of the original image according to the comparison result of the aspect ratios of the operation area and the original image to obtain a target image that matches the size of the operation area. Optionally, the size of the target image may be smaller than or equal to the size of the operation area.

[0067] Specifically, if the aspect ratio of the original image is smaller than the aspect ratio of the operation area, the annotation device can further determine the comparison result between the height of the original image and the height of the operation area. If the height of the original image is greater than the height of the operation area, the height of the original image can be adjusted so that the height of the adjusted target image is the same as the height of the operation area. At this time, the width of the target image is the product of the height of the target image and the aspect ratio of the original image.

[0068] If the aspect ratio of the original image is greater than or equal to the aspect ratio of the operation area, the annotation device can further determine the comparison result between the width of the original image and the width of the operation area. If the width of the original image is greater than the width of the operation area, the width of the original image can be adjusted so that the width of the adjusted target image is the same as the width of the operation area. At this time, the height of the target image is the division result between the width of the target image and the aspect ratio of the original image.

[0069] The above process of determining the size of the target image can be achieved by executing the following code:

[0070]

[0071] Among them, conAspect indicates the aspect ratio of the operation area, containerWidth is the width of the operation area, and containerHeight is the height of the operation area. aspect indicates the aspect ratio of the original image, naturalWidth is the width of the original image, and naturalHeight is the height of the original image. displayWidth is the width of the target image, and displayHeight is the height of the target image.

[0072] In this embodiment, by comparing the aspect ratio of the operation area with the aspect ratio of the original image, the size of the original image is adjusted to obtain a target image that matches the size of the operation area. Since the size of the target image matches the size of the operation area, the display effect of the target image in the target page can be improved.

[0073] Figure 5 A flowchart of another image annotation method provided by an embodiment of the present invention. Figure 5 The steps of the embodiment shown are: Figure 1 The specific process of step S302 in the embodiment shown in the figure is as follows. Figure 5 As shown, the method may include the following steps:

[0074] S501, determining a target identity corresponding to the color according to the color of the canvas area to which the trigger position belongs.

[0075] S502: For the object corresponding to the target identity, if the selected state of the object is marked as selected, the object corresponding to the target identity is determined as the object selected by the user.

[0076] In practice, any object in the target image may be associated with an array. Optionally, the array associated with any object may include an identity identifier and a selected state identifier corresponding to the object. The identity identifier is used to distinguish different objects, and different objects have different corresponding identity identifiers. The selected state identifier is used to indicate whether the object is currently selected by the user.

[0077] Optionally, the first canvas area and the second canvas area can be marked with different colors in the first canvas. And the color of the canvas area can correspond to the identity of the object in the target image, that is, different colors correspond to the identity of different objects. Based on this, the annotation device can determine the target identity corresponding to this color according to the color of the trigger position. Afterwards, for the object corresponding to the target identity, it is determined whether the selection state mark of this object is selected. If the selection state mark of the object is selected, the annotation device can determine the object corresponding to the target identity as the object selected by the user.

[0078] Optionally, the selected state identifier may include a state identifier in which the user clicks the object using an interactive tool or a state identifier in which the user hovers over the object using an interactive tool.

[0079] Optionally, the color of the canvas area may include colors represented by different intensities of red, green, and blue. The color of the canvas area may correspond to the identity of the object in the target image. The identity may be represented by an id, which may be a 24-bit binary number, wherein 0 to 8 bits of binary numbers correspond to blue, 8 to 16 bits of binary numbers correspond to green, and 16 to 24 bits of binary numbers correspond to red.

[0080] For example, when id=255, since 255 represents a binary number of 0 to 8 bits, the color corresponding to id can be blue. When id=255<<8, where << is a left shift operator, its function is to shift the bits of the left operand to the left, and the number of bits shifted is determined by the right operand, that is, 255 is shifted left by 8 bits, that is, 255<<8 represents a binary number of 8 to 16 bits, so the color corresponding to id can be green. When id=255<<16, that is, 255 is shifted left by 16 bits, that is, 255<<16 represents a binary number of 16 to 24 bits, so the color corresponding to id can be red. In addition, when id is a value close to 0, such as 3, it means that the color value is low, and it can be considered that there is no color at this time, and the color corresponding to id can be black. When id=(255<<8)+(255<<16), the corresponding color is a mixture of green and red, and the mixture is yellow.

[0081] When the color of the canvas area to which the trigger position belongs is blue, the id corresponding to blue can be determined, that is, id = 255. Assuming that 255 corresponds to the second object, the annotation device can further determine whether the selected state mark of the second object is selected. If the selected state mark of the second object is selected, the second object is determined as the object selected by the user.

[0082] In this embodiment, the object corresponding to the trigger position can be accurately determined by using the correspondence between the color of the canvas area and the identity mark. Further, by determining whether the selected state mark of the object is selected, the accuracy of determining the object selected by the user can be improved.

[0083] exist Figure 5 On the basis of the illustrated embodiment, further, in order to improve the accuracy of determining the target identity, optionally, the labeling device can determine the color corresponding to each element in the preset canvas area centered on the trigger position. If the colors of the elements in the preset canvas area are the same, the labeling device can determine the identity corresponding to the color as the target identity.

[0084] Specifically, the annotation device may first obtain the coordinates of the trigger position on the canvas, i.e., the center coordinates of the preset canvas area, and the width and height of the preset canvas area. The width and height of the preset canvas area are the same, for example, the preset canvas area may be a 3*3 canvas area centered on the trigger position, or may be a 5*5 canvas area centered on the trigger position, and so on.

[0085] Then, the annotation device can first determine the position of the element located in the upper left corner of the preset canvas area based on the center coordinates and the width and height of the preset canvas area. Then, based on the position of the element in the upper left corner, determine the color of the element in the upper left corner, and obtain the array associated with the color of the element in the upper left corner. Optionally, the color of any element in the preset canvas area can be associated with an array. The array can be represented by data, which includes four elements, namely data[0], data[1], data[2], and data[3]. Among them, data[0] corresponds to the value of the red channel, data[1] corresponds to the value of the green channel, data[2] corresponds to the value of the blue channel, and data[3] corresponds to the transparency value of the canvas.

[0086] For example, assume that the center coordinates can be represented by (c0, d0), the width and height of the preset canvas area can both be represented by sampleNum, and the value of sampleNum is 3. The position of the element located in the upper left corner of the preset canvas area can be represented by (c1, d1). Then, for determining c1, specifically, the annotation device can first calculate the difference between sampleNum and 1, and then divide the difference by 2 to obtain a first division result. Afterwards, c0 can be subtracted from the first division result to obtain c1. For determining d1, specifically, d0 can be subtracted from the first division result to obtain d1.

[0087] For a 3*3 canvas area, the total number of elements is 9. The annotation device can compare the color-associated arrays of the other 8 elements in the preset canvas area with the color-associated array of the first element in sequence. If the color-associated arrays corresponding to the 9 elements in the preset canvas area are the same, indicating that the colors corresponding to the 9 elements in the preset canvas area are the same, the annotation device can determine the identity corresponding to the color as the target identity. If the color-associated arrays corresponding to the 9 elements in the preset canvas area are different, the annotation device may not use the identity corresponding to this trigger position as the target identity.

[0088] In this embodiment, considering that the trigger position may be the edge of the outline of the object displayed in the first canvas, and the color of the edge point may be a gradient color, the gradient color may correspond to multiple identity identifiers, which will obviously lead to inaccurate determination of the target identity identifier. Therefore, this scheme can accurately determine whether the trigger position is an edge point by judging the color of the area around the trigger position, thereby improving the accuracy of determining the target identity identifier.

[0089] Figure 3 In the illustrated embodiment, it has been mentioned that the first canvas includes a first canvas area and a second canvas area. For determining the first canvas area, specifically, the annotation device can determine the first position of the outline of the first object in the first canvas according to the coordinates of the outline of the first object in the target image and the first offset. Then, based on the first position, the first canvas area of ​​the image area where the first object in the target image is located is determined. The first offset is the offset between the vertex coordinates of the target vertices in the first canvas and the target image in the same direction.

[0090] Specifically, for determining the first position, the annotation device may add the coordinates of the outline of the first object in the target image to the first offset, and determine the addition result as the first position of the outline of the first object in the first canvas. Optionally, the size of the target image may be less than or equal to the size of the first canvas, or the size of the target image may be greater than the size of the first canvas.

[0091] For determining the coordinates of the contour of the first object in the target image, the annotation device may optionally first obtain the coordinates of the contour of the first object in the original image. Then, the annotation device may divide the coordinates of the contour of the first object in the original image by the size of the original image to obtain a second division result, and multiply the second division result by the size of the target image to obtain the coordinates of the contour of the first object in the target image.

[0092] Optionally, the target vertex may include the upper left corner of the first canvas and the target image. The first offset may include the horizontal offset of the coordinates of the upper left corners of the first canvas and the target image, and the vertical offset of the coordinates of the upper left corners of the first canvas and the target image. The horizontal offset may be represented by left1, and the value of left1 may be half of the difference between the width of the first canvas and the width of the target image. The vertical offset may be represented by top1, and the value of top1 may be half of the difference between the height of the first canvas and the height of the target image.

[0093] The above-mentioned process of determining the first position can also be implemented by executing the following code:

[0094] x1=point[0] / naturalWidth*displayWidth+left1

[0095] y1=point[1] / naturalHeight*displayHeight+top1

[0096] Among them, (x1, y1) represents the coordinates of the first position, and (point[0], point[1]) represents the coordinates of the outline of the first object in the original image.

[0097] Similar to the determination process of the first canvas area, for the determination of the second canvas area, specifically, the annotation device can add the coordinates of the outline of the second object in the target image to the first offset, and determine the addition result as the second position of the outline of the second object in the first canvas. Then, the annotation device can determine the second canvas area annotated with the difference between the second object in the target image and the image area where the second object is located, based on the difference between the second position and the first position. Among them, the determination process of the coordinates of the outline of the second object in the target image can refer to the determination process of the coordinates of the outline of the first object in the target image, which will not be repeated here.

[0098] In this embodiment, for the case where the image regions of multiple objects in the target image overlap, the solution can accurately calculate the positions of the multiple objects with overlapping regions in the canvas. Further, based on this position, the annotation device can accurately extract the canvas regions corresponding to the multiple objects with overlapping regions.

[0099] In the above embodiment, it has been mentioned that the first canvas area and the second canvas area in the first canvas have different marking styles, and the first canvas area and the second canvas area are marked with different colors in the first canvas, that is, the marking style includes filling the canvas area with color.

[0100] For the annotation of the first canvas area and the second canvas area, specifically, the annotation device can determine the colors corresponding to the first object and the second object according to the identity of the object in the target image, wherein the color corresponding to the first object is different from the color corresponding to the second object.

[0101] Then, the labeling device can fill the image area where the first object is located in the target image according to the color corresponding to the first object to obtain the first canvas area. The labeling device can also fill the image area where the second object is located in the target image according to the color corresponding to the second object to obtain the second canvas area.

[0102] Based on the above description, the schematic diagram of the first canvas can be combined with Figure 6 Understand. Figure 6 As shown in the figure, the black area represents the second canvas area, and the light gray area represents the first canvas area. In addition, it should be noted that Figure 6 The black and light gray in the figure are only a schematic representation of color distinction and do not represent the actual color of the object. For example, in practice, the first canvas area may be a yellow area and the second canvas area may be a red area.

[0103] In this embodiment, the colors corresponding to the objects in the target image can be accurately determined according to the identity of the objects in the target image. Further, the marking device can better mark the image areas corresponding to the objects on the first canvas according to the colors corresponding to the objects.

[0104] When the user selects the second object, optionally, the marking device may also mark the second canvas area in a third canvas area of ​​the second canvas.

[0105] Specifically, the annotation device may first obtain the annotation style of the second canvas. The annotation style of the second canvas includes a stroke result of the outline of the object in the target image and a fill result inside the outline of the object in the target image. Optionally, the stroke result may include a stroke trace in a dotted style. The fill result may include a black trace with a transparency of 0.5.

[0106] Afterwards, the marking device marks the second canvas area on the third canvas area of ​​the second canvas according to the marking style of the second canvas, wherein the marking styles of the third canvas area, the second canvas area and the first canvas area are different.

[0107] Following the above Figure 3 In the example of step S302 in the illustrated embodiment, it is assumed that the second object is a tree and the first object is a bird located on the tree. Then the schematic diagram of the second canvas can be as follows: Figure 7 As shown. Figure 7 As shown, the second canvas area is the image area where the tree is located except for the bird, and the second canvas area is presented in black with a transparency of 0.5. Figure 7 The outlines of the trees and birds are rendered as dashed lines.

[0108] In this embodiment, since the annotation style of the second canvas includes the stroke result of the outline of the object in the target image and the fill result inside the outline of the object in the target image, the stroke result and the fill result can clearly reflect the boundary of the difference canvas area. Therefore, this scheme can better mark the second canvas area on the second canvas according to the annotation style of the second canvas.

[0109] Figure 1In the illustrated embodiment, it has been mentioned that the target area in the target image corresponding to the difference canvas area can be displayed in the processing result area of ​​the target page.

[0110] In practice, considering that the contours of the objects in the target image may be complex, this will reduce the accuracy of object extraction in the target image, and thus lead to inaccurate final determination of the target area. Figure 8 The method of the illustrated embodiment determines a target area displayed in a processing result area. Figure 8 Flow chart of a method for determining a target area provided by an embodiment of the present invention. Figure 8 As shown, the method may include the following steps:

[0111] S801: If the user selects the second object, the outlines of the first object and the second object in the target image are drawn in a third canvas.

[0112] If the user selects the second object, the annotation device may draw the contours of the first object and the second object in the target image in the third canvas according to the coordinates of the contours of the first object and the second object in the third canvas.

[0113] For the coordinates of the outline of the second object in the third canvas, specifically, the annotation device can determine the coordinates of the minimum rectangular area in the target picture according to the coordinates of the minimum rectangular area in the original picture and the sizes of the original picture and the target picture. The coordinates of the minimum rectangular area in the target picture may include (minX, minY), (maxX, minY), (maxX, maxY), (minX, maxY). minX and maxX respectively represent the minimum coordinate value and the maximum coordinate value of the outline of the second object in the horizontal direction in the target picture, and minY and maxY respectively represent the minimum coordinate value and the maximum coordinate value of the outline of the second object in the vertical direction in the target picture.

[0114] Then, the annotation device may subtract the coordinates of the minimum rectangular area in the target image from (minX, minY), and then multiply the subtraction result by the scaling factor to obtain the coordinates of the outline of the second object in the third canvas.

[0115] The process of determining the coordinates of the outline of the second object in the third canvas can be implemented by executing the following code:

[0116] x3=(x2-minX)*scale

[0117] y3=(y2-minY)*scale

[0118] Wherein, (x3, y3) represents the coordinates of the outline of the second object in the third canvas, (x2, y2) represents the coordinates of the outline of the second object in the target image, and scale represents the scaling factor.

[0119] For determining the scaling factor, specifically, the annotation device may first calculate a first difference between maxX and minX, and a second difference between maxY and minY. Then, the width or height of the processing result area is divided by the first difference to obtain a third division result. The width or height of the processing result area is also divided by the second difference to obtain a fourth division result. Optionally, the width and height of the processing result area may be the same. The width or height of the processing result area may be represented by size. Afterwards, the annotation device may determine the minimum division result of the third division result and the fourth division result, and the minimum division result is the scaling factor.

[0120] For determining the size of the third canvas, one case may be: if the scaling factor is less than 1, indicating that the image area corresponding to the minimum rectangular area in the target image is large and cannot be fully displayed in the processing result area, the annotation device can further determine the size relationship between the third division result and the fourth division result. If the third division result is less than the fourth division result, size can be directly determined as the width of the third canvas. And the division result between size and the first difference and the second difference can be multiplied to obtain the height of the third canvas. If the third division result is greater than the fourth division result, size can be directly determined as the height of the third canvas. And the division result between size and the second difference and the first difference can be multiplied to obtain the width of the third canvas.

[0121] Another case may be: if the zoom factor is greater than 1, it indicates that the image area corresponding to the minimum rectangular area in the target image is small and can be completely displayed in the processing result area, then the annotation device can directly determine the first difference as the width of the third canvas and the second difference as the height of the third canvas. At this time, the zoom factor can be reset to 1.

[0122] S802: Draw the image area corresponding to the minimum rectangular area in the target image on the fourth canvas.

[0123] S803, adjusting the transparency of the corresponding canvas area in the fourth canvas according to the respective transparency of different canvas areas in the third canvas, so as to obtain a target area displayed in the processing result area of ​​the target page, wherein the third canvas and the fourth canvas respectively include a difference canvas area between the outline of the second object and the outline of the first object and a remaining canvas area, the transparency of the difference canvas area in the third canvas is 1, and the transparency of the remaining canvas area in the third canvas is 0.

[0124] Then, the marking device may draw the image area corresponding to the minimum rectangular area in the target image on the fourth canvas. At this time, the image areas where the second object and the first object in the target image are located are displayed on the fourth canvas.

[0125] Afterwards, the annotation device can adjust the transparency of the corresponding canvas area in the fourth canvas according to the respective transparency of different canvas areas in the third canvas, so as to obtain the target area displayed in the processing result area of ​​the target page. Among them, the third canvas and the fourth canvas respectively include the difference canvas area between the outline of the second object and the outline of the first object and the remaining canvas area. The transparency of the difference canvas area in the third canvas is 1, and the transparency of the remaining canvas area in the third canvas is 0. That is, the annotation device can adjust the transparency of the difference canvas area in the fourth canvas to 1, and adjust the transparency of the remaining canvas area in the fourth canvas to 0. At this time, the fourth canvas displays the difference between the image areas where the second object and the first object in the picture are respectively located. The display effect of the processing result area of ​​the target page can be combined with the above Figure 4 understand.

[0126] In this embodiment, taking into account the situation that the outline of the object in the target image is relatively complex, the solution can accurately determine the target area of ​​the target image by drawing the canvas and setting the transparency of the canvas, thereby improving the accuracy of the display of the target area.

[0127] After obtaining the annotation result obtained by superimposing the target image, the first canvas, and the second canvas, the user may optionally use an interactive tool to move the target image and the annotation trace in the annotation result respectively.

[0128] For the movement of the target image in the annotation result, specifically, the annotation device can obtain the position coordinates of the interactive tool in the target page before the movement and the position coordinates of the interactive tool in the target page after the movement in response to the movement of the interactive tool, and the position of the annotation result in the target page moves with the movement of the interactive tool. Then, a second offset between the position coordinates of the interactive tool before and after the movement is calculated, and according to the second offset, a third position of the movement result of the target image in the annotation result in the canvas is determined.

[0129] The canvas includes a first canvas and a second canvas. The second offset can be represented by (left2, top2), and the second offset can include a horizontal offset of left2 between the position coordinates of the interactive tool before the movement and the position coordinates of the interactive tool after the movement, and a vertical offset of top2 between the position coordinates of the interactive tool before the movement and the position coordinates of the interactive tool after the movement.

[0130] For the movement of the annotation trace in the annotation result, specifically, the annotation device can determine the fourth position of the movement result of the annotation trace in the annotation result in the canvas according to the position coordinates of the annotation trace mapped to the original picture, the sizes of the target picture and the original picture, and the third position. The annotation trace includes a first annotation trace and a second annotation trace, and the first annotation trace corresponds to the first canvas, and the second annotation trace corresponds to the second canvas.

[0131] Finally, according to the third position and the fourth position, the annotation device can overlay and render the movement result of the annotation trace on the movement result of the target image to obtain the annotation result after the movement, that is, to complete the movement of the annotation result. Fig. 9 Understand. Fig. 9 As shown, the circles represent the position coordinates of the interactive tool in the target page before and after the move.

[0132] For determining the fourth position, specifically, the annotation device may perform a division operation on the position coordinates of any annotation trace mapped to the original image and the size of the original image to obtain a second division result. Then, the second division result is multiplied by the size of the target image to obtain a first multiplication result. Then, the first multiplication result is added to the third position to obtain the fourth position.

[0133] The process of determining the fourth position can be implemented by executing the following code:

[0134] x4=a / naturalWidth*displayWidth+x3;

[0135] y4=b / naturalHeight*displayHeight+y3;

[0136] Among them, (x3, y3) represents the third position, (x4, y4) represents the fourth position, and (a, b) represents the position coordinates of the annotation trace mapped to the original image.

[0137] In this embodiment, by using an interactive tool to move the target image and the annotation trace in the annotation result, the annotation result after the move can be better displayed in the operation area of ​​the target page.

[0138] After obtaining the annotation results, the user may optionally use an interactive tool to zoom in and out the target image and the annotation trace in the annotation results.

[0139] For the scaling of the target image in the annotation result, specifically, the annotation device can respond to the scaling operation triggered on the annotation result, and use the current position of the interactive tool in the canvas as the scaling center. Then, calculate the third offset between the scaling center and any vertex coordinate of the canvas. Then, multiply the difference between the third offset and the first offset by the scaling factor corresponding to the scaling operation to obtain a second multiplication result. After that, determine the fifth position of the scaling result of the target image in the annotation result in the canvas according to the difference between the third offset and the second multiplication result.

[0140] The third offset can be represented by (left3, top3). Optionally, any vertex can be the upper left corner of the zoom center and the canvas. Then the third offset can include the horizontal offset of the coordinates of the zoom center and the upper left corner of the canvas, left3, and the vertical offset of the coordinates of the zoom center and the upper left corner of the canvas, top3.

[0141] Regarding the scaling of the annotation trace in the annotation result, specifically, the annotation device can determine the sixth position of the scaling result of the annotation trace in the annotation result in the canvas according to the position coordinates of the annotation trace mapped to the original picture, the size of the scaling result, the size of the original picture, and the fifth position. The annotation trace includes a first annotation trace and a second annotation trace, the first annotation trace corresponds to the first canvas, and the second annotation trace corresponds to the second canvas. The process of determining the position coordinates of the annotation trace mapped to the original picture can be referred to. Figure 8 Description of the illustrated embodiment.

[0142] Finally, according to the fifth position and the sixth position, the annotation device can overlay and render the zoom result of the annotation trace on the zoom result of the target image to obtain the zoomed annotation result, that is, to complete the zooming of the annotation result. Fig.10 Understand. Fig.10 As shown, the five-pointed star indicates the position of the zoom center.

[0143] The determination process of the fifth position can also be obtained by executing the following code:

[0144] x5=left3-(left3-left1)*dollyScale

[0145] y5=top3-(top3-top1)*dollyScale

[0146] Wherein, (x5, y5) represents the fifth position, and dollyScale represents the scaling factor. Optionally, dollyScale may be greater than 1 or less than 1. If dollyScale is greater than 1, it can be determined that a zoom-in operation is performed on the target image. If dollyScale is less than 1, it can be determined that a zoom-out operation is performed on the target image.

[0147] For determining the sixth position, specifically, the annotation device may perform a division operation on the position coordinates of the annotation trace mapped to the original image and the size of the original image to obtain a third division result. Then, the third division result and the size of the scaling result are multiplied to obtain a third multiplication result. Then, the third multiplication result and the fifth position are added to obtain the sixth position.

[0148] The above-mentioned determination process of the sixth position can be implemented by executing the following code:

[0149] x6=a / naturalWidth*zoomWidth+x5;

[0150] y6=b / naturalHeight*zoomHeight+y5;

[0151] Wherein, (x6, y6) indicates the sixth position. zoomWidth is the width of the zoom result of the target image, and zoomHeight is the height of the zoom result of the target image. (zoomWidth, zoomHeight) indicates the size of the zoom result of the target image.

[0152] In this embodiment, by using an interactive tool to perform a zooming operation on the target image and the annotation trace in the annotation result, the zoomed annotation result can be better displayed in the operation area of ​​the target page.

[0153] For ease of understanding, the specific implementation process of the above-provided image annotation method can be illustrated in combination with the artificial intelligence image modification scenario, as follows:

[0154] In the AI ​​image modification scenario, Fig.11 As shown, the target image may include a human face, a tree, and a bird on the tree. When the user wants to modify the tree, he or she may use the interactive tool to trigger a selection operation on the tree. The annotation device may respond to the selection operation triggered by the user using the interactive tool, obtain the trigger position of the selection operation in the first canvas, and Fig.11 The black area in the first canvas shown indicates that the object selected by the user is a tree.

[0155] Then, the marking device can determine the difference canvas area between the black area and the light gray area in the first canvas, that is, the second canvas area mentioned in the above embodiments. Then, the marking device can mark the second canvas area in Fig.11 In the third canvas area of ​​the second canvas shown. The black area and light gray area shown in the first canvas are only for illustration of color distinction. In fact, the colors of the canvas area may include red, green, blue, etc. The second canvas area is the annotation result of the picture area where the tree without the bird is located on the first canvas. The third canvas area shows the dotted line stroke result of the outline of the second canvas area and the filling result inside the outline. The filling result is presented as black with a transparency of 0.5.

[0156] In order to improve the user's interactive experience with the target page, the annotation device can overlay the target image, the first canvas, and the second canvas and display them on the Fig.11 The target area corresponding to the second canvas area in the target image can also be displayed in the processing result area of ​​the target page. The shaded part in the processing result area is the target area, that is, the image area where the tree actually selected by the user is located.

[0157] In addition, the contents not described in detail in this embodiment and the technical effects that can be achieved can all be found in the relevant descriptions in the above embodiments, and will not be repeated here.

[0158] The following will describe in detail one or more embodiments of the image annotation device of the present invention. Those skilled in the art will appreciate that these image annotation devices can be configured using commercially available hardware components through the steps taught in this solution.

[0159] Fig.12 A schematic diagram of the structure of a picture annotation device provided by an embodiment of the present invention is shown in FIG. Fig.12 As shown, the device comprises:

[0160] The acquisition module 11 is used to respond to an object selection operation triggered by a user using an interactive tool, and obtain a trigger position of the object selection operation in a first canvas, wherein the first canvas includes a first canvas area marked with a picture area where a first object in a target picture is located, and a second canvas area marked with a difference between a second object in the target picture and a picture area where the first object is located, wherein the picture areas where the first objects are located are included in the picture area where the second object is located, and the first canvas area and the second canvas area have different marking styles.

[0161] The object determination module 12 is used to determine the object selected by the user according to the marking style of the canvas area to which the trigger position belongs.

[0162] The display module 13 is configured to display a target area in the target image corresponding to the second canvas area in the processing result area of ​​the target page if the user selects the second object.

[0163] The marking module 14 is used for marking the second canvas area in the third canvas area of ​​the second canvas if the user selects the second object, and the target image, the first canvas and the second canvas are superimposed and displayed in the operation area of ​​the target page.

[0164] Optionally, the device also includes: a size adjustment module 15, which is used to obtain the aspect ratio of the operation area and the aspect ratio of the original image; according to the comparison result of the aspect ratios of the operation area and the original image, adjust the size of the original image to obtain a target image that matches the size of the operation area.

[0165] Optionally, the first canvas area and the second canvas area are marked with different colors in the first canvas; the color of the canvas area corresponds to the identity of the object in the target image; based on this, the object determination module 12 is used to determine the target identity corresponding to the color according to the color of the canvas area to which the trigger position belongs; for the object corresponding to the target identity, if the selection state of the object is marked as selected, the object corresponding to the target identity is determined as the object selected by the user.

[0166] Optionally, the object determination module 12 is used to determine the color corresponding to each element in a preset canvas area centered on the trigger position; if the colors of the elements in the preset canvas area are the same, the identity corresponding to the color is determined as the target identity.

[0167] Optionally, the annotation style includes filling the canvas area with color. Based on this, the device also includes: a filling module 16, used to determine the colors corresponding to the first object and the second object respectively according to the identity identification of the object in the target image, the color corresponding to the first object is different from the color corresponding to the second object; according to the color corresponding to the first object, the image area where the first object is located in the target image is filled; according to the color corresponding to the second object, the image area where the second object is located in the target image is filled.

[0168] Optionally, the device also includes: a canvas area determination module 17, used to determine a first position of the outline of the first object in the first canvas according to the coordinates of the outline of the first object in the target image and a first offset, the first offset being the offset between the vertex coordinates of target vertices in the same direction of the first canvas and the target image; and based on the first position, determine the first canvas area that is marked with the image area where the first object in the target image is located.

[0169] Optionally, the canvas area determination module 17 is used to determine a second position of the outline of the second object in the first canvas based on the coordinates of the outline of the second object in the target image and the first offset; and determine a second canvas area marked with the difference between the image areas where the second object and the first object are located in the target image based on the difference between the second position and the first position.

[0170] Optionally, the annotation module 14 is used to obtain the annotation style of the second canvas if the user selects the second object; and annotate the second canvas area on a third canvas area of ​​the second canvas according to the annotation style of the second canvas, wherein the annotation style of the third canvas area is different from that of the first canvas area and the second canvas area; wherein the annotation style of the second canvas includes the stroke result of the outline of the object in the target image, and the fill result inside the outline of the object in the target image.

[0171] Optionally, the display module 13 is used to draw the outlines of the first object and the second object in the target image on a third canvas if the user selects the second object; draw the image area corresponding to the minimum rectangular area in the target image on a fourth canvas; adjust the transparency of the corresponding canvas area in the fourth canvas according to the respective transparencies of different canvas areas in the third canvas, so as to obtain the target area displayed in the processing result area of ​​the target page; wherein the third canvas and the fourth canvas respectively include the difference canvas area between the outline of the second object and the outline of the first object and the remaining canvas area, the transparency of the difference canvas area in the third canvas is 1, and the transparency of the remaining canvas area in the third canvas is 0.

[0172] Optionally, the device further includes an interaction adjustment module 18, which is used to obtain the position of the interaction tool in the target page before and after the movement in response to the movement of the interaction tool, the position of the annotation result in the target page moves with the movement of the interaction tool, and the annotation result is the superposition result of the target image, the first canvas and the second canvas; according to the second offset between the position coordinates of the interaction tool before and after the movement, determine the third position of the movement result of the target image in the annotation result in the canvas, and the canvas includes the first canvas and the second canvas; according to the position coordinates of the annotation trace mapped to the original image, the respective sizes of the target image and the original image, and the third position, determine the fourth position of the movement result of the annotation trace in the annotation result in the canvas; wherein the annotation trace includes a first annotation trace and a second annotation trace, the first annotation trace corresponds to the first canvas, and the second annotation trace corresponds to the second canvas; according to the third position and the fourth position, the movement result of the annotation trace is superimposed and rendered on the movement result of the target image to obtain the annotation result after the movement.

[0173] Optionally, the interactive adjustment module 18 is further used to respond to a zoom operation triggered on the annotation result, using the current position of the interactive tool in the canvas as the zoom center; determine the fifth position of the zoom result of the target image in the annotation result in the canvas according to a third offset between the zoom center and any vertex coordinate of the canvas, the first offset, and the zoom coefficient corresponding to the zoom operation; determine the sixth position of the zoom result of the annotation trace in the annotation result in the canvas according to the position coordinates of the annotation trace mapped to the original image, the size of the zoom result, the size of the original image, and the fifth position; and superimpose and render the zoom result of the annotation trace on the zoom result of the target image according to the fifth position and the sixth position to obtain the zoomed annotation result.

[0174] Fig.12 The device shown can perform Figures 1 to 11 For the method of the embodiment shown in the figure, the part not described in detail in this embodiment can be referred to Figures 1 to 11 The implementation process and technical effects of this technical solution refer to Figures 1 to 11 The description in the illustrated embodiment will not be repeated here.

[0175] In a possible design, the structure of the image annotation device can be implemented as an electronic device, such as Fig.13 As shown, the electronic device may include: a processor 21 and a memory 22. The memory 22 is used to store the electronic device to perform the above Figures 1 to 11The program of the picture annotation method provided in the illustrated embodiment, the processor 21 is configured to execute the program stored in the memory 22 .

[0176] The program includes one or more computer instructions, wherein the one or more computer instructions can implement the following steps when executed by the processor 21:

[0177] In response to an object selection operation triggered by a user using an interactive tool, a trigger position of the object selection operation in a first canvas is obtained, wherein the first canvas includes a first canvas area marked with a picture area where a first object in a target picture is located, and a second canvas area marked with a difference between picture areas where a second object in the target picture and the first object are located, the picture area where the first object is located is included in the picture area where the second object is located, and the first canvas area and the second canvas area have different marking styles;

[0178] Determining the object selected by the user according to the marking style of the canvas area to which the trigger position belongs;

[0179] If the user selects the second object, displaying the target area in the target image corresponding to the second canvas area in the processing result area of ​​the target page;

[0180] If the user selects the second object, the second canvas area is marked in the third canvas area of ​​the second canvas, and the target image, the first canvas and the second canvas are superimposed and displayed in the operation area of ​​the target page.

[0181] Optionally, the processor 21 is further configured to execute the aforementioned Figures 1 to 11 All or part of the steps in the illustrated embodiments.

[0182] The structure of the electronic device may further include a communication interface 23 for the electronic device to communicate with other devices or a communication network.

[0183] In addition, an embodiment of the present invention provides a computer storage medium for storing computer software instructions used by the above electronic device, which includes instructions for executing the above Figures 1 to 11 The procedures involved in the picture annotation method shown.

[0184] In addition, an embodiment of the present invention provides a computer program product. The computer program product includes a computer program or an instruction. When the computer program or the instruction is executed by a processor, the processor is enabled to implement the above Figures 1 to 11 The steps or functions of the method shown.

[0185] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for image annotation, It is characterized in that include: In response to an object selection operation triggered by a user using an interactive tool, a trigger position of the object selection operation in a first canvas is obtained, wherein the first canvas includes a first canvas area marked with a picture area where a first object in a target picture is located, and a second canvas area marked with a difference between picture areas where a second object in the target picture and the first object are located, the picture area where the first object is located is included in the picture area where the second object is located, and the first canvas area and the second canvas area have different marking styles; Determining the object selected by the user according to the marking style of the canvas area to which the trigger position belongs; If the user selects the second object, displaying the target area in the target image corresponding to the second canvas area in the processing result area of ​​the target page; If the user selects the second object, the second canvas area is marked in the third canvas area of ​​the second canvas, and the target image, the first canvas and the second canvas are superimposed and displayed in the operation area of ​​the target page.

2. The method according to claim 1, It is characterized in that The method further comprises: Obtaining the aspect ratio of the operation area and the aspect ratio of the original image; According to the comparison result of the aspect ratios of the operation area and the original picture, the size of the original picture is adjusted to obtain a target picture matching the size of the operation area.

3. The method according to claim 1, It is characterized in that The first canvas area and the second canvas area are marked with different colors in the first canvas; the colors of the canvas areas correspond to the identities of the objects in the target image; The determining the object selected by the user according to the annotation style of the canvas area to which the trigger position belongs includes: Determine the target identity corresponding to the color according to the color of the canvas area to which the trigger position belongs; For the object corresponding to the target identity, if the selected state of the object is identified as selected, the object corresponding to the target identity is determined as the object selected by the user.

4. The method according to claim 3, It is characterized in that The determining, according to the color of the canvas area to which the trigger position belongs, the target identity corresponding to the color includes: Determine the color corresponding to each element in a preset canvas area centered on the trigger position; If the colors of the elements in the preset canvas area are the same, the identity corresponding to the color is determined as the target identity.

5. The method according to claim 1, It is characterized in that The annotation style includes filling a canvas area with a color, and the method further includes: Determining, according to the identity identifier of the object in the target image, colors corresponding to the first object and the second object, respectively, where the color corresponding to the first object is different from the color corresponding to the second object; Filling the image region where the first object is located in the target image according to the color corresponding to the first object; Fill the image region where the second object is located in the target image according to the color corresponding to the second object.

6. The method according to claim 1, It is characterized in that The method further comprises: determining a first position of the outline of the first object in the first canvas according to the coordinates of the outline of the first object in the target image and a first offset, wherein the first offset is an offset between vertex coordinates of target vertices in the first canvas and the target image in the same direction; Determine, according to the first position, the first canvas area in which the image area where the first object in the target image is located is marked.

7. The method according to claim 6, It is characterized in that The method further comprises: determining a second position of the outline of the second object in the first canvas according to the coordinates of the outline of the second object in the target image and the first offset; According to the difference between the second position and the first position, a second canvas area marked with the difference between the image areas where the second object and the first object in the target image are located is determined.

8. The method according to claim 1, It is characterized in that If the user selects the second object, marking the second canvas area in the third canvas area of ​​the second canvas includes: If the user selects the second object, obtaining the annotation style of the second canvas; According to the marking style of the second canvas, marking the second canvas area on a third canvas area of ​​the second canvas, wherein the marking style of the third canvas area is different from that of the first canvas area and the second canvas area; The annotation style of the second canvas includes a stroke result of the outline of the object in the target image and a fill result inside the outline of the object in the target image.

9. The method according to claim 1, It is characterized in that If the user selects the second object, displaying the target area in the target image corresponding to the second canvas area in the processing result area of ​​the target page includes: If the user selects the second object, the outlines of the first object and the second object in the target image are drawn in a third canvas; Drawing the image area corresponding to the minimum rectangular area in the target image on the fourth canvas; According to the respective transparencies of different canvas areas in the third canvas, the transparencies of corresponding canvas areas in the fourth canvas are adjusted to obtain a target area displayed in the processing result area of ​​the target page; The third canvas and the fourth canvas respectively include a difference canvas area between the outline of the second object and the outline of the first object and a remaining canvas area, the transparency of the difference canvas area in the third canvas is 1, and the transparency of the remaining canvas area in the third canvas is 0.

10. The method according to claim 2, It is characterized in that The method further comprises: In response to the movement of the interactive tool, the position of the interactive tool in the target page before and after the movement is acquired, the position of the annotation result in the target page moves with the movement of the interactive tool, and the annotation result is a superposition result of the target image, the first canvas, and the second canvas; Determining, according to a second offset between the position coordinates of the interactive tool before and after the movement, a third position of the movement result of the target image in the annotation result in the canvas, the canvas including the first canvas and the second canvas; Determine a fourth position of a movement result of the annotation trace in the annotation result in the canvas according to the position coordinates of the annotation trace mapped to the original picture, the sizes of the target picture and the original picture, and the third position; The annotation traces include a first annotation trace and a second annotation trace, the first annotation trace corresponds to the first canvas, and the second annotation trace corresponds to the second canvas; According to the third position and the fourth position, the movement result of the annotation trace is superimposed and rendered on the movement result of the target image to obtain the annotation result after the movement.

11. The method according to claim 10, It is characterized in that The method further comprises: In response to a zoom operation triggered on the annotation result, taking a current position of the interactive tool in the canvas as a zoom center; Determine a fifth position of a scaling result of the target image in the annotation result in the canvas according to a third offset between the scaling center and any vertex coordinate of the canvas, the first offset, and a scaling coefficient corresponding to the scaling operation; Determine a sixth position of the scaling result of the annotation trace in the annotation result in the canvas according to the position coordinates of the annotation trace mapped to the original picture, the size of the scaling result, the size of the original picture, and the fifth position; According to the fifth position and the sixth position, the scaling result of the annotation trace is superimposed and rendered on the scaling result of the target image to obtain a scaled annotation result.

12. A picture annotation device, It is characterized in that include: an acquisition module, configured to, in response to an object selection operation triggered by a user using an interactive tool, acquire a trigger position of the object selection operation in a first canvas, wherein the first canvas includes a first canvas area marked with a picture area where a first object in a target picture is located, and a second canvas area marked with a difference between picture areas where a second object in the target picture and the first object are located, the picture area where the first object is located is included in the picture area where the second object is located, and the first canvas area and the second canvas area have different annotation styles; A determination module, used to determine the object selected by the user according to the annotation style of the canvas area to which the trigger position belongs; a display module, configured to display a target area in the target image corresponding to the second canvas area in a processing result area of ​​the target page if the user selects the second object; The marking module is used for marking the second canvas area in the third canvas area of ​​the second canvas if the user selects the second object, and the target image, the first canvas and the second canvas are superimposed and displayed in the operation area of ​​the target page.

13. An electronic device, It is characterized in that include: A memory and a processor; wherein the memory stores executable codes, and when the executable codes are executed by the processor, the processor executes the image labeling method according to any one of claims 1 to 11.

14. A non-transitory machine-readable storage medium, It is characterized in that The non-transitory machine-readable storage medium stores executable codes, and when the executable codes are executed by a processor of an electronic device, the processor is caused to execute the image tagging method according to any one of claims 1 to 11.

Citation Information

Cited By

  • Image labeling method, client, terminal equipment and storage medium

    CN119963694A

  • Image annotation method, client, terminal device and storage medium

    CN119963694B