Image annotation method, client, terminal device and storage medium

Abstract

The present application discloses an image annotation method, client, terminal device, and storage medium, relating to the field of image processing technology. The method, applied to the client, includes: after rendering the image to be annotated into a target container, calculating the scaling ratio of the image to be annotated before and after rendering, as well as the offset of the image to be annotated relative to the target container after rendering into the target container; performing target detection on the image to be annotated based on a preset target detection model to obtain a target detection result for the image to be annotated; correcting the target detection result using the offset and scaling ratio to obtain target annotation information for the image to be annotated; and annotating the image to be annotated rendered into the target container based on the target annotation information. The present application can improve the accuracy of client image annotation.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image annotation method, a client, a terminal device, and a storage medium. Background Art

[0002] Currently, when the target detection model recognizes and detects an image, the annotation information it returns is based on the original image size. When the client annotates the image based on this annotation information, if the image is scaled when adapting to the display interface, the annotations may be misplaced or distorted. Summary of the Invention

[0003] The main purpose of this application is to provide an image annotation method, client, terminal device and storage medium, aiming to solve the technical problem of misalignment or distortion of image annotation based on target detection models in related technologies.

[0004] To achieve the above objectives, the present application provides an image annotation method, which is applied to a client and includes:

[0005] After rendering the image to be annotated into the target container, calculating the scaling ratio of the image to be annotated before and after rendering, and the offset of the image to be annotated relative to the target container after rendering into the target container;

[0006] Performing target detection on the image to be labeled based on a preset target detection model to obtain a target detection result of the image to be labeled;

[0007] Correcting the target detection result by using the offset and the scaling ratio to obtain target labeling information of the image to be labeled;

[0008] The image to be labeled rendered into the target container is labeled according to the target labeling information.

[0009] In one embodiment, the step of calculating the scaling ratio of the image to be annotated before and after rendering, and the offset of the image to be annotated relative to the target container after rendering into the target container, includes:

[0010] Obtaining the original size and logical size of the image to be annotated, and calculating the scaling ratio of the image to be annotated before and after rendering based on the original size and the logical size;

[0011] Layout information of the image to be annotated rendered into the target container and the container size of the target container are obtained, and an offset of the image to be annotated relative to the target container after being rendered into the target container is calculated based on the layout information, the container size, and the logical size.

[0012] In one embodiment, the client is in communication with a server, and a preset object detection model is deployed on the server;

[0013] The step of performing target detection on the image to be labeled based on a preset target detection model to obtain a target detection result of the image to be labeled includes:

[0014] Sending the image to be labeled to the server, so that the server performs target detection on the image to be labeled based on a preset target detection model and generates a target detection result for the image to be labeled;

[0015] Receive the target detection result returned by the server based on the image to be labeled.

[0016] In one embodiment, the target detection result includes initial annotation information of the image to be annotated, and the step of correcting the target detection result using the offset and the scaling ratio to obtain the target annotation information of the image to be annotated includes:

[0017] Performing scaling correction on the initial annotation information according to the scaling ratio to obtain first annotation information of the image to be annotated;

[0018] Performing offset correction on the first annotation information according to the offset to obtain second annotation information of the image to be annotated;

[0019] The second annotation information is determined as target annotation information of the image to be annotated.

[0020] In one embodiment, the step of labeling the image to be labeled rendered into the target container according to the target labeling information includes:

[0021] Annotating the image to be annotated rendered into the target container on a preset transparent canvas according to the target annotation information;

[0022] The transparent canvas completely overlaps with the target container, and the transparent canvas is located on an upper layer of the target container.

[0023] In one embodiment, before the step of annotating the image to be annotated rendered into the target container on a preset transparent canvas according to the target annotation information, the method further includes:

[0024] Detecting whether a transparent canvas that completely overlaps with the target container is provided on an upper layer of the target container;

[0025] If a transparent canvas that completely overlaps with the target container is not provided on the upper layer of the target container, a transparent canvas that completely overlaps with the target container is provided on the upper layer of the target container.

[0026] In one embodiment, after the step of detecting whether a transparent canvas completely overlapping with the target container is provided on the upper layer of the target container, the method further includes:

[0027] If a transparent canvas completely overlapping the target container is provided on the upper layer of the target container, detecting whether a mark is drawn on the transparent canvas;

[0028] If a mark is drawn on the transparent canvas, the transparent canvas is cleared, and a new transparent canvas completely overlapping with the target container is set on the upper layer of the target container.

[0029] In addition, to achieve the above objectives, the present application also provides a client, which includes:

[0030] a calculation module, configured to calculate, after rendering the image to be annotated into a target container, a scaling ratio of the image to be annotated before and after rendering, and an offset of the image to be annotated relative to the target container after rendering the image to be annotated into the target container;

[0031] A detection module is used to perform target detection on the image to be labeled based on a preset target detection model to obtain a target detection result of the image to be labeled;

[0032] a correction module, configured to correct the target detection result by using the offset and the scaling ratio to obtain target annotation information of the image to be annotated;

[0033] The labeling module is configured to label the image to be labeled rendered into the target container according to the target labeling information.

[0034] In addition, to achieve the above-mentioned purpose, the present application also provides a terminal device, which includes: a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and when the computer program is executed by the processor, the steps of the image annotation method as described above are implemented.

[0035] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium. A computer program is stored on the storage medium, and when the computer program is executed by a processor, the steps of the image annotation method as described above are implemented.

[0036] The present application provides an image annotation method, client, terminal device, and storage medium, relating to the field of image processing technology. The method is applied to the client and includes: after rendering the image to be annotated into a target container, calculating the scaling ratio of the image to be annotated before and after rendering, as well as the offset of the image to be annotated relative to the target container after rendering into the target container; performing target detection on the image to be annotated based on a preset target detection model to obtain a target detection result for the image to be annotated; correcting the target detection result by the offset and scaling ratio to obtain target annotation information for the image to be annotated; and annotating the image to be annotated rendered into the target container based on the target annotation information. The present application can improve the accuracy of client image annotation. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0038] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

[0039] Figure 1 Schematic diagram of the process of the image annotation method in the embodiment of the present application;

[0040] Figure 2 Schematic diagram of the calculation process of the scaling ratio and offset in the embodiment of the present application;

[0041] Figure 3 Schematic diagram of the correction process of target detection results in the embodiment of the present application;

[0042] Figure 4 This is a schematic diagram of the module structure of the client in an embodiment of the present application;

[0043] Figure 5 Schematic diagram of the device structure of the hardware operating environment involved in the image annotation method in the embodiment of the present application.

[0044] The purpose, features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0045] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0046] It should be understood that the specific embodiments described herein are merely used to explain the technical solutions of the present application and are not intended to limit the present application.

[0047] Currently, when the target detection model recognizes and detects an image, the annotation information it returns is based on the original image size. When the client annotates the image based on this annotation information, if the image is scaled when adapting to the display interface, the annotations may be misplaced or distorted.

[0048] In response to this technical problem, the main solution of this application is an image annotation method, which is applied to the client and includes: after rendering the image to be annotated into a target container, calculating the scaling ratio of the image to be annotated before and after rendering, and the offset of the image to be annotated relative to the target container after rendering into the target container; performing target detection on the image to be annotated based on a preset target detection model to obtain a target detection result of the image to be annotated; correcting the target detection result by using the offset and the scaling ratio to obtain target annotation information of the image to be annotated; and annotating the image to be annotated rendered into the target container according to the target annotation information.

[0049] This application calculates the scaling ratio and offset and uses these parameters to correct the target detection results, ensuring that even if the image changes in size or position when rendered into the container, the annotation can still be accurately reflected in the correct position on the image, greatly improving the accuracy of the annotation and avoiding the misalignment problems that may occur in traditional methods.

[0050] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0051] Please refer to Figure 1 , Figure 1 Schematic diagram of the process of image annotation method in the embodiment of the present application.

[0052] In this embodiment, the image annotation method is applied to the client, and the method includes steps S100 to S400:

[0053] Step S100, after rendering the image to be annotated into the target container, calculating the scaling ratio of the image to be annotated before and after rendering, and the offset of the image to be annotated relative to the target container after rendering into the target container;

[0054] It should be noted that, in this embodiment, the client can be a web browser or a mobile application. The image to be annotated refers to the image that needs to be annotated on the client. The target container refers to a specific area in the client for displaying the image to be annotated, which can be a element, a view controller of a mobile app, etc.

[0055] It's also important to note that the scaling ratio refers to the change in size from the original size of the image to be annotated to the rendered size, which can be specifically divided into height scaling and width scaling. The offset refers to the displacement of the image within the target container relative to the top-left corner of the target container after it is rendered into the target container, and includes both horizontal and vertical components.

[0056] Specifically, in one example, Figure 2 As shown, the step of calculating the scaling ratio of the image to be annotated before and after rendering, and the offset of the image to be annotated relative to the target container after rendering to the target container in step S100 includes steps S110 to S120:

[0057] Step S110, obtaining the original size and logical size of the image to be annotated, and calculating the scaling ratio of the image to be annotated before and after rendering based on the original size and logical size;

[0058] Step S120 , obtaining layout information of the image to be annotated rendered into the target container and the container size of the target container, and calculating the offset of the image to be annotated relative to the target container after being rendered into the target container based on the layout information, container size and logical size.

[0059] It should be noted that, in this embodiment, the original size refers to the actual width and height of the image to be annotated in its original file, usually in pixels. The logical size refers to the size of the image to be annotated calculated according to the design or preset rules when it is rendered into the target container. For example, when the image to be annotated is rendered into the target container in a Web (World Wide Web) web page, the logical size of the image to be annotated can be set directly or indirectly through CSS (Cascading Style Sheets). The container size refers to the logical width and height of the target container. For example, when creating a target container in a Web page, the container size of the target container can be set through CSS. The layout information may include but is not limited to the positioning method of the image to be annotated in the target container (such as centering, left alignment, etc.), margin settings, padding, etc., all of which will affect the position of the final image in the container.

[0060] In this example, the client first needs to know the original dimensions of the image to be annotated and the logical size at which it should be rendered. This step is necessary because only with these two dimensions can the image be accurately scaled. Once the original and logical dimensions are known, the scaling ratio can be calculated using simple math. For example, if the original width is 800 pixels and the logical width is 400 pixels, the width scaling ratio is 0.5. The height scaling ratio can be calculated similarly.

[0061] Through step S110 , this example ensures that subsequent target detection results can be adjusted according to the actual display conditions of the image to be annotated, thereby ensuring that the accuracy of annotation is not affected by image scaling.

[0062] In this example, layout information and container dimensions are involved in understanding how the image should be placed within the target container. Different layout strategies result in different offsets. For example, if the image is center-aligned, its horizontal offset will be half the difference between the container width and the image width. The container dimensions also determine the maximum possible offset range for the image.

[0063] Using layout information, container size, and logical size, this example can perform some geometric calculations to determine the specific offset of the image relative to the upper-left corner of the container. This ensures that even if the image is shifted within the container, the client still knows its exact position within the container.

[0064] By accurately calculating the scaling ratio and offset, this example's method effectively addresses the problem of misaligned annotations caused by image resizing. This ensures consistent and accurate annotations across images displayed in web browsers and mobile apps. This approach enhances system flexibility and robustness, improving the user experience and is particularly critical for applications that rely on high-precision image analysis.

[0065] It is not difficult to understand that in addition to the scaling ratio and offset calculation methods provided in the above examples, the scaling ratio and offset can also be calculated or obtained by other methods. For example, when the image to be annotated is rendered into the target container, the relationship between the logical width of the image to be annotated and the logical width of the target container is set to 100% through CSS, and the ratio between the height and width of the image to be annotated remains unchanged and is centered in the target container. The scaling ratio can be directly determined by obtaining the relationship between the width in CSS, and the offset can be calculated based on the layout information, container size and aspect ratio of the image to be annotated.

[0066] Step S200, performing target detection on the image to be labeled based on a preset target detection model to obtain a target detection result of the image to be labeled;

[0067] It should be noted that the preset object detection model is a pre-trained machine learning or deep learning model that can identify specific types of objects in an image and provide their location information (such as bounding boxes). The object detection result is the output of the object detection model after performing object detection on the image to be annotated. It can include the object categories detected by the object detection model in the image to be annotated, as well as their corresponding location information, sequence number, and confidence level.

[0068] Specifically, in one example, a client is connected to a server for communication, and a preset object detection model is deployed on the server;

[0069] like Figure 3 As shown, step S200 may include steps S210 to S220:

[0070] Step S210: sending the image to be annotated to the server, so that the server performs target detection on the image to be annotated based on a preset target detection model and generates a target detection result for the image to be annotated;

[0071] Step S220: receiving the target detection result returned by the server based on the image to be annotated.

[0072] In this example, the object detection model is deployed on the server, and the client communicates with the server, offloading image processing and object detection tasks to the server. This effectively reduces the client's computational burden, especially for devices with limited computing resources (such as mobile phones and tablets). This helps improve the overall performance of the application and ensures smooth operation even on low-end devices.

[0073] Furthermore, since the object detection model is deployed on the server, it can be easily updated or maintained through centralized management without requiring users to download new application versions. This is crucial for keeping the system up to date and secure.

[0074] In this example, once the server completes object detection, it quickly returns the results to the client, allowing the user to see the annotation results almost immediately. This fast response time improves the user experience, making the user feel that the operation is immediate and effective. Moreover, compared to running a simplified or outdated model on the client, this example ensures higher detection accuracy by using high-performance server hardware and the latest object detection model, providing more reliable object detection results and thus achieving more accurate image annotation.

[0075] Through collaboration between the client and server, this example method not only achieves efficient image object detection, but also improves the system's flexibility, maintainability, and user experience. Furthermore, the separation of computationally intensive tasks and server-side processing makes the system more adaptable to future expansion and technological advancements.

[0076] Step S300: Correcting the target detection result by using the offset and scaling ratio to obtain target annotation information of the image to be annotated;

[0077] It should be noted that, in this embodiment, since the target detection result generated by the target detection model is based on the original size of the image to be annotated, after scaling and displacement occur during the process of rendering the image to be annotated into the target container, if the annotation information in the target detection result is directly used to annotate the image to be annotated rendered into the target container, it is easy for the annotation to be misplaced or distorted. Therefore, this embodiment adaptively corrects the target detection result according to the scaling ratio and offset during the process of rendering the image to be annotated into the target container, and can obtain annotation information that matches the image to be annotated currently rendered into the target container, that is, the target annotation information.

[0078] For example, in one example, the target detection result includes initial annotation information of the image to be annotated, and step S300 may include steps S310 to S330:

[0079] Step S310, performing scaling correction on the initial annotation information according to the scaling ratio to obtain first annotation information of the image to be annotated;

[0080] Step S320, performing offset correction on the first annotation information according to the offset to obtain second annotation information of the image to be annotated;

[0081] Step S330: Determine the second annotation information as target annotation information of the image to be annotated.

[0082] It should be noted that the initial annotation information refers to the annotation information generated by the target detection model based on the original size of the image to be annotated. The first annotation information refers to the annotation information obtained after applying the scaling ratio to the initial annotation information for scaling correction. The first annotation information is consistent in size with the image to be annotated rendered in the target container. The second annotation information refers to the annotation information obtained after applying the offset to the first annotation information for offset correction. The second annotation information is consistent not only in size with the image to be annotated rendered in the target container, but also in the degree of offset.

[0083] It should also be noted that scale correction refers to adjusting the position parameters in the annotation information based on the calculated scale ratio to adapt to the logical size of the image after rendering. Offset correction refers to adjusting the position parameters in the annotation information based on the calculated offset to ensure that the annotation information can accurately reflect the exact position of the image in the target container, correcting the annotation misalignment problem caused by the displacement of the image in the container (i.e., center alignment, left alignment, etc.).

[0084] In this example, scale correction ensures that the annotation information remains consistent with the image's logical size. Even if the image is scaled up or down during rendering, the annotation remains precisely aligned to the correct size within the image. Offset correction addresses the issue of annotation misalignment caused by image movement within the container (i.e., different layouts). Whether the image is centered, left, or right-aligned, or has margins set, offset correction ensures that the annotation information matches the image's true position.

[0085] This example effectively solves the annotation problems caused by image size changes and position shifts by introducing two steps: scaling correction and offset correction. This ensures that even if the image size or position changes during display, the annotations are still accurately reflected in the correct position on the image. This improves the accuracy and reliability of annotations, enhances the flexibility and robustness of the system, and provides users with a more efficient and intuitive image annotation solution.

[0086] Step S400 : annotating the image to be annotated rendered into the target container according to the target annotation information.

[0087] In one example, step S400 may include step S410:

[0088] Step S410: annotating the image to be annotated rendered into the target container on a preset transparent canvas according to the target annotation information;

[0089] The transparent canvas completely overlaps with the target container, and the transparent canvas is located on the upper layer of the target container.

[0090] It should be noted that the transparent canvas is a layer located above the target container. While completely overlapping with the target container, it is transparent and does not obscure the target container and image below. This transparent canvas is used to draw annotation information.

[0091] In this embodiment, the transparent canvas may be set on the target container in advance, so as to complete the drawing of the annotation information on the transparent canvas.

[0092] It's important to note that the transparent canvas completely overlaps with the target container, indicating that its logical size and position match those of the target container. Therefore, when annotating the image rendered into the target container on the transparent canvas, the target annotation information can be directly applied without misalignment or distortion.

[0093] This example achieves a visual separation between annotation information and the original image by using a transparent canvas. This not only allows users to clearly distinguish between image content and annotation information, but also allows them to adjust the color and style of the annotation information as needed without affecting the display quality of the image itself. Furthermore, the transparent canvas, as an independent layer, allows users to dynamically modify or delete annotation information without re-rendering the entire image, significantly reducing the client's computational burden, speeding up the response time of annotation drawing, and increasing the flexibility of user interaction, making the annotation process more intuitive and convenient.

[0094] In this example, step S410 uses a transparent canvas as the annotation carrier, combined with the previously calculated target annotation information, to achieve accurate annotation of the image to be annotated. This approach not only maintains the original appearance of the image, but also provides a clear, flexible, and efficient annotation interface. Through features such as visual separation, interactive friendliness, and accuracy assurance, the user experience and work efficiency are greatly improved. At the same time, the application of the transparent canvas also lays the foundation for system performance optimization and function expansion, making the image annotation solution more complete and advanced.

[0095] Furthermore, in a feasible implementation manner, before step S410, steps A10 to A20 may be further included:

[0096] Step A10: detecting whether a transparent canvas completely overlapping the target container is provided on the upper layer of the target container;

[0097] Step A20 : If a transparent canvas that completely overlaps with the target container is not provided on the upper layer of the target container, a transparent canvas that completely overlaps with the target container is provided on the upper layer of the target container.

[0098] Before executing step S410, this embodiment first ensures that a transparent canvas completely overlaps the target container. For example, in Web technology, the existence of a transparent canvas can be determined by checking the DOM (Document Object Model) structure for an element with a specific identifier or class name.

[0099] When it is determined in step A10 that no transparent canvas that meets the requirements exists above the target container, this embodiment needs to create a new transparent canvas above the target container. For example, in Web technology, this can be achieved through the Canvas API (Canvas Application Programming Interface).

[0100] This embodiment introduces steps A10 and A20 to ensure that a new transparent canvas is created only when necessary, avoiding unnecessary consumption of system resources, thereby reducing the initial loading time of the interface, improving the startup speed of the application, and making the entire process more robust and efficient.

[0101] In addition, when the size of the target container changes during actual application, this process can also dynamically adjust or rebuild the transparent canvas to ensure that it always completely overlaps with the target container, thereby maintaining a high-quality annotation experience.

[0102] Furthermore, in another feasible implementation manner, after step A10, steps A30 to A40 may be further included:

[0103] Step A30: If a transparent canvas completely overlapping the target container is provided on the upper layer of the target container, then detecting whether a mark is drawn on the transparent canvas;

[0104] In step A40 , if a mark is drawn on the transparent canvas, the transparent canvas is cleared, and a new transparent canvas is set on the upper layer of the target container to completely overlap with the target container.

[0105] After confirming in step A10 that a transparent canvas completely overlaps the target container, this embodiment next checks in step A30 whether any annotation information has been drawn on this transparent canvas. For example, in Web technologies, the presence of any non-default drawing operations can be determined by querying the state of the Canvas drawing context or traversing saved drawing command records. In some cases, a Boolean value or flag can be directly stored to indicate whether the canvas has been modified.

[0106] In this embodiment, the purpose of step A30 is to determine the current state of the transparent canvas in order to determine the subsequent operation. If the transparent canvas has been used for annotation, then the processing method of these annotations needs to be considered; if not, then it can directly proceed to the next stage.

[0107] In this embodiment, if it is found through step A30 that there are indeed annotations (whether one or more) on the transparent canvas, it is necessary to clear these annotations through step A40 and reset a new transparent canvas to ensure that each time a new annotation task is started, one can work on a clean, uncontaminated transparent canvas.

[0108] Through this embodiment, whenever a new image to be annotated is rendered into a target container for drawing, the previously used transparent canvas can be cleared and a new transparent canvas can be provided for annotation and drawing.

[0109] It is not difficult to understand that when a new image to be annotated is rendered into the target container, the original image to be annotated will be cleared synchronously.

[0110] Furthermore, it should be noted that, in this embodiment, when the image to be annotated is rendered into the target container, the logical size of the image to be annotated should not be larger than the container size of the target container to prevent the image to be annotated from overflowing the target container. Preferably, in this embodiment, when the image to be annotated is rendered into the target container, the image to be annotated is centered within the target container without changing its aspect ratio, and its logical width and height are the same as those of the target container.

[0111] This application calculates the scaling ratio and offset and uses these parameters to correct the target detection results, ensuring that even if the image changes in size or position when rendered into the container, the annotation can still be accurately reflected in the correct position on the image, greatly improving the accuracy of the annotation and avoiding the misalignment problems that may occur in traditional methods.

[0112] In addition, please refer to Figure 4 , Figure 4 This is a schematic diagram of the module structure of the client in the embodiment of the present application.

[0113] The present application also provides a client, which includes:

[0114] The calculation module 10 is used to calculate the scaling ratio of the image to be annotated before and after rendering, and the offset of the image to be annotated relative to the target container after rendering the image to be annotated into the target container.

[0115] A detection module 20 is configured to perform target detection on the image to be labeled based on a preset target detection model to obtain a target detection result for the image to be labeled;

[0116] The correction module 30 is used to correct the target detection result by using the offset and scaling ratio to obtain target annotation information of the image to be annotated;

[0117] The labeling module 40 is configured to label the image to be labeled rendered into the target container according to the target labeling information.

[0118] In one embodiment, the calculation module 10 is further configured to:

[0119] Obtain the original size and logical size of the image to be annotated, and calculate the scaling ratio of the image to be annotated before and after rendering based on the original size and logical size;

[0120] Obtain the layout information of the target container in which the image to be annotated is rendered, as well as the container size of the target container. Calculate the offset of the image to be annotated relative to the target container after it is rendered into the target container based on the layout information, container size, and logical size.

[0121] In one embodiment, the client is in communication with the server, and the preset object detection model is deployed on the server;

[0122] The detection module 20 is further configured to:

[0123] Send the image to be labeled to the server, so that the server can perform target detection on the image to be labeled based on the preset target detection model and generate the target detection result of the image to be labeled;

[0124] Receive the target detection results returned by the server based on the image to be annotated.

[0125] In one embodiment, the object detection result includes initial annotation information of the image to be annotated, and the correction module 30 is further configured to:

[0126] Performing scaling correction on the initial annotation information according to the scaling ratio to obtain first annotation information of the image to be annotated;

[0127] Performing offset correction on the first annotation information according to the offset to obtain second annotation information of the image to be annotated;

[0128] The second annotation information is determined as target annotation information of the image to be annotated.

[0129] In one embodiment, the annotation module 40 is further configured to:

[0130] According to the target annotation information, the image to be annotated rendered into the target container is annotated on a preset transparent canvas;

[0131] The transparent canvas completely overlaps with the target container, and the transparent canvas is located on the upper layer of the target container.

[0132] In one embodiment, the annotation module 40 is further configured to:

[0133] Detect whether a transparent canvas that completely overlaps with the target container is set on the upper layer of the target container;

[0134] If a transparent canvas that completely overlaps with the target container is not set on the upper layer of the target container, a transparent canvas that completely overlaps with the target container is set on the upper layer of the target container.

[0135] In one embodiment, the annotation module 40 is further configured to:

[0136] If a transparent canvas that completely overlaps with the target container is set on the upper layer of the target container, then check whether there is a label drawn on the transparent canvas;

[0137] If a markup is drawn on the transparent canvas, the transparent canvas is cleared, and a new transparent canvas is set on the upper layer of the target container to completely overlap with the target container.

[0138] The client provided in this application utilizes the image annotation method described in the aforementioned embodiments, resolving the technical issues of misalignment or distortion in image annotation based on target detection models in related technologies. Compared to the prior art, the client provided in this application achieves the same beneficial effects as the image annotation method described in the aforementioned embodiments, and the client's other technical features are the same as those disclosed in the image annotation method described in the aforementioned embodiments, which are not further detailed here.

[0139] In addition, please refer to Figure 5 , Figure 5 Schematic diagram of the device structure of the hardware operating environment involved in the image annotation method in the embodiment of the present application.

[0140] The present application also provides a terminal device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of the image annotation method in the above embodiment.

[0141] Reference below Figure 5 , which shows a schematic diagram of the structure of a terminal device suitable for implementing the embodiments of the present application. The terminal device in the embodiments of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and fixed terminals such as digital TVs (Televisions), desktop computers, and the like, or any other terminal device capable of implementing the above functions. Figure 5 The terminal device shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0142] like Figure 5 As shown, the terminal device may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes based on programs stored in a read-only memory (ROM) 1002 or programs loaded from a storage device 1003 into a random access memory (RAM) 1004. RAM 1004 also stores various programs and data required for the terminal device's operation. Processing device 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems may be connected to I / O interface 1006: input devices 1007, such as a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008, such as a liquid crystal display (LCD), speaker, vibrator, etc.; storage device 1003, such as a magnetic tape or hard disk; and communication device 1009. The communication device 1009 can allow the terminal device to communicate with other devices wirelessly or wired to exchange data. Although the figure shows a terminal device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or provided instead.

[0143] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0144] The terminal device provided in this application utilizes the image annotation method described in the above-mentioned embodiments, which can resolve the technical issues of misalignment or distortion in image annotation based on target detection models in related technologies. Compared with the prior art, the beneficial effects of the terminal device provided in this application are the same as those of the image annotation method described in the above-mentioned embodiments, and the other technical features of the terminal device are the same as those disclosed in the above-mentioned embodiments, which are not further described here.

[0145] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0146] The above are merely specific embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the above claims.

[0147] In addition, the present application also provides a storage medium, which is a computer-readable storage medium having computer-readable program instructions (ie, a computer program) stored thereon, and the computer-readable program instructions are used to execute the steps of the image annotation method in the above embodiment.

[0148] The computer-readable storage medium provided herein may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems, or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including, but not limited to, wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0149] The computer-readable storage medium may be included in the client; or may exist independently without being assembled into the client.

[0150] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by a client, the client is caused to: after rendering the image to be labeled into a target container, calculate the scaling ratio of the image to be labeled before and after rendering, and the offset of the image to be labeled relative to the target container after rendering into the target container; perform target detection on the image to be labeled based on a preset target detection model to obtain a target detection result for the image to be labeled; correct the target detection result using the offset and scaling ratio to obtain target labeling information for the image to be labeled; and label the image to be labeled rendered into the target container based on the target labeling information.

[0151] Computer program code for performing the operations of the present application may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, and C++, as well as conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0152] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0153] The modules described in the embodiments of the present application may be implemented in software or hardware, wherein the name of a module does not necessarily limit the unit itself.

[0154] The storage medium provided in this application stores computer-readable program instructions for executing the steps of the aforementioned image annotation method, resolving the technical issues of misalignment or distortion in image annotations based on object detection models in related technologies. Compared to the prior art, the beneficial effects of the storage medium provided in this application are similar to those of the image annotation method in the aforementioned embodiments and are not further elaborated here.

[0155] In addition, an embodiment of the present application further provides a program product, which is a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the steps of the image annotation method in the above embodiment are implemented.

[0156] The program product provided in this application can solve the technical problem of misalignment or distortion in image annotation based on target detection models in related technologies. Compared with the prior art, the beneficial effects of the program product provided in the embodiments of this application are the same as those of the image annotation method in the above embodiments, and will not be repeated here.

[0157] The above description is only part of the embodiments of the present application and does not limit the patent scope of the present application. All equivalent structural transformations made by using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. An image annotation method, characterized in that: The method is applied to a client, and includes: After rendering the image to be annotated into the target container, calculating the scaling ratio of the image to be annotated before and after rendering, and the offset of the image to be annotated relative to the target container after rendering into the target container; Performing target detection on the image to be labeled based on a preset target detection model to obtain a target detection result of the image to be labeled; Correcting the target detection result by using the offset and the scaling ratio to obtain target labeling information of the image to be labeled; Annotating the image to be annotated rendered into the target container according to the target annotation information; The step of calculating the scaling ratio of the image to be annotated before and after rendering, and the offset of the image to be annotated relative to the target container after rendering to the target container, includes: Obtaining the original size and logical size of the image to be annotated, and calculating the scaling ratio of the image to be annotated before and after rendering based on the original size and the logical size; Layout information of the image to be annotated rendered into the target container and the container size of the target container are obtained, and an offset of the image to be annotated relative to the target container after being rendered into the target container is calculated based on the layout information, the container size, and the logical size.

2. The image annotation method according to claim 1, wherein: The client is in communication with the server, and a preset target detection model is deployed on the server; The step of performing target detection on the image to be labeled based on a preset target detection model to obtain a target detection result of the image to be labeled includes: Sending the image to be labeled to the server, so that the server performs target detection on the image to be labeled based on a preset target detection model and generates a target detection result for the image to be labeled; Receive the target detection result returned by the server based on the image to be labeled.

3. The image annotation method according to claim 2, wherein: The target detection result includes initial annotation information of the image to be annotated, and the step of correcting the target detection result by using the offset and the scaling ratio to obtain the target annotation information of the image to be annotated includes: Performing scaling correction on the initial annotation information according to the scaling ratio to obtain first annotation information of the image to be annotated; Performing offset correction on the first annotation information according to the offset to obtain second annotation information of the image to be annotated; The second annotation information is determined as target annotation information of the image to be annotated.

4. The image annotation method according to claim 3, wherein: The step of annotating the image to be annotated rendered into the target container according to the target annotation information includes: Annotating the image to be annotated rendered into the target container on a preset transparent canvas according to the target annotation information; The transparent canvas completely overlaps with the target container, and the transparent canvas is located on an upper layer of the target container.

5. The image annotation method according to claim 4, wherein: Before the step of annotating the image to be annotated rendered into the target container on a preset transparent canvas according to the target annotation information, the method further includes: Detecting whether a transparent canvas that completely overlaps with the target container is provided on an upper layer of the target container; If a transparent canvas that completely overlaps with the target container is not provided on the upper layer of the target container, a transparent canvas that completely overlaps with the target container is provided on the upper layer of the target container.

6. The image annotation method according to claim 4, wherein: After the step of detecting whether a transparent canvas completely overlapping with the target container is provided on the upper layer of the target container, the method further includes: If a transparent canvas completely overlapping the target container is provided on the upper layer of the target container, detecting whether a mark is drawn on the transparent canvas; If a mark is drawn on the transparent canvas, the transparent canvas is cleared, and a new transparent canvas completely overlapping with the target container is set on the upper layer of the target container.

7. A client, characterized in that: The client includes: a calculation module, configured to calculate, after rendering the image to be annotated into a target container, a scaling ratio of the image to be annotated before and after rendering, and an offset of the image to be annotated relative to the target container after rendering the image to be annotated into the target container; A detection module is used to perform target detection on the image to be labeled based on a preset target detection model to obtain a target detection result of the image to be labeled; a correction module, configured to correct the target detection result by using the offset and the scaling ratio to obtain target annotation information of the image to be annotated; a labeling module, configured to label the image to be labeled rendered into the target container according to the target labeling information; The calculation module is further configured to: Obtaining the original size and logical size of the image to be annotated, and calculating the scaling ratio of the image to be annotated before and after rendering based on the original size and the logical size; Layout information of the image to be annotated rendered into the target container and the container size of the target container are obtained, and an offset of the image to be annotated relative to the target container after being rendered into the target container is calculated based on the layout information, the container size, and the logical size.

8. A terminal device, characterized in that: The terminal device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the steps of the image annotation method according to any one of claims 1 to 6 when executed by the processor.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the image annotation method according to any one of claims 1 to 6 are implemented.

Images