A multi-format labeling generation method and system for construction material detection

Abstract

This application relates to a multi-format annotation generation method and system for construction material detection, comprising: acquiring target images of building materials and performing instance segmentation and annotation on each object in the target images to obtain an instance segmentation annotation file; wherein, the instance segmentation annotation file includes image information and annotation data, the image information including image path, image width and height, and annotation data including annotation labels and mask data; generating horizontal annotation boxes based on the annotation data of each object according to a horizontal box detection model; generating rotated annotation boxes based on the annotation data of each object according to a rotated box detection model; constraining the generated horizontal and rotated annotation boxes based on the width and height of the image information, filtering out horizontal and rotated annotation boxes that exceed the image information, and obtaining a valid multi-format annotation file. This application achieves automated generation of multiple annotation formats, improves the reusability and sharing efficiency of datasets, and increases the efficiency of data annotation work.

Description

Technical Field

[0001] This application relates to the fields of computer vision and image annotation, and in particular to a method and system for generating multi-format annotations for construction material detection. Background Technology

[0002] With the development of computer vision and deep learning technologies, intelligent detection based on two-dimensional images has been increasingly applied in the field of construction material management. For example, in the inventory management of construction materials such as steel bars, steel pipes, structural steel, and plates, the traditional inventory method involves manual counting and recording, which is time-consuming, labor-intensive, and prone to errors. However, automatic recognition and counting of material end-face images based on computer vision can greatly reduce labor costs and improve management efficiency. In order to train an effective detection model, it is usually necessary to build a high-quality labeled dataset in the target domain. However, the labeling method used in the dataset is often different for different tasks. This leads to the need to repeatedly perform multiple sets of annotations in an engineering scenario, even if the original images are the same, which increases the manpower and time costs significantly. Therefore, there is an urgent need for a low-cost and highly automated annotation generation method to improve the reuse efficiency of labeled data.

[0003] Currently, common annotation methods in material inspection include bounding box annotation (object detection) and mask annotation (instance segmentation), which can be used to train relevant models. However, different tasks have different requirements for annotation formats. For example, rebar counting tasks emphasize real-time performance and require rapid object location identification, so bounding box annotation is typically used; while rebar size measurement tasks require the model to accurately obtain the object's boundary, necessitating mask annotation. Due to the different requirements for annotation formats across tasks, the same image often needs to be annotated multiple times in multi-task scenarios, increasing the cost of manual annotation.

[0004] Furthermore, numerous studies have shown that the diversity of construction materials in terms of shape, size, and stacking methods makes it difficult for a single annotation method to meet the needs of different material detection tasks. For example, when square steel pipes have arbitrary rotation angles, using horizontal rectangular boxes for annotation may introduce noise and redundant areas, requiring the use of rotated rectangular boxes to accurately enclose the target. For irregularly shaped cross-section materials such as I-beams and angle steel, the bounding boxes may overlap significantly when densely stacked, requiring the use of mask annotation to accurately describe the target shape and avoid affecting model training. This diversity of material shapes and scenario requirements necessitates that datasets include multiple annotation methods, but existing annotation tools and methods often struggle to efficiently generate multiple annotation formats simultaneously, further limiting the reusability and sharing efficiency of datasets. Summary of the Invention

[0005] This application provides a method and system for generating multi-format annotations for construction material detection. By achieving efficient and automated generation of various annotation formats, it automatically enriches the annotation formats when the dataset is large and the annotation format is singular, provides model training data for tasks with different needs, improves the reusability and sharing efficiency of the dataset, and effectively improves the work efficiency of data annotation.

[0006] This application provides a method for generating multi-format annotations for construction material detection, comprising the following steps: S201, acquiring a target image of building materials, and performing instance segmentation annotation on each object in the target image to obtain an instance segmentation annotation file; wherein, the instance segmentation annotation file includes image information and annotation data for each object, the image information including the image path, image width and height, and the annotation data including annotation labels and mask data; S202, generating horizontal annotation boxes based on the annotation data of each object in the instance segmentation annotation file according to a horizontal box detection model; S203, generating rotated annotation boxes based on the annotation data of each object in the instance segmentation annotation file according to a rotated box detection model; S204, constraining the generated horizontal annotation boxes and rotated annotation boxes based on the width and height of the image information, filtering out horizontal annotation boxes and rotated annotation boxes that exceed the image information, and obtaining a filtered valid multi-format annotation file.

[0007] Optionally, S202 includes: based on the mask data of each object in the instance segmentation annotation file, obtaining the minimum and maximum boundary pixel values ​​of each object in the horizontal coordinate system according to the mask boundary coordinates of each object; determining the circumscribed rectangle with the smallest area covering the mask of each object according to the mask boundary coordinates; determining the coordinates of the four vertices of the circumscribed rectangle based on the minimum and maximum boundary pixel values ​​of the mask boundary coordinates in the horizontal coordinate system, and generating a horizontal annotation box.

[0008] Optionally, S203 includes: determining the mask boundary coordinates of each object based on the mask data of each object in the instance segmentation annotation file; determining the circumscribed rectangle with the smallest area covering the mask of each object based on the mask boundary coordinates; obtaining the rotation angle of the circumscribed rectangle based on the horizontal coordinate system and determining the rotation matrix; rotating the horizontal coordinate system and the mask boundary points of the circumscribed rectangle, and obtaining the minimum and maximum boundary pixel values ​​of the mask boundary coordinates in the rotated coordinate system based on the rotated mask boundary coordinates; determining the coordinates of the four vertices of the circumscribed rectangle based on the minimum and maximum boundary pixel values ​​of the mask boundary coordinates in the rotated coordinate system; and obtaining the coordinates of the four vertices of the circumscribed rectangle in the horizontal coordinate system based on the inverse matrix of the rotation matrix, thereby generating a rotated annotation box.

[0009] Optionally, determining the circumscribed rectangle with the smallest area covering each object based on the mask boundary coordinates includes: obtaining the smallest convex polygon containing all coordinate points based on the mask boundary coordinates; forming a circumscribed rectangle by placing a pair of parallel lines close to one side of the convex polygon and two other perpendicular calipers respectively touching the farthest point of the convex polygon; recording the rectangle with the smallest area among all possible enclosing directions to obtain the desired circumscribed rectangle with the smallest area.

[0010] Optionally, obtaining the rotation angle of the circumscribed rectangle and determining the rotation matrix based on the horizontal coordinate system includes: obtaining the rotation angle of the circumscribed rectangle based on the angle between the side containing the width of the circumscribed rectangle and the x-axis; wherein the side of the circumscribed rectangle closest to the x-axis is defined as the width; and determining the rotation matrix based on the rotation angle.

[0011] Optionally, it also includes: setting the label of the horizontal annotation box to be consistent with the label of the corresponding mask; setting the label of the rotated annotation box to be consistent with the label of the corresponding mask.

[0012] Optionally, step S204 includes: comparing the vertex coordinates of the generated horizontal annotation box and / or the rotated annotation box with the width and height of the image information to determine whether the vertex coordinates exceed the boundary of the image; saving all the horizontal annotation boxes and / or the rotated annotation boxes located within the image boundary in a recognizable annotation file format, and simultaneously saving the annotation labels of the corresponding objects to obtain a valid multi-format annotation file.

[0013] Optionally, comparing the vertex coordinates of the generated horizontal and / or rotated annotation boxes with the width and height of the image information to determine whether the vertex coordinates exceed the boundary of the image includes: determining whether the horizontal coordinate value of the vertex is greater than or equal to 0 and less than or equal to W; determining whether the vertical coordinate value of the vertex is greater than or equal to 0 and less than or equal to H; wherein W represents the width of the image information and H represents the height of the image information.

[0014] Optionally, the step of saving all the horizontal and / or rotated annotation boxes located within the image boundary in a recognizable annotation file format, and simultaneously saving the corresponding object's annotation label to obtain a valid multi-format annotation file, includes: extracting size and position information based on the filtered horizontal and / or rotated annotation boxes; wherein the size and position information includes the center x-coordinate, center y-coordinate, and four vertex coordinates of the horizontal annotation box, and the center x-coordinate, center y-coordinate, four vertex coordinates, and rotation angle of the rotated annotation box; and determining the valid multi-format annotation file corresponding to the instance segmentation mask recognizable by the annotation software based on the size and position information and simultaneously saving the corresponding object's annotation label.

[0015] Secondly, this application provides a multi-format annotation generation system for construction material detection. The system includes: an acquisition module, used to acquire target images of building materials and perform instance segmentation annotation on each object in the target image to obtain an instance segmentation annotation file; wherein the instance segmentation annotation file includes image information and annotation data for each object, the image information including the image path, image width, and height, and the annotation data including annotation labels and mask data; a first annotation module, used to generate horizontal annotation boxes based on the annotation data of each object in the instance segmentation annotation file, according to a horizontal box detection model; a second annotation module, used to generate rotated annotation boxes based on the annotation data of each object in the instance segmentation annotation file, according to a rotated box detection model; and a generation module, used to constrain the generated horizontal and rotated annotation boxes based on the width and height of the image information, filtering out horizontal and rotated annotation boxes that exceed the image information, to obtain a filtered, valid multi-format annotation file.

[0016] This application has at least the following advantages:

[0017] The above steps mainly involve acquiring target images of building materials taken on-site, manually segmenting and labeling these images to obtain instance segmentation labels, and then generating bounding horizontal and rotated bounding boxes with the smallest area based on these labels. This achieves automatic generation of horizontal and rotated bounding box labels from a unified instance segmentation label file, ensuring consistency in object category, location, and range across different labeling formats. It avoids errors and inconsistencies that may arise from manual multi-round labeling, greatly reducing the workload and time cost of manual labeling, and enabling the rapid preparation of training data for detection models with different needs.

[0018] Secondly, the boundary constraint filtering step based on image size automatically removes invalid annotation boxes whose vertex coordinates exceed the image boundaries, ensuring the validity and usability of the generated multi-format annotation files and improving the quality of subsequent model training data. The efficient and automated generation of multiple annotation formats automatically enriches the annotation formats even with large datasets and limited annotation formats, effectively improving the efficiency of data annotation and enhancing the shared value of the dataset. Attached Figure Description

[0019] Figure 1 This is a diagram illustrating the application environment of a multi-format annotation generation method for construction material detection in one embodiment.

[0020] Figure 2 This is a flowchart illustrating the steps of a method for generating multi-format annotations for construction material inspection in one embodiment.

[0021] Figure 3 This is a schematic diagram showing the horizontal and rotated annotation boxes with the smallest generated area in one embodiment;

[0022] Figure 4 This is a schematic diagram illustrating the process of filtering out annotation boxes that contain more information than the image to obtain a valid multi-format annotation file in one embodiment.

[0023] Figure 5 This is a structural block diagram showing a multi-format annotation generation system for construction material detection in one embodiment;

[0024] Figure 6 This is a schematic structural diagram of a computer device in one embodiment. Detailed Implementation

[0025] The present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the scope of the present application.

[0026] For ease of understanding, the system to which this application applies will first be described. This application provides a multi-format annotation generation method for construction material inspection, which can be applied to, for example... Figure 1The system architecture shown includes a user-space file server 103 and a terminal device 101. The terminal device 101 communicates with the user-space file server 103 via a network. The user-space file server 103 can be a file server based on the NFSv3 / v4 protocol, running in a Linux environment. NFS (Network File System) is a network abstraction on top of a file system, allowing remote clients running on the terminal device 101 to access the file system over the network in a manner similar to a local file system. The terminal device 101 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, etc. The user-space file server 103 can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0027] Figure 2 This application provides a flowchart illustrating a method for generating multi-format annotations for construction material inspection, which may include the following steps:

[0028] S201. Obtain the target image of the building materials, and perform instance segmentation and annotation on each object in the target image to obtain an instance segmentation and annotation file; wherein, the instance segmentation and annotation file includes image information and annotation data for each object, the image information includes the image path, image width and height, and the annotation data includes annotation labels and mask data;

[0029] S202. Based on instance segmentation of the annotation data of each object in the annotation file, generate horizontal annotation boxes according to the horizontal box detection model;

[0030] S203. Based on instance segmentation of the annotation data of each object in the annotation file, generate a rotated annotation box according to the rotated box detection model;

[0031] S204. Constrain the generated horizontal and rotated annotation boxes based on the width and height of the image information, filter out the horizontal and rotated annotation boxes that exceed the image information, and obtain the filtered valid multi-format annotation file.

[0032] In this embodiment, it should be noted that by acquiring target images of building materials taken on-site, instance segmentation and annotation are performed on the target images to obtain instance segmentation and annotation files. Then, based on the instance segmentation and annotations, the bounding horizontal rectangle and the bounding rotated rectangle with the smallest area are generated. Annotation boxes that do not meet the boundary constraints are filtered according to the width and height of the target image and the vertex coordinates of the generated annotation boxes, resulting in valid multi-format annotation files that can be recognized by the annotation software. This achieves efficient and automated generation of multiple annotation formats, automatically enriching the annotation formats when the dataset is large and the annotation format is singular, avoiding errors and inconsistencies that may be caused by manual multi-round annotation, providing model training data for tasks with different needs, effectively improving the efficiency of data annotation, and enhancing the sharing value of the dataset.

[0033] The following is a detailed explanation of each step:

[0034] Please refer to Figure 2 As shown, in step S201, the target image of the building materials is obtained, and each object in the target image is segmented and labeled to obtain an instance segmentation and labeling file;

[0035] In this embodiment, it should be noted that instance segmentation refers to pixel-level annotation of the contour of each object in an image, accurately delineating every pixel of the object's contour. The instance segmentation annotation file includes image information and annotation data for each object. The image information includes the image path, image width, and height, and the annotation data includes the annotation label and mask data for each object.

[0036] Specifically, using Labelme annotation software, manual workers complete instance segmentation annotations on images of building materials taken at the construction site using imaging equipment, and obtain instance segmentation annotation files in JSON format.

[0037] Please refer to Figure 2 , Figure 3 As shown, in step S202, the annotation data of each object in the annotation file is segmented based on the instance, and a horizontal annotation box is generated according to the horizontal box detection model;

[0038] In this embodiment, it should be noted that, based on the mask data of each object in the instance segmentation annotation file, the minimum and maximum boundary pixel values ​​of each object in the horizontal coordinate system are obtained according to the mask boundary coordinates of each object; the circumscribed rectangle with the smallest area covering the mask of each object is determined according to the minimum and maximum boundary pixel values ​​of each object in the horizontal coordinate system; the coordinates of the four vertices of the circumscribed rectangle are determined according to the minimum and maximum boundary pixel values ​​of the mask boundary coordinates in the horizontal coordinate system, and a horizontal annotation box is generated.

[0039] Specifically, based on the mask boundary coordinates, the bounding rectangle with the smallest area covering each object is determined. Among all horizontal enclosing directions, the horizontal bounding rectangle with the smallest area is obtained. Finally, the coordinates of the four vertices of the bounding rectangle are determined based on the minimum and maximum boundary pixel values ​​of the mask boundary coordinates in the horizontal coordinate system, generating a horizontal bounding box. Specifically, in one example, the determination of the four vertices of the bounding rectangle based on the minimum and maximum boundary pixel values ​​of the mask boundary coordinates in the horizontal coordinate system is described by the following formula:

[0040] ;

[0041] Among them, (x) i , y i Let be the coordinates of the i-th boundary point of the mask, and min i x i min i y i max i x i max i y i v represents the minimum and maximum boundaries of the mask in the horizontal coordinate system. j The coordinates of the four vertices of the circumscribed horizontal rectangle are (j=1,2,3,4).

[0042] Please continue to refer to Figure 2 , Figure 3 As shown, in step S203, the annotation data of each object in the annotation file is segmented based on the instance, and a rotated annotation box is generated according to the rotated box detection model;

[0043] In this embodiment, it should be noted that, based on the mask data of each object in the instance segmentation annotation file, the mask boundary coordinates of each object are determined; the circumscribed rectangle with the smallest area covering each object is determined based on the mask boundary coordinates; the rotation angle of the circumscribed rectangle is obtained based on the horizontal coordinate system, and the rotation matrix is ​​determined; the horizontal coordinate system and the mask boundary points of the circumscribed rectangle are rotated, and the minimum and maximum boundary pixel values ​​of the mask boundary coordinates in the rotated coordinate system are obtained based on the rotated mask boundary coordinates; the coordinates of the four vertices of the circumscribed rectangle are determined based on the minimum and maximum boundary pixel values ​​of the mask boundary coordinates in the rotated coordinate system; and the coordinates of the four vertices of the circumscribed rectangle in the horizontal coordinate system are obtained based on the inverse matrix of the rotation matrix, generating a rotated annotation box.

[0044] In this embodiment, it should also be noted that the circumscribed rectangle with the smallest area covering each object is determined based on the mask boundary coordinates. Specifically, the smallest convex polygon containing all coordinate points is obtained based on the mask boundary coordinates; a circumscribed rectangle is formed by placing a pair of parallel lines close to one side of the convex polygon and placing two other perpendicular calipers to the farthest point of the convex polygon; and the rectangle with the smallest area is recorded in all possible enclosing directions to obtain the desired circumscribed rectangle with the smallest area.

[0045] The rotation angle of the circumscribed rectangle and the determination of the rotation matrix are obtained based on the horizontal coordinate system. Specifically, the rotation angle of the circumscribed rectangle is determined by the angle between the side containing its width and the x-axis in the horizontal coordinate system; the side of the circumscribed rectangle closest to the x-axis is defined as its width. The rotation matrix is ​​then determined based on the rotation angle. Finally, the coordinates of the four vertices of the circumscribed rectangle in the horizontal coordinate system are obtained from the inverse of the rotation matrix, generating a rotation annotation box, which is described by the following formula:

[0046] ;

[0047] In the formula, (x i , y i () represents the coordinates of the i-th boundary point of the mask. θ Let be the rotation angle of the bounding rectangle with the smallest area. R ( θ ) is a rotation matrix, ( C x ,C y () represents the coordinates of the center point of the rectangle. Rotation of the horizontal coordinate system θ The coordinates of the i-th boundary point of the mask are as follows. Let j be the coordinates of the four vertices of the circumscribed rotating rectangle in the horizontal coordinate system (j=1,2,3,4). , , , The minimum and maximum boundaries of the mask after rotation by θ in the horizontal coordinate system.

[0048] Here, the horizontal and rotated bounding boxes are generated by calculating the minimum bounding rectangle of the mask. In particular, the rotational caliper method is used to find the minimum bounding rectangle, which closely fits the object's outline and generates high-quality rotated annotation boxes. This is especially suitable for irregularly or obliquely arranged materials in construction scenarios. Specifically, such as... Figure 3 As shown, this paper uses six commonly used construction materials on site—steel bars, square steel pipes, timber, I-beams, angle steel, and ring fasteners—as examples to demonstrate the method for generating multi-format annotations. It also illustrates the correspondence between instance segmentation masks, horizontal annotation frames, and rotated annotation frames for these six construction materials.

[0049] In addition, based on the generated horizontal and / or rotated annotation boxes, the method further includes: setting the label of the horizontal annotation box to be consistent with the label of the corresponding mask; and setting the label of the rotated annotation box to be consistent with the label of the corresponding mask.

[0050] In this embodiment, it should also be noted that after generating the horizontal and rotated annotation boxes, the annotation labels corresponding to each object in the image should be matched with the corresponding horizontal or rotated annotation boxes so as to form a valid annotation file and achieve visual inspection.

[0051] Reference Figure 2 , Figure 4 As shown, in step S204, the generated horizontal and rotated annotation boxes are constrained based on the width and height of the image information, and annotation boxes that exceed the image information are filtered out to obtain the filtered valid multi-format annotation file.

[0052] In this embodiment, it should be noted that the vertex coordinates of the generated horizontal and / or rotated annotation boxes are compared with the width and height of the image information to determine whether the vertex coordinates exceed the image boundary; all horizontal and / or rotated annotation boxes located within the image boundary are saved in a recognizable annotation file format, and the corresponding object's annotation label is saved at the same time to obtain a valid multi-format annotation file.

[0053] In some embodiments, the vertex coordinates of the generated horizontal and / or rotated bounding boxes are compared with the width and height of the image information to determine whether the vertex coordinates exceed the boundaries of the image. This mainly involves determining whether the x-coordinate value of the vertex is greater than or equal to 0 and less than or equal to W; and whether the y-coordinate value of the vertex is greater than or equal to 0 and less than or equal to H. Here, W represents the width of the image information and H represents the height of the image information.

[0054] Specifically, in one example, based on the coordinates of the four vertices of the horizontal and rotated annotation boxes, it is determined that the vertex coordinates do not exceed the image boundaries. This requires that the x-coordinate of the vertex is greater than or equal to 0 and less than or equal to W, and the y-coordinate is greater than or equal to 0 and less than or equal to H. This is described by the following formula:

[0055] ;

[0056] ;

[0057] In the formula, v j Let j = the coordinates of the four vertices of the horizontal label box (j=1,2,3,4), min i x i min i y i maxi x i max i y i These are the minimum and maximum boundaries of the mask in the horizontal coordinate system;

[0058] And / or,

[0059] ;

[0060] ；

[0061] In the formula, (j=1,2,3,4) are the four vertices of the rotated label box.

[0062] In addition, all horizontal and / or rotated annotation boxes located within the image boundary are saved in a recognizable annotation file format, and the corresponding object annotation labels are also saved to obtain a valid multi-format annotation file. This includes: extracting size and position information from the filtered horizontal and / or rotated annotation boxes; and determining the valid multi-format annotation file corresponding to the instance segmentation mask that the annotation software can recognize based on the size and position information and the corresponding object annotation labels.

[0063] In this embodiment, it should be noted that the saved size and position information includes, for horizontal bounding boxes, typically the coordinates of their center point and four vertices, and must also include their label. For rotated bounding boxes, typically the coordinates of their center point, rotation angle, and four vertices, and must also include their label. The final output is a valid multi-format (horizontal bounding box, rotated bounding box) annotation file corresponding to the original instance segmentation annotations, facilitating visualization and inspection. Specifically, the horizontal bounding box visualization software used can be LabelImg, and the saved file format can be an XML file containing annotation data such as the minimum and maximum boundary pixel values ​​of the horizontal bounding box; the rotated bounding box visualization software used can be roLabelImg, and the saved file format is an XML file containing annotation data such as the center point coordinates, width and height of the rotated bounding box, and rotation angle. This approach offers good versatility and flexibility, effectively reducing manual annotation costs, achieving efficient automated generation of multiple annotation formats, and improving the reusability and sharing efficiency of the dataset.

[0064] The implementation principle of this embodiment is as follows: The above steps mainly involve acquiring target images of building materials taken on-site, manually segmenting and labeling these images to obtain instance segmentation and labeling files, providing a data foundation for subsequent multi-format labeling. Based on the instance segmentation and labeling, bounding horizontal and rotated bounding boxes with the smallest area are generated. Labeling boxes that do not meet boundary constraints are filtered based on the width and height of the target image and the vertex coordinates of the generated boxes. The filtered valid labels are saved as valid multi-format labeling files recognizable by the labeling software. This provides labeling files that can be further viewed in the labeling software, allowing users to check whether the labeling quality meets training requirements. In cases of large datasets and a single labeling format, the labeling formats are automatically enriched, achieving efficient and automated generation of multiple labeling formats. This provides model training data for tasks with different needs, improves the reusability and sharing efficiency of the dataset, and reduces the cost of manual labeling.

[0065] Reference Figure 5 As shown in the figure, this application embodiment also provides a multi-format annotation generation system for construction material detection. This system may include: an acquisition module 301, a first annotation module 302, a second annotation module 303, and a generation module 304; the main functions of each component module are as follows:

[0066] The acquisition module 301 is used to acquire a target image of building materials and perform instance segmentation and annotation on each object in the target image to obtain an instance segmentation and annotation file; wherein, the instance segmentation and annotation file includes image information and annotation data for each object, the image information includes the image path, image width and height, and the annotation data includes annotation labels and mask data;

[0067] The first annotation module 302 is used to segment the annotation data of each object in the annotation file based on the instance, and generate horizontal annotation boxes according to the horizontal box detection model;

[0068] The second annotation module 303 is used to segment the annotation data of each object in the annotation file based on the instance, and generate a rotated annotation box according to the rotated box detection model;

[0069] The generation module 304 is used to constrain the generated horizontal annotation box and the rotated annotation box based on the width and height of the image information, filter out the horizontal annotation box and the rotated annotation box that exceed the image information, and obtain the filtered valid multi-format annotation file.

[0070] like Figure 6The diagram shown is a block diagram of a computer device according to an embodiment of this application. The term "computer device" is intended to represent various forms of digital computers or mobile devices. The digital computer may include a desktop computer, a portable computer, a workbench, a personal digital assistant, a server, a mainframe computer, and other suitable computers. The mobile device may include a tablet computer, a smartphone, a wearable device, etc.

[0071] like Figure 6 As shown, device 600 includes a computing unit 601, a ROM 602, a RAM 603, a bus 604, and an I / O interface 605. The computing unit 601, ROM 602, and RAM 603 are interconnected via the bus 604. The I / O interface 605 is also connected to the bus 604.

[0072] The computing unit 601 can execute various processes in the method embodiments of this application according to computer instructions stored in the read-only memory (ROM) 602 or computer instructions loaded from the storage unit 608 into the random access memory (RAM) 603. The computing unit 601 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. The computing unit 601 can include, but is not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. In some embodiments, the methods provided in the embodiments of this application can be implemented as computer software programs, which are tangibly contained in a computer-readable storage medium, such as the storage unit 608.

[0073] RAM 603 can also store various programs and data required for the operation of device 600. Part or all of the computer program can be loaded and / or installed on device 600 via ROM 602 and / or communication unit 609.

[0074] The input unit 606, output unit 607, storage unit 608, and communication unit 609 in device 600 can be connected to I / O interface 605. The input unit 606 can be, for example, a keyboard, mouse, touchscreen, or microphone; the output unit 607 can be, for example, a display, speaker, or indicator light. Device 600 can exchange information and data with other devices through the communication unit 609.

[0075] It should be noted that the device may also include other components necessary for normal operation. It may also include only the components necessary for implementing the solution of this application, without necessarily including all the components shown in the figures.

[0076] Various implementations of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SOCs), payload programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof.

[0077] The computer instructions used to implement the methods of this application may be written in any combination of one or more programming languages. These computer instructions may be provided to the computing unit 601 such that when executed by the computing unit 601, such as a processor, the computer instructions cause the execution of the steps involved in the embodiments of the methods of this application.

[0078] The computer-readable storage medium provided in this application can be a tangible medium that can contain or store computer instructions for performing the steps involved in the method embodiments of this application. The computer-readable storage medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, and other forms of storage media.

[0079] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for generating multi-format annotations for construction material inspection, characterized in that, Includes the following steps: S201. Obtain the target image of the building materials, and perform instance segmentation and annotation on each object in the target image to obtain an instance segmentation and annotation file; wherein, the instance segmentation and annotation file includes image information and annotation data for each object, the image information includes the path, width and height of the image, and the annotation data includes annotation labels and mask data; S202. Based on the instance segmentation annotation data of each object in the annotation file, generate horizontal annotation boxes according to the horizontal box detection model; including: Based on the mask data of each object in the instance segmentation annotation file, the minimum and maximum boundary pixel values ​​of each object in the horizontal coordinate system are obtained according to the mask boundary coordinates of each object; the bounding rectangle with the smallest area covering the mask of each object is determined according to the minimum and maximum boundary pixel values ​​of the object in the horizontal coordinate system; the coordinates of the four vertices of the bounding rectangle are determined according to the minimum and maximum boundary pixel values ​​of the mask boundary coordinates in the horizontal coordinate system, and a horizontal annotation box is generated. S203. Based on the instance segmentation of the annotation data for each object in the annotation file, and according to the rotated bounding box detection model, generate a rotated annotation box; including: Based on the mask data of each object in the instance segmentation annotation file, determine the mask boundary coordinates of each object; determine the circumscribed rectangle with the smallest area covering the mask of each object based on the mask boundary coordinates; obtain the rotation angle of the circumscribed rectangle based on the horizontal coordinate system and determine the rotation matrix; rotate the horizontal coordinate system and the mask boundary points of the circumscribed rectangle, and obtain the minimum and maximum boundary pixel values ​​of the mask boundary coordinates in the rotated coordinate system based on the rotated mask boundary coordinates; determine the coordinates of the four vertices of the circumscribed rectangle in the rotated coordinate system based on the minimum and maximum boundary pixel values ​​of the mask boundary coordinates in the rotated coordinate system; obtain the coordinates of the four vertices of the circumscribed rectangle in the horizontal coordinate system based on the inverse matrix of the rotation matrix based on the coordinates of the four vertices of the circumscribed rectangle in the rotated coordinate system, and generate a rotated annotation box; S204. Constrain the generated horizontal annotation box and the rotated annotation box based on the width and height of the image information, filter out the horizontal annotation box and the rotated annotation box that exceed the image information, and obtain the filtered valid multi-format annotation file.

2. The method according to claim 1, characterized in that, Determining the circumscribed rectangle with the smallest area covering the mask for each object based on the mask boundary coordinates includes: The smallest convex polygon containing all coordinate points is obtained based on the mask boundary coordinates; A circumscribed rectangle is formed by placing a pair of parallel lines close to one side of the convex polygon and two other perpendicular calipers to the farthest point of the convex polygon. Record the rectangle with the smallest area among all possible enclosing directions to obtain the desired bounding rectangle with the smallest area.

3. The method according to claim 1, characterized in that, The step of obtaining the rotation angle of the circumscribed rectangle based on the horizontal coordinate system and determining the rotation matrix includes: The rotation angle of the circumscribed rectangle is obtained based on the angle between the side containing the width of the circumscribed rectangle and the x-axis, using a horizontal coordinate system; wherein, the width of the circumscribed rectangle is defined relative to the side closest to the x-axis. The rotation matrix is ​​determined based on the rotation angle.

4. The method according to claim 1, characterized in that, Also includes: Set the label of the horizontal label box to be consistent with the label of the corresponding mask; Set the label of the rotated label box to be consistent with the label of the corresponding mask.

5. The method according to claim 4, characterized in that, S204 includes: The vertex coordinates of the generated horizontal and / or rotated annotation boxes are compared with the width and height of the image information to determine whether the vertex coordinates exceed the boundaries of the image. All horizontal and / or rotated annotation boxes located within the image boundary are saved in a recognizable annotation file format, and the corresponding annotation labels of the objects are saved at the same time to obtain a valid multi-format annotation file.

6. The method according to claim 5, characterized in that, The step of determining whether the vertex coordinates exceed the image boundary includes: Determine whether the x-coordinate value of the vertex is greater than or equal to 0 and less than or equal to W; Determine whether the ordinate value of the vertex is greater than or equal to 0 and less than or equal to H; Wherein, W represents the width of the image information, and H represents the height of the image information.

7. The method according to claim 5, characterized in that, The step of saving all the horizontal and / or rotated annotation boxes located within the image boundary in a recognizable annotation file format, and simultaneously saving the corresponding object's annotation label, to obtain a valid multi-format annotation file, includes: Dimension position information is extracted from the filtered horizontal annotation box and / or the rotated annotation box; wherein, the dimension position information includes the center x-coordinate, center y-coordinate, and four vertex coordinates of the horizontal annotation box, and the center x-coordinate, center y-coordinate, four vertex coordinates, and rotation angle of the rotated annotation box; Based on the size and position information, and simultaneously saving the corresponding object's annotation label, determine the valid multi-format annotation file corresponding to the instance segmentation mask that the annotation software can recognize.

8. A multi-format labeling generation system for construction material inspection, characterized in that, include: The acquisition module is used to acquire target images of building materials and perform instance segmentation and annotation on each object in the target image to obtain an instance segmentation and annotation file; wherein, the instance segmentation and annotation file includes image information and annotation data for each object, the image information includes the path, width and height of the image, and the annotation data includes annotation labels and mask data; The first annotation module is used to: segment the annotation data of each object in the annotation file based on the instance; generate a horizontal annotation box according to the horizontal box detection model; obtain the minimum and maximum boundary pixel values ​​of each object in the horizontal coordinate system based on the mask data of each object in the instance segmented annotation file; determine the circumscribed rectangle with the smallest area covering the mask of each object based on the minimum and maximum boundary pixel values ​​of the object in the horizontal coordinate system; and determine the coordinates of the four vertices of the circumscribed rectangle based on the minimum and maximum boundary pixel values ​​of the mask boundary coordinates in the horizontal coordinate system, thereby generating the horizontal annotation box. The second annotation module is used to: segment the annotation data of each object in the annotation file based on the instance; generate a rotated annotation box according to the rotated bounding box detection model; determine the mask boundary coordinates of each object based on the mask data of each object in the segmented annotation file based on the instance; determine the circumscribed rectangle with the smallest area covering the mask of each object based on the mask boundary coordinates; obtain the rotation angle of the circumscribed rectangle based on the horizontal coordinate system and determine the rotation matrix; rotate the horizontal coordinate system and the mask boundary points of the circumscribed rectangle; obtain the minimum and maximum boundary pixel values ​​of the mask boundary coordinates in the rotated coordinate system based on the rotated mask boundary coordinates; determine the coordinates of the four vertices of the circumscribed rectangle in the rotated coordinate system based on the minimum and maximum boundary pixel values ​​of the mask boundary coordinates in the rotated coordinate system; and obtain the coordinates of the four vertices of the circumscribed rectangle in the horizontal coordinate system based on the inverse matrix of the rotation matrix, and generate the rotated annotation box. The generation module is used to constrain the generated horizontal annotation box and the rotated annotation box based on the width and height of the image information, filter out the horizontal annotation box and the rotated annotation box that exceed the image information, and obtain the filtered valid multi-format annotation file.

Images