A visualization method and system for intelligently labeling non-living objects

Abstract

The present invention discloses a method and system for intelligently labeling non-living objects. The method includes extracting and collecting object information, including text information, a one-dimensional sampling data set, and image information; converting the text information into a text pixel file and the one-dimensional sampling data into a one-dimensional sampling data pixel file based on a pixel matrix conversion algorithm; and generating an annotated graph from the text pixel file and the one-dimensional sampling pixel file using a spiral layout. This application comprehensively considers the main sources of non-living information and generates standardized annotated graphs in a unified manner, providing powerful automated assistance for the production of deep learning materials. The annotated graph can be constructed and displayed based on intelligent grouping of data features.

Description

Technical Field

[0001] The present invention relates to the fields of data encryption and data graphical analysis, and in particular to a visualization method and system for intelligently marking non-living objects. Background Art

[0002] As deep learning is widely applied in industrial manufacturing, materials, and machine vision projects, a large amount of deep learning material needs to be collected and organized. This material primarily consists of visual information. Existing technologies lack a unified and standardized method for generating annotated images, making it difficult to provide automated assistance for the production of deep learning material. Summary of the Invention

[0003] In order to solve the above technical problems existing in the prior art, the present invention proposes a visualization method and system for intelligently labeling non-living objects to solve the above technical problems.

[0004] According to one aspect of the present invention, a visualization method for intelligently labeling non-living objects is proposed, comprising:

[0005] S1: Extract and collect object information, which includes text information, one-dimensional sampling data set and image information;

[0006] S2: Based on the pixel matrix conversion algorithm, the text information is converted into a text pixel file and the one-dimensional sampling data is converted into a one-dimensional sampling data pixel file;

[0007] S3: Use spiral layout to generate annotation graph from text pixel file and one-dimensional sampling pixel file.

[0008] In some specific embodiments, the pixel matrix conversion algorithm includes a text information conversion pixel matrix algorithm and a one-dimensional sampling data set conversion pixel matrix algorithm.

[0009] In some specific embodiments, the one-dimensional sampling data set is a single-column data table whose primary key is the sampling time, and the data column value is a 32-bit floating point value.

[0010] In some specific embodiments, the text information conversion pixel matrix algorithm includes:

[0011] All submitted text information files are processed using natural language processing algorithms to generate a hot word index table, which is sorted by the frequency of hot words.

[0012] Convert each hot word into a UNCODE16 string, convert the frequency of occurrence into a 16-bit binary code Q, and generate a hot word file.

[0013] Generate pixel data stream and save it as a text pixel file.

[0014] In some specific embodiments, the text pixel file is composed of RGBA32 pixels, each UNCODE16 encoded character corresponds to RG, BA corresponds to the occurrence frequency Q, and the text pixel file storage format is {RGBA, RGAB...}.

[0015] In some specific embodiments, the one-dimensional sampling data set conversion pixel matrix algorithm includes:

[0016] Convert the time in all submitted one-dimensional sampling data set files into millisecond deviation integers and store them as 16-bit binary integers A, and extract the data floating point numbers into 16-bit binary integers B according to the memory format and store them;

[0017] Map integer A to RG and integer B to BA to form a pixel point information and save it as a one-dimensional sampling data pixel file.

[0018] In some specific embodiments, the storage format of the one-dimensional sampling data pixel file is the same as that of the text pixel file.

[0019] In some specific embodiments, S3 specifically includes:

[0020] Search the display width and height of all image information files, and find a group of image files with the largest width and height as the central image of the spiral layout;

[0021] Other image files are placed around the central image to form a peripheral frame;

[0022] Rotate and fill the one-dimensional sampling data pixel file and text pixel file around the central image;

[0023] According to the display ratio of the visualization area, compress the image, complete the annotation, and convert the format for storage.

[0024] According to a second aspect of the present invention, a computer-readable storage medium is provided, on which one or more computer programs are stored. When the one or more computer programs are executed by a computer processor, any one of the above methods is implemented.

[0025] According to a third aspect of the present invention, a visualization system for intelligently labeling non-living objects is proposed, the system comprising:

[0026] a data acquisition unit configured to extract and collect object information, the object information including text information, a one-dimensional sampling data set, and image information;

[0027] Pixel matrix conversion unit: configured to convert text information into text pixel files and convert one-dimensional sampling data into one-dimensional sampling data pixel files based on a pixel matrix conversion algorithm;

[0028] Annotation map generating unit: configured to generate an annotation map from a text pixel file and a one-dimensional sampling pixel file using a spiral layout.

[0029] In some specific embodiments, the pixel matrix conversion algorithm includes a text information conversion pixel matrix algorithm and a one-dimensional sampling data set conversion pixel matrix algorithm, wherein the text information conversion pixel matrix algorithm includes:

[0030] All submitted text information files are processed using natural language processing algorithms to generate a hot word index table, which is sorted by the frequency of hot words.

[0031] Convert each hot word into a UNCODE16 string, convert the frequency of occurrence into a 16-bit binary code Q, and generate a hot word file.

[0032] Generate pixel data stream and save it as text pixel file;

[0033] The algorithm for converting one-dimensional sampled data sets into pixel arrays includes:

[0034] Convert the time in all submitted one-dimensional sampling data set files into millisecond deviation integers and store them as 16-bit binary integers A, and extract the data floating point numbers into 16-bit binary integers B according to the memory format and store them;

[0035] Map integer A to RG and integer B to BA to form a pixel point information and save it as a one-dimensional sampling data pixel file.

[0036] In some specific embodiments, the one-dimensional sampling data set is a single-column data table whose primary key is the sampling time, the data column value is a 32-bit floating-point value, the text pixel file is composed of RGBA32 pixels, each UNCODE16 encoded character corresponds to RG, BA corresponds to the occurrence frequency Q, and the text pixel file storage format is {RGBA, RGAB...}.

[0037] In some specific embodiments, the annotated image generation unit is specifically configured to search the display width and height of all image information files, identify a group of image files with the largest width and height, and use them as the central image of the spiral layout; place other image files around the central image to form a peripheral image frame; rotate and fill the central image with the one-dimensional sampled data pixel file and the text pixel file; compress the images according to the display ratio of the visualization area, complete the annotated image, and convert the format for storage.

[0038] The present invention proposes a visualization method and system for intelligently labeling non-living objects. For the labeling of any non-living object, it is necessary to first extract and collect a batch of object information, and then generate and display a labeled image through the method or system of the present invention. The labeled image can be constructed and displayed by intelligently grouping data features, providing powerful automated assistance for the production of deep learning materials. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings are included to provide a further understanding of the embodiments and are incorporated into and constitute a part of this specification. The accompanying drawings illustrate the embodiments and, together with the description, serve to explain the principles of the invention. Other embodiments and many of the intended advantages of the embodiments will be readily apparent as they become better understood by reference to the following detailed description. Other features, objects, and advantages of the present application will become more apparent by reading the detailed description of the non-limiting embodiments made with reference to the following drawings:

[0040] Figure 1 This is a flowchart of a visualization method for intelligently labeling non-living objects according to an embodiment of the present application;

[0041] Figure 2 This is a data annotation process diagram of a specific embodiment of the present application;

[0042] Figure 3 This is a flowchart of intelligent data annotation according to a specific embodiment of the present application;

[0043] Figure 4 This is a framework diagram of a visualization system for intelligently labeling non-living objects according to an embodiment of the present application;

[0044] Figure 5 It is a structural diagram of a computer system suitable for implementing the electronic device of the embodiment of the present application. DETAILED DESCRIPTION

[0045] The present application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the relevant invention and are not intended to limit the invention. It should also be noted that, for ease of description, only portions relevant to the relevant invention are shown in the accompanying drawings.

[0046] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0047] According to an embodiment of the present application, a method for intelligently labeling a non-living object is used for visualization. Figure 1 FIG. 1 shows a flow chart of a method for intelligently labeling a non-living object according to an embodiment of the present application. Figure 1 As shown, the method includes:

[0048] S101: Extract and collect object information, which includes text information, one-dimensional sampling data set, and image information. The one-dimensional sampling data set is a single-column data table with the primary key being the sampling time. The data column value is a 32-bit floating-point value, and the specific format is as follows:

[0049] {

[0050] {Record 1: time t, data} ...

[0052] }

[0053] S102: Convert text information into text pixel files and convert one-dimensional sampling data into one-dimensional sampling data pixel files based on pixel matrix conversion algorithms. The pixel matrix conversion algorithms include a text information to pixel matrix conversion algorithm and a one-dimensional sampling data set to pixel matrix conversion algorithm.

[0054] In a specific embodiment, the text information conversion pixel matrix algorithm includes the following steps:

[0055] In the first step, all submitted text information files are processed using the natural language processing algorithm NLP to generate a hot word index table, which is sorted according to the frequency of occurrence of hot words.

[0056] The second step is to convert the hot words into UNCODE16 encoded String one by one, convert the frequency of occurrence into 16-bit binary code Q, and generate a hot word file

[0057] The hot word file storage format is:

[0058]

[0059] The third step is to generate a pixel data stream and save it as a text pixel file. The text pixel file consists of RGBA32 pixels. Each UNCODE16 character corresponds to RG, and BA corresponds to the frequency of occurrence Q. The text pixel file is stored in the format of {RGBA, RGAB...}.

[0060] In a specific embodiment, the one-dimensional sampling data set conversion pixel matrix algorithm includes the following steps:

[0061] The first step is to convert the time in all submitted one-dimensional sampling data set files into millisecond deviation integers and store them as 16-bit binary integers A. The floating point data is extracted and stored as 16-bit binary integers B according to the memory format.

[0062] The second step is to map integer A to RG and integer B to BA to form a pixel point information and save it as a one-dimensional sampling data pixel file;

[0063] In a specific embodiment, the storage format of the one-dimensional sampling data pixel file and the text pixel file is the same. With this same storage format, the same format of the annotation graph can be generated and output in the subsequent steps.

[0064] S103: Using a spiral layout to generate a labeled image from the text pixel file and the one-dimensional sampling pixel file. The spiral layout is implemented using an overall intelligent layout evaluation algorithm, specifically including the following steps:

[0065] The first step is to search the display width and height of all image information files, find the first group of image files with the largest width and height, and use them as the central image of the spiral layout;

[0066] In the second step, other image files are placed around the central image to form a peripheral frame;

[0067] The third step is to rotate and fill the one-dimensional sampling data pixel file and the text pixel file around the image;

[0068] The fourth step is to compress the image according to the display ratio of the visual area of ​​the device, complete the annotation map, and convert it to PNG format for storage.

[0069] The visualization method for intelligently labeling non-living objects in this application comprehensively considers the three main sources of non-living information, standardizes the rule information of non-living objects to generate a labeling map, and constructs a labeling map for display based on intelligent grouping of data features, providing a powerful automated auxiliary device for the production of deep learning materials.

[0070] Continue to refer Figure 4 , Figure 4 A framework diagram of a visualization system for intelligently labeling non-living objects according to an embodiment of the present invention is shown. The system specifically includes a data acquisition unit 401, a pixel matrix conversion unit 402, and an annotation graph generation unit 403. The data acquisition unit 401 is configured to extract and collect object information, including text information, a one-dimensional sampling data set, and image information; the pixel matrix conversion unit 402 is configured to convert text information into a text pixel file and one-dimensional sampling data into a one-dimensional sampling data pixel file based on a pixel matrix conversion algorithm; and the annotation graph generation unit 403 is configured to generate an annotation graph from the text pixel file and the one-dimensional sampling pixel file using a spiral layout.

[0071] In a specific embodiment, Figure 2 A data annotation process diagram according to a specific embodiment of the present application is shown in FIG. Figure 2As shown, the system will automatically select the most representative sample data from the unlabeled dataset, which is also the one that needs to be labeled first. Users need to manually label these recommended sample data. In order to improve labeling efficiency, the system will also provide pre-labeling for users to modify and confirm. In the AI ​​model's image intelligent labeling, users can select "Show pre-labeling" to turn on this auxiliary function, and select "Satisfied with pre-labeling results" to confirm the pre-labeling results; in text intelligent labeling, the system will automatically display the pre-labeling label. Select "Confirm" or "Confirm all on this page" on the right side of each text to confirm the pre-labeling.

[0072] Figure 3 A schematic diagram of the process of intelligent data annotation according to a specific embodiment of the present application is shown in FIG. Figure 3 As shown, after the data set is created, smart labeling is started, the recommended labeling data is filtered (partially labeled data is labeled), the smart labeling is ended, the data set labeling is completed and the results are sent to the training model. In a specific embodiment, after confirming the pre-labeling of all recommended data, the user can independently choose whether to proceed to the next round of data screening. In image smart labeling, if the user does not terminate smart labeling, the system will automatically proceed to the next round; in text smart labeling, since the scale of text data sets is generally large, the labor cost of confirming data pre-labeling is high. In order to improve the user experience, the system does not default to entering the next round of iteration, and the user can choose "Optimize smart labeling results" to proceed to the next round of screening. Through multiple rounds of screening, the accuracy of data pre-labeling will continue to improve. In order to ensure the intelligence of data labeling, it is recommended that users perform at least one round of data screening or "optimize smart labeling".

[0073] Reference below Figure 5 , which shows a structural diagram of a computer system suitable for implementing an electronic device of an embodiment of the present application. Figure 5 The electronic device shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0074] like Figure 5 As shown, the computer system includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage unit 508 into a random access memory (RAM) 503. Various programs and data required for the operation of the system 500 are also stored in the RAM 503. The CPU 501, ROM 502, and RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0075] The following components are connected to the I / O interface 505: an input section 506 including a keyboard, a mouse, and the like; an output section 507 including a liquid crystal display (LCD) and speakers; a storage section 508 including a hard disk; and a communication section 509 including a network interface card such as a LAN card or a modem. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 510 as needed, so that computer programs read therefrom can be installed into the storage section 508 as needed.

[0076] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 509, and / or installed from the removable medium 511. When the computer program is executed by the central processing unit (CPU) 501, the above-mentioned functions defined in the method of the present application are executed. It should be noted that the computer-readable storage medium of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including, but not limited to, an electromagnetic signal, an optical signal, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable storage medium other than a computer-readable storage medium that can transmit, propagate, or transfer a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0077] Computer program code for performing the operations of the present application can be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0078] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0079] The modules described in the embodiments of the present application may be implemented by software or hardware.

[0080] As another aspect, the present application also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiment; or it may exist independently and not be assembled into the electronic device. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed by the electronic device, the electronic device: generates a metadata warehouse file using a metadata compression algorithm, wherein the metadata compression algorithm includes the following steps: extracting and collecting object information, the object information including text information, a one-dimensional sampling data set, and image information; converting the text information into a text pixel file and the one-dimensional sampling data into a one-dimensional sampling data pixel file based on a pixel matrix conversion algorithm; and generating a labeling diagram from the text pixel file and the one-dimensional sampling pixel file using a spiral layout.

[0081] The above description is merely a preferred embodiment of the present application and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also encompasses other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned inventive concept. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A visualization method for intelligently labeling non-living objects, characterized in that: include: S1: extracting and collecting object information, wherein the object information includes text information, a one-dimensional sampling data set, and image information; S2: Based on a pixel matrix conversion algorithm, the text information is converted into a text pixel file, and the one-dimensional sampling data is converted into a one-dimensional sampling data pixel file; S3: Generate a labeling graph from the text pixel file and the one-dimensional sampling pixel file using a spiral layout; The pixel matrix conversion algorithm includes a text information conversion pixel matrix algorithm and a one-dimensional sampling data set conversion pixel matrix algorithm; The text information conversion pixel matrix algorithm includes: Generate a hot word index table using a natural language processing algorithm for all submitted text information, and sort the hot words according to their frequency of occurrence; Convert the hot words one by one into UNCODE16 encoded String, convert the occurrence frequency into 16-bit binary code Q, and generate a hot word file; Generate pixel data stream and save it as text pixel file; The one-dimensional sampling data set conversion pixel matrix algorithm includes: Convert the time in all submitted one-dimensional sampling data sets into millisecond deviation integers and store them as 16-bit binary integers A, and extract the data floating point numbers into 16-bit binary integers B according to the memory format and store them; Map integer A to RG and integer B to BA to form a pixel point information and save it as a one-dimensional sampling data pixel file.

2. The method for intelligently labeling a non-living object according to claim 1, characterized in that: The one-dimensional sampling data set is a single-column data table whose primary key is the sampling time, and the data column value is a 32-bit floating point value.

3. The method for intelligently labeling a non-living object according to claim 1, characterized in that: The text pixel file is composed of RGBA32 pixel points, each UNCODE16 encoded character corresponds to RG, and BA corresponds to the frequency of occurrence Q. The storage format of the text pixel file is {RGBA, RGAB...}.

4. The method for intelligently labeling a non-living object according to claim 1, characterized in that: The storage format of the one-dimensional sampling data pixel file is the same as that of the text pixel file.

5. The method for intelligently labeling a non-living object according to claim 1, characterized in that: The S3 specifically includes: Searching the display width and height of all image information files, finding a group of image files with the largest width and height as the central image of the spiral layout; Other image files are placed around the central image to form a peripheral image frame; Rotate and fill the one-dimensional sampling data pixel file and the text pixel file around the central image; According to the display ratio of the visualization area, compress the image, complete the annotation, and convert the format for storage.

6. A computer-readable storage medium having one or more computer programs stored thereon, characterized in that: When the one or more computer programs are executed by a computer processor, the method according to any one of claims 1 to 5 is implemented.

7. A visualization system for intelligently labeling non-living objects, characterized in that: The system comprises: a data acquisition unit configured to extract and collect object information, wherein the object information includes text information, a one-dimensional sampling data set, and image information; A pixel matrix conversion unit: configured to convert the text information into a text pixel file and convert the one-dimensional sampling data into a one-dimensional sampling data pixel file based on a pixel matrix conversion algorithm; A labeling graph generating unit is configured to generate a labeling graph from the text pixel file and the one-dimensional sampling pixel file using a spiral layout; The pixel matrix conversion algorithm includes a text information conversion pixel matrix algorithm and a one-dimensional sampling data set conversion pixel matrix algorithm, wherein the text information conversion pixel matrix algorithm includes: Generate a hot word index table using a natural language processing algorithm for all submitted text information, and sort the hot words according to their frequency of occurrence; Convert the hot words one by one into UNCODE16 encoded String, convert the occurrence frequency into 16-bit binary code Q, and generate a hot word file; Generate pixel data stream and save it as text pixel file; The one-dimensional sampling data set conversion pixel matrix algorithm includes: Convert the time in all submitted one-dimensional sampling data sets into millisecond deviation integers and store them as 16-bit binary integers A, and extract the data floating point numbers into 16-bit binary integers B according to the memory format and store them; Map integer A to RG and integer B to BA to form a pixel point information and save it as a one-dimensional sampling data pixel file.

8. The intelligent non-living body visualization system according to claim 7, characterized in that: The one-dimensional sampling data set is a single-column data table whose primary key is the sampling time, and the data column value is a 32-bit floating point value. The text pixel file is composed of RGBA32 pixels, each UNCODE16 encoded character corresponds to RG, and BA corresponds to the occurrence frequency Q. The text pixel file storage format is {RGBA, RGAB...}.

9. The intelligent non-living body visualization system according to claim 7, characterized in that: The annotation image generation unit is specifically configured to search the display width and height of all image information files, find a group of image files with the largest width and height, and use them as the central image of the spiral layout; place other image files around the central image to form a peripheral frame; rotate and fill the one-dimensional sampling data pixel file and the text pixel file around the central image; compress the image according to the display ratio of the visualization area, complete the annotation image, and convert the format for storage.

Images