Image data management method and system and computer readable storage medium

Abstract

The invention discloses a natural environment test image data management method and system based on an inspection robot, and a computer readable storage medium. The method comprises the following steps: inputting natural environment test sample information into a sample information recording table; making an inspection task; issuing the inspection task to an inspection robot, and enabling the inspection robot to carry out data collection according to the inspection task; receiving a data acquisition result returned by the robot; performing image recognition on a data acquisition result; and carrying out data integration and structured storage according to an identification result. According to the method, the labor cost can be effectively reduced, the data processing efficiency is improved, and the value of the structured data formed after treatment is remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the field of natural environment tests, and particularly relates to a method, a system, and a computer-readable storage medium for governing natural environment test image data based on an inspection robot. Background Art

[0002] The intelligent inspection robot system realizes the automatic acquisition of image data. For example, in the patent application with the application number 202310530964.6, a method for optimizing natural environment test image acquisition, an acquisition method, and a storage medium are provided. However, in this patent application, the acquired image data is unstructured data, which is difficult to manage and use. The governance of environmental effect image data has always been a difficult point in the industry. Currently, the image data processing methods adopted in the industry are to manually discriminate and label the taken photos and form data compilations, atlases, and other documents, store the documents in formats such as word and PDF, or directly pack multiple pictures into a database. The data products formed by the above two data processing methods are all unstructured data. The efficiency of manual data processing is low, and subsequent use is difficult, seriously affecting the excavation of the value of image data. Summary of the Invention

[0003] In view of this, the purpose of the present invention is to provide a method, a system, and a computer-readable storage medium for governing natural environment test image data based on an inspection robot. This method converts the acquired unstructured original image data into structured data that is convenient for storage and use, realizes the automatic processing of converting unstructured data into structured data, effectively improves the efficiency of image data processing and evaluation, and enhances the use value of the data.

[0004] On the one hand, the present invention provides a method for governing natural environment test image data based on an inspection robot, and the method includes:

[0005] Step 1: Enter the natural environment test sample information into the sample information record table. Among them, the sample information record table includes the sample name, the test rack number where the sample is located, the sample number, the row and column in the test rack where the sample is located, and the basic information of the sample. The basic information of the sample is respectively recorded in the basic information table corresponding to the material category of the sample;

[0006] Step 2: Formulate an inspection task, and the inspection task includes the inspection task name, the inspection task number, the map number associated with the target sample, the test rack number, the sample number, the inspection time, and the format requirements for data acquisition;

[0007] Step 3: Send the inspection task to the inspection robot, so that the inspection robot performs data acquisition according to the inspection task. The data acquired by the inspection robot includes the original image and the cut image that removes the background information and only retains the target sample;

[0008] Step 4: Receive the data collection result returned by the robot;

[0009] Step 5: Perform image recognition on the data collection result;

[0010] Step 6: Integrate and structurally store the data according to the recognition result.

[0011] Furthermore, the categories of samples are divided into bare metal samples, metal coating samples, and coating samples. The basic information tables corresponding to the sample categories include the basic information table for bare metal samples, the basic information table for metal coating samples, and the basic information table for protective coating system samples.

[0012] Furthermore, after the inspection task is sent to the inspection robot, the inspection robot will generate an information confirmation result for the inspection task according to the inspection task information and the current status of the inspection robot.

[0013] Furthermore, before performing image recognition on the data collection result, it also includes checking the data collection result to determine whether the data is complete. If the data is complete, image recognition is performed; otherwise, it indicates that the inspection task is not completed.

[0014] Furthermore, after the check, if the data is complete, the collected data is cleaned according to the preset cleaning judgment criteria, and the cleaning results are classified into two categories: qualified and unqualified. For the data determined to be qualified, it is saved and image recognition is performed.

[0015] Furthermore, the cleaning judgment criteria are a combination of one or more of the clarity criteria, exposure criteria, color deviation criteria, regional integrity criteria, repeatability criteria, etc.

[0016] Furthermore, for each inspection task, the finally generated image data includes the original image collected by the inspection robot, the cut image, and the recognized image after image recognition. Step 6 specifically includes: integrating the finally generated image data and the information related to the inspection task into the sample information form of the target sample corresponding to the inspection task, and associating the sample information form with the image data through the image data file name field in the sample information form to achieve the structural storage of the data.

[0017] Furthermore, the image recognition result includes generating a recognized image, damage type, damage area ratio, damage level, etc.

[0018] Another aspect of the present invention provides a natural environment test image data governance system based on an inspection robot. The system includes:

[0019] A memory configured to store a computer program;

[0020] A processor configured to execute the computer program to implement the method for managing natural environment test image data based on an inspection robot as described above.

[0021] The present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method for managing natural environment test image data based on an inspection robot as described above is implemented.

[0022] The beneficial effects of the present invention are:

[0023] The present invention conducts the acquisition and management of natural environment test image data from the source, sorts out and standardizes the entire process of data management including original information entry, task management, data association, data verification, data cleaning, image recognition, data integration, data warehousing, and structured storage, so as to realize the operations of data classification and processing by a computer instead of manual labor, which can effectively reduce labor costs and improve data processing efficiency, and the value of the structured data formed after management is significantly improved.

[0024] Other advantages, objectives, and features of the present invention will be described to some extent in the subsequent description, and to some extent, will be obvious to those skilled in the art based on the study of the following text, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the drawings, where:

[0026] Figure 1 is a schematic flowchart of data management;

[0027] Figure 2 is an example of image data. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0028] The following will refer to the drawings to describe the preferred embodiments of the present invention in detail. It should be understood that the preferred embodiments are only for illustrating the present invention, rather than for limiting the protection scope of the present invention.

[0029] The method described in the present invention requires the use of an intelligent inspection robot system, including a front-end robot (i.e., "intelligent inspection robot", abbreviated as "inspection robot") and a back-end management software (also known as "intelligent inspection task management and data transfer platform").

[0030] The intelligent inspection robot includes components such as a moving chassis, navigation sensors, binocular cameras, robotic arms, drive motors, batteries, and shock absorbers. The moving chassis of the robot is preferably a crawler chassis or a wheeled moving chassis, which can adapt to the muddy ground of the natural environment test field and has the climbing ability of at least a 30-degree slope. The binocular camera has an optical zoom lens, and at the same time has the functions of taking pictures and videos, and has intelligent focusing ability, and its resolution is not less than 10 million pixels. The navigation sensors are equipped with lidar and Beidou satellite positioning navigation, and the accuracy is better than ±0.2m. The motor battery is selected according to the usage conditions of the natural environment test fields in different regions, but it should meet the requirement of normal use for 2 hours under the rated power.

[0031] As the backend management software, the intelligent inspection task management and data transfer platform mainly includes a task management module and a data processing module, which can realize the informatization management of natural environment test tasks, formulate, issue, and supervise inspection tasks through background information, and upload the image data collected by the inspection robot (hereinafter referred to as the "front end") to the data management terminal of the transfer platform (hereinafter referred to as the "backend"), and realize functions such as data storage and data management through the backend.

[0032] Figure 1 It is a schematic flowchart of the image data governance method for natural environment tests based on inspection robots proposed by the present invention. As Figure 1 shown, the method includes the following steps:

[0033] Step 1: Enter the natural environment test sample information into the sample information record table. Among them, the sample information record table contains the sample name, the test rack number where the sample is located, the sample number, the row and column in the test rack where the sample is located, and the basic information of the sample. The basic information of the sample is recorded in the corresponding basic information table according to the material category of the sample;

[0034] Step 2: Formulate an inspection task, which includes the inspection task name, inspection task number, map number associated with the target sample, test rack number, sample number, inspection time, and format requirements for data collection;

[0035] Step 3: Send the inspection task to the inspection robot, so that the inspection robot collects data according to the inspection task. The data collected by the inspection robot includes the original image and the cut image that removes the background information and only retains the target sample;

[0036] Step 4: Receive the data collection result returned by the robot;

[0037] Step 5: Perform image recognition on the data collection result;

[0038] Step 6: Perform data integration and structured storage according to the recognition result.

[0039] In step 1, by establishing a test task management module at the backend, the sample information of the natural environment test task is entered in the form of Table 1.

[0040] Table 1 Sample Information Record Form

[0041] Primary Key Sample Name Test Stand Number Sample Number Row Column Basic Information Such as JJ-Q420-SZ-1-2 1～N 1～N Material Category, Test Method, etc.

[0042] Among them, the "primary key" in Table 1 represents the unique code of each piece of data in the sample information record form, which can be manually input or automatically generated; in the "sample name", "JJ" corresponds to the test location, where "JJ" indicates that the test location is Jiangjin, and "Q420" corresponds to the sample material grade or process grade, and "Q420" indicates that the sample material grade or process grade is Q420; "SZ" corresponds to the detected performance, and "SZ" represents the corrosion weight loss specimen; the "1" after SZ indicates that the sampling period of the natural environment test is the first period; the last "2" represents the second parallel sample. The "test rack number" represents the serial number of the test rack where the sample is located (each test rack has a different number to distinguish the test racks), and N represents any natural number. The "sample number" represents the number of a specific sample on this test rack (each sample has a different number to distinguish the samples), and N represents any natural number. "Row" and "column" represent the specific row and column where the sample is located in the test rack. The "basic information" column gives general information such as the material category and test method of the sample. The specific basic information of the sample can be recorded separately in the corresponding basic information form according to the category of the sample.

[0043] The categories of samples can be divided into metal bare material samples, metal coating samples, and coating samples. Then, the basic information forms corresponding to the sample categories include the basic information form of metal bare material samples (as shown in Table 2), the basic information form of metal coating samples (as shown in Table 3), and the basic information form of protective coating system samples (as shown in Table 4). It should be noted that only the information represented by each column in Tables 2 - 4 is given in the tables, and the specific content of each column of information is not given, and these contents can be recorded according to the actual situation. In addition, some information in Tables 2 to 4 needs to be entered in advance, and the remaining information is automatically filled after the inspection. For example, "serial number", "material category (or process category)", "material variety (or process variety)", "material grade (or process grade)", "substrate material", "substrate grade", "heat treatment status (or surface treatment process, protective coating composition)", "production unit", "production time", "test station number", "test station name", "test method", "test date", "parallel sample number" are entered in advance before the test, and the remaining information is filled after the inspection.

[0044] Table 2 Basic Information Form of Metal Bare Material Samples

[0045]

[0046] Table 3 Basic Information Table of Metal Coating Samples

[0047]

[0048] Table 4 Basic Information Table of Protective Coating Systems

[0049]

[0050] For the formulation of the inspection task, first, a digital map of the test site is established (this part is completed in advance), and the positions of the test stand and samples are located through the Beidou system and lidar. The inspection task includes the inspection task name, inspection task number, map number associated with the target sample (i.e., on which map the sample is located), test stand number, sample number, inspection time, and format requirements for data collection (e.g., the format of the collected photos, the resolution of the photos, etc.). After the inspection task is determined, a task code is generated. The task code is used to number the task. For example, a task code composed of a preset length (e.g., 19 digits) of numbers can be generated, such as "1854093306030252033". This task code can be used to name the images after the robot collects them, so as to associate the task with the task data.

[0051] After that, the task is sent to the robot, that is, the backend software sends control commands to the frontend robot, including inspection task information. The communication rules during task sending can be seen in Table 5.

[0052] Table 5 Communication Rules for Task Sending

[0053]

[0054]

[0055] The task sending interface is that the intelligent inspection task management and data transfer platform calls the robot-side http protocol interface to transfer the task instructions to the robot side. The interface content is as follows:

[0056]

[0057] It should be noted that in the above interface content, the "inspection task name" is the task code, which is a string of characters randomly generated by the computer, such as "1854093306030252033"; the "task number" refers to the task sorting for batch sending. For example, if 10 tasks are sent in batch, the task numbers for this batch are: 001 to 010.

[0058] In some embodiments, after the inspection task is assigned to the inspection robot, the inspection robot generates an information confirmation result for the inspection task based on the inspection task information and the current state of the inspection robot.

[0059] For example, after the robot receives the information, the returned parameters are as follows:

[0060]

[0061] Based on the parameters returned by the robot, it is possible to determine whether the robot has received the inspection task; the name of the received inspection task; the task number; the name of the map used; the status of the current task queue (for example, Pending means waiting; Conflict_error means there is a conflict error); the time when the task was received; whether there is an error, and if there is an error, the error code is returned; the message describing the current state of the task queue; the response time; whether the request to perform the inspection task was successful, etc.

[0062] After the robot obtains the inspection task information, through background retrieval, it can determine the position of the target sample on the test rack and the spatial position information, and based on the map name field in the task information, select the corresponding digital map, autonomously plan the route and navigate to the predetermined position, and then return to the initial position after completing all tasks.

[0063] After the robot completes the inspection task, it uploads the collected data, that is, returns the inspection status and the result of data collection to the backend. When uploading the data, the communication rules used are shown in Table 6.

[0064] Table 6 Data Upload Communication Rules

[0065]

[0066]

[0067] The interface for uploading the collected data is called by the robot side to call the http protocol interface of the intelligent inspection task management and data transfer platform, and upload the collected data compressed package to the platform. The interface content is as follows:

[0068]

[0069] After the robot task is completed and returns, the collected image data is uploaded to the backend in the form of a compressed package. The image data in the compressed package includes the original collected images and the cropped image data with the background information removed and only the target sample remaining.

[0070] The data file naming rules are as follows:

[0071] The name of the original image is: 1854093306030252033.jpg,

[0072] The name of the cut image is: 1854093306030252033_cut.jpg.

[0073] Among them, "1854093306030252033" is the task code when the task is issued.

[0074] In some embodiments, before performing image recognition on the data acquisition result, it also includes checking the data acquisition result to determine whether the data is complete. If the data is complete, image recognition is performed; otherwise, it indicates that the inspection task is not completed. Specifically, by automatically checking the data name and task code, it is determined that the amount of data returned is equal to the amount of data required by the task issued. If the check result shows that there is data missing, the tasks that have not been executed normally are confirmed according to the task code, and the tasks are re - formulated.

[0075] After the automatic check, if the data is complete, the collected data is cleaned according to the preset cleaning judgment criteria, and the cleaning results are judged into two categories: qualified and unqualified. For the data judged to be qualified, it is saved and image recognition is performed.

[0076] Data cleaning refers to cleaning the image data taken by the robot through an algorithm, quickly and accurately judging the quality of the photos, eliminating unqualified images, and ensuring the reliability and integrity of the data.

[0077] According to the outdoor scene of the natural environment test and the image characteristics of the standard flat specimen, the image data quality evaluation criteria are set. The cleaning judgment criteria are a combination of one or more of the clarity criteria, exposure criteria, color deviation criteria, regional integrity criteria, repeatability criteria, etc.

[0078] For the clarity criteria, if the image blur exceeds a certain threshold, it is judged as unqualified;

[0079] For the exposure criteria, if the image is over - exposed or under - exposed and cannot correctly display the test phenomenon, it is judged as unqualified;

[0080] For the color deviation criteria, through color analysis, it is determined whether there is a serious color deviation in the image. If the color deviation is too large, it is regarded as unqualified;

[0081] For regional integrity, it is judged whether the test area in the image is complete, whether there is occlusion or missing. If so, it is judged as unqualified;

[0082] For the repeatability criteria, it is judged whether there are multiple repeated photos in the same test scene. If the repeated photos do not provide additional valid information, they can be marked as redundant photos.

[0083] According to the above cleaning judgment criteria, a machine learning algorithm based on support vector machines is used to classify each picture and determine whether it belongs to the "qualified" or "unqualified" category.

[0084] For the image data determined to be "unqualified", automatic marking is performed, and then submitted for manual review to make a final determination. For the image data marked as "qualified", it is saved to the database for subsequent use. Through this step, invalid image data can be automatically cleared, improving the quality of environmental test data.

[0085] After data cleaning, for the images determined to be qualified, intelligent image recognition technology (such as a method for quantitative evaluation of metal corrosion damage based on image recognition provided in Application No. 202110547289.9) is used to process them, and the recognition results are output. The image recognition results include the generated recognition image, damage type, proportion of damage area, damage level, etc. The recognition images are named using the same naming rule. For example, for the inspection task with the task code "1854093306030252033", the finally obtained recognition image can be named: 1854093306030252033_recognition.jpg.

[0086] After the above steps, the automatic acquisition and processing of natural environment test image data have been completed, and based on the data transmission and processing rules formulated by this method, the association of data streams has been achieved. For a single task, the generated image data includes the original image, the cut image, and the recognition image. Refer to Figure 2 , Figure 2 In which, Figure (a) in Figure 2 represents the original image; Figure 2 Figure (b) in

[0087] represents the cut image;

[0088] Figure (c) in

[0089]

[0090]

[0091] Further, another aspect of the present invention provides a natural environment test image data governance system based on an inspection robot, and the system includes:

[0092] A memory configured to store a computer program;

[0093] A processor configured to execute the computer program to implement the method for managing image data of natural environment tests based on an inspection robot as described above.

[0094] The present invention also provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, it implements the method for managing image data of natural environment tests based on an inspection robot as described above.

[0095] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the present technical solution, and they should all be covered within the scope of the claims of the present invention.

Claims

1. A natural environment test image data management method based on inspection robots, characterized in that: include: Step 1: Enter the natural environment test sample information into the sample information record table, wherein the sample information record table includes the sample name, the test rack number where the sample is located, the sample number, the row and column of the test rack where the sample is located, and the basic information of the sample. The basic information of the sample is recorded in the basic information table corresponding to the sample category according to the category of the sample; Step 2: Formulate inspection tasks, which include inspection task name, inspection task number, map number associated with target sample, test rack number, sample number, inspection time, and format requirements for data collection; Step 3: Send the inspection task to the inspection robot, so that the inspection robot collects data according to the inspection task. The data collected by the inspection robot includes the original image and the cut image with only the target sample retained after the background information is removed; Step 4: Receive the data collection results returned by the robot; Step 5: Perform image recognition on the data collection results; Step 6: Integrate and structure data based on the recognition results.

2. The natural environment test image data management method based on the inspection robot according to claim 1 is characterized in that: The samples are divided into bare metal samples, metal covering layer samples and coating samples. The basic information tables corresponding to the sample categories include bare metal sample basic information table, metal covering layer sample basic information table and protective coating system sample basic information table.

3. The natural environment test image data management method based on the inspection robot according to claim 1 is characterized in that: After the inspection task is sent to the inspection robot, the inspection robot will generate an information confirmation result for the inspection task based on the inspection task information and the current status of the inspection robot.

4. The natural environment test image data management method based on the inspection robot according to claim 1 is characterized in that: Before performing image recognition on the data collection results, the data collection results are also checked to determine whether the data is complete. If the data is complete, image recognition is performed; otherwise, it means that the inspection task is not completed.

5. The natural environment test image data management method based on the inspection robot according to claim 4 is characterized in that: After verification, if the data is complete, the collected data is cleaned according to the preset cleaning judgment criteria, and the cleaning results are judged as qualified and unqualified. For the data judged as qualified, it is saved and image recognition is performed.

6. The method for managing natural environment test image data based on inspection robots according to claim 5 is characterized in that: The cleaning judgment standard is a combination of one or more of clarity standard, exposure standard, color deviation standard, area integrity standard, repeatability standard, etc.

7. The natural environment test image data management method based on the inspection robot according to claim 1 is characterized in that: For each inspection task, the image data finally generated includes the original image collected by the inspection robot, the cut image, and the recognized image after image recognition. Step 6 specifically includes: integrating the finally generated image data and the information related to the inspection task into the sample information form of the target sample corresponding to the inspection task, and associating the sample information form with the image data through the image data file name field in the sample information form to realize structured storage of data.

8. The method for managing natural environment test image data based on inspection robots according to claim 1, characterized in that: Image recognition results include generated recognition images, damage type, damage area percentage, damage level, etc.

9. A natural environment test image data management system based on inspection robots, characterized in that: include: a memory configured to store a computer program; A processor is configured to execute the computer program to implement the natural environment test image data management method based on the inspection robot as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the natural environment test image data management method based on the inspection robot as described in any one of claims 1 to 8.

Images