Image data management method and system based on engineering supervision platform

Abstract

The invention provides an image data management method and system based on an engineering supervision platform, and relates to the field of electric digital data processing, and the method comprises the steps: collecting original image data of a construction site through a monitoring camera, and carrying out the data cleaning, and obtaining initial image data; determining a target deep learning model from a plurality of pre-trained deep learning models with different image recognition granularities according to an accuracy level demand and an efficiency level demand of model detection; inputting the initial image data into a target deep learning model for target detection, and outputting an identification result; and checking and verifying the feature data of the target object based on a preset rule engine to obtain a verification result, and further generating a final marking result of the original image data. By implementing the method, the accuracy and efficiency of model identification in the automatic marking process of the image data can be automatically balanced according to user requirements.

Description

Technical Field

[0001] The present application relates to the field of electronic digital data processing, and in particular to an image data management method and system based on an engineering supervision platform. Background Art

[0002] In the field of modern engineering supervision, the application of image data has become one of the key technologies to improve engineering quality and safety. By using surveillance cameras to continuously collect images of the construction site, the progress and safety of the project can be monitored in real time to ensure that the construction activities meet the predetermined quality and safety standards. The application of this technology helps project managers to respond quickly to various situations occurring on site and make effective decisions and interventions.

[0003] In related technologies, engineering supervision mainly uses monitoring systems to automatically collect image data from construction sites and conduct preliminary processing of these data, such as data cleaning and simple image recognition. In this process, it usually relies on preset algorithms to identify key elements in the image (such as construction personnel, equipment usage, etc.), and conducts some basic analysis and reporting based on these recognition results.

[0004] However, traditional image data processing methods rely on the data processing capabilities of image recognition models. When processing large-scale and complex image data, they often sacrifice accuracy while improving processing speed, or cause processing efficiency to decrease while improving high accuracy, which seriously affects the actual application effect and response speed of engineering supervision. Summary of the invention

[0005] The present application provides an image data management method and system based on an engineering supervision platform, which is used to address the problem of difficulty in balancing accuracy and efficiency during automatic labeling of image data.

[0006] In a first aspect, the present application provides an image data management method based on an engineering supervision platform, which is applied to an image data management system. The method includes: The original image data of the construction site is collected by monitoring cameras and cleaned to obtain preliminary image data; According to the accuracy level and efficiency level requirements of model detection, determine the target deep learning model from multiple pre-trained deep learning models of different image recognition granularities; Inputting the preliminary image data into the target deep learning model for target detection, and outputting a recognition result, the recognition result including the target object and feature data of the target object; Based on the preset rule engine, the characteristic data of the target object is reviewed and verified to obtain the verification result; The final marking result of the original image data is generated according to the verification result.

[0007] Through the above embodiments, the image data management system collects the original image data of the construction site, performs data cleaning, and then selects the target deep learning model for target detection and verification of the recognition results based on the user's requirements for the accuracy level and efficiency level of model detection, thereby reducing the impact of unilateral pursuit of model recognition accuracy or efficiency on model recognition, and achieving automatic balancing of model recognition accuracy and efficiency in the process of automatic labeling of image data according to user needs.

[0008] In some embodiments, before the step of determining a target deep learning model from a plurality of pre-trained deep learning models of different image recognition granularities based on the accuracy level requirement and efficiency level requirement of model detection, the method further includes: Collect sample image data at different stages of construction and annotate them to obtain a training data set; Model training is performed at different network levels according to the training data set to obtain the pre-trained deep learning model at different image recognition granularities, and different deep learning models correspond to different network levels; According to the test results of the deep learning model on the test set, the accuracy level and efficiency level corresponding to the deep learning model at different image recognition granularities are divided.

[0009] Through the above embodiments, the image data management system collects sample image data at different stages of construction to construct a training data set. Then, the deep learning model is trained at different network levels to obtain pre-trained deep learning models at different image recognition granularities. Then, according to the test results of these deep learning models on the test set, they are divided into different accuracy levels and efficiency levels. This method can obtain deep learning models with different performances at different image recognition granularities, providing more options for the subsequent flexible selection of appropriate target models according to needs, and further improving the adaptability and practicality of the entire image data management process.

[0010] In some embodiments, the step of performing model training at different network levels based on the training data set to obtain the pre-trained deep learning model at different image recognition granularities specifically includes: Construct the initial model of the deep learning model based on convolutional neural network, region proposal network and feature pyramid network; The initial model is trained using the training data set to determine model parameters corresponding to different image recognition granularities; The pre-trained deep learning model at different image recognition granularities is determined based on the model parameters.

[0011] Through the above embodiments, the image data management system can improve the target detection capability and accuracy of the pre-trained model by combining multiple deep learning network architectures and optimizing model parameters using training data sets. This provides a more reliable and efficient basis for subsequent model selection and helps to further improve the overall level of image data management.

[0012] In some embodiments, before the step of determining a target deep learning model from a plurality of pre-trained deep learning models of different image recognition granularities based on the accuracy level requirement and efficiency level requirement of model detection, the method further includes: Get the construction phase corresponding to the current project; According to the construction stage, the key areas for marking are matched from the database, as well as the accuracy level requirements and efficiency level requirements corresponding to the key areas.

[0013] Through the above embodiments, the image data management system matches the corresponding key marking areas and the corresponding accuracy and efficiency requirements according to the current construction stage of the project. The focus of image recognition and data management can be determined more accurately according to the characteristics of different construction stages, avoiding the problem of affecting processing efficiency due to too high requirements or affecting recognition accuracy due to too low requirements. At the same time, matching the construction stage with the accuracy and efficiency requirements can make the entire image data management process more intelligent and adaptive, reduce manual intervention, and improve the efficiency and reliability of project supervision.

[0014] In some embodiments, before the step of determining a target deep learning model from a plurality of pre-trained deep learning models of different image recognition granularities based on the accuracy level requirement and efficiency level requirement of model detection, the method further includes: Get the text description information uploaded by the user; Use a natural language processing model to analyze the text description information and determine the key areas for marking; The corresponding accuracy level requirements and efficiency level requirements are matched from the database according to the key area.

[0015] Through the above embodiments, the image data management system performs natural language analysis on the text description information uploaded by the user to determine the key areas for marking, and then matches the corresponding accuracy and efficiency requirements. The focus of image data management can be specified more flexibly to meet the personalized needs of different users or projects. At the same time, the use of natural language processing technology to automatically analyze user descriptions reduces the user's workload and makes human-computer interaction more efficient and intelligent. Combining image recognition requirements with user language descriptions helps the engineering supervision system better understand and adapt to the actual situation on site.

[0016] In some embodiments, after the step of inputting the preliminary image data into the target deep learning model for target detection and outputting the recognition result, the step further includes: Using an optical flow algorithm to estimate the motion state of pixels between consecutive frames in the preliminary image data, to obtain optical flow information; The moving trajectory of each target object in the video frame is tracked according to the optical flow information.

[0017] Through the above embodiments, the image data management system uses the optical flow algorithm to estimate the motion state of pixels between consecutive frames in the preliminary image data, and tracks the movement trajectory of each target object in the video frame. On the basis of static target recognition, it can further realize the tracking and analysis of dynamic targets, obtain more comprehensive and accurate target state information, provide a reliable basis for safety and quality control, and greatly improve the automation level of engineering supervision.

[0018] In some embodiments, the step of collecting raw image data of the construction site through a monitoring camera and performing data cleaning to obtain preliminary image data specifically includes: Collect raw image data of the construction site through surveillance cameras; Obtain parameter information corresponding to the original image data, where the parameter information is a picture frame parameter that affects the deep learning model for target detection; The parameter information corresponding to the original image data is scored based on a preset scoring system, and the original image data with a score higher than a preset threshold is cleaned to obtain preliminary image data.

[0019] Through the above embodiments, the image data management system analyzes the parameter information of the original image data collected at the construction site and scores the parameter information of the original image data based on a preset scoring system, thereby automatically screening high-quality original image data with a score higher than a preset threshold for data cleaning to obtain preliminary image data. Low-quality or useless data can be filtered out, reducing the computational overhead of subsequent processing and improving overall processing efficiency.

[0020] In a second aspect, the present application provides an image data management system, the image data management system comprising: one or more processors and a memory; The memory is coupled to the one or more processors, and the memory is used to store computer program code, which includes computer instructions. The one or more processors call the computer instructions so that the image data management system can implement an image data management method based on an engineering supervision platform provided in the above embodiment, which will not be repeated here.

[0021] In a third aspect, the present application provides a computer-readable storage medium comprising instructions. When the instructions are executed on an image data management system, the image data management system can implement an image data management method based on an engineering supervision platform provided in the above-mentioned embodiment, which will not be repeated here.

[0022] Fourthly, the present application provides a computer program product. When the computer program product runs on an image data management system, the image data management system can implement an image data management method based on an engineering supervision platform provided in the above embodiment, which will not be repeated here.

[0023] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. The image data governance system automatically selects the optimal model from pre-trained deep learning models of different granularities for target detection and recognition based on the user's requirements for model detection accuracy and efficiency. This mechanism can balance accuracy and efficiency in different application scenarios and improve the adaptability and practicality of the system. At the same time, by dynamically adjusting the model selection strategy at different construction stages and user needs, the system's intelligence and adaptability are further enhanced, reducing manual intervention.

[0024] 2. The image data management system collects sample data at different stages of construction, builds a multi-stage training data set, and trains deep learning models at different network levels to obtain pre-trained models of different granularities. By combining multiple deep learning architectures and model parameter optimization, the detection capability and accuracy of the pre-trained model are improved.

[0025] 3. The image data governance system uses a variety of information for analysis, including image data parameters, user text descriptions, target motion trajectories, etc. Among them, the data cleaning process is optimized by scoring and screening image parameters; the user description is analyzed using natural language to achieve flexible and intelligent human-computer interaction; the target motion is tracked in combination with the optical flow algorithm to expand the system's dynamic analysis capabilities. The fusion analysis of multimodal information enables the system to fully understand and adapt to complex engineering supervision scenarios, provide reliable analysis basis, and improve the automation and intelligence level of engineering supervision. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a flow chart of an image data management method based on an engineering supervision platform in an embodiment of the present application; Figure 2 It is another flow chart of an image data management method based on an engineering supervision platform in an embodiment of the present application; Figure 3 It is a schematic diagram of the physical device structure of the image data management system in the embodiment of the present application. DETAILED DESCRIPTION

[0027] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to be used as limitations to the present application. As used in the specification and appended claims of the present application, the singular expressions "one", "a kind of", "said", "above", "the" and "this" are intended to also include plural expressions, unless there is a clear indication to the contrary in the context. It should also be understood that the term "and / or" used in the present application refers to any or all possible combinations comprising one or more listed items.

[0028] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

[0029] For ease of understanding, the following is a description of the process of the method provided by this implementation. Figure 1 , which is a flow chart of an image data management method based on an engineering supervision platform in an embodiment of the present application.

[0030] S101, collecting original image data of the construction site through a monitoring camera and performing data cleaning to obtain preliminary image data.

[0031] The image data management system continuously collects raw image data of the construction site through surveillance cameras deployed at the construction site. These raw image data may include static pictures and dynamic videos, recording the real-time conditions of various areas and different time periods of the construction site.

[0032] After collecting the raw image data, in order to ensure the efficiency and accuracy of subsequent processing and analysis, the image data management system cleans the collected raw image data, thereby filtering out image data of poor quality or irrelevant to the engineering supervision task, and obtaining a relatively standardized, accurate and usable preliminary image data set.

[0033] S102. Determine a target deep learning model from a plurality of pre-trained deep learning models of different image recognition granularities based on accuracy level requirements and efficiency level requirements of model detection.

[0034] Considering the complexity and variability of engineering supervision scenarios, as well as the different emphasis on recognition accuracy and processing efficiency in practical applications, a single deep learning model is often difficult to meet all requirements. Therefore, the key to this step is to flexibly select the most appropriate target detection model based on the current task requirements.

[0035] Specifically, the image data management system has prepared multiple pre-trained deep learning models, each of which has different granularity of image recognition (i.e., the degree of refinement of target types and details). Models with higher granularity can recognize more categories of targets and give more detailed descriptions, with higher accuracy but usually more computational complexity; models with lower granularity have coarser details and lower accuracy but faster processing speed.

[0036] Furthermore, the image data governance system classifies the accuracy and efficiency of each model into different levels, such as high (accuracy exceeding 98% or efficiency exceeding 100 frames per second), etc., to obtain a model-accuracy-efficiency relationship table.

[0037] When a user selects a deep learning model, the image data management system matches the corresponding target deep learning model or the closest target deep learning model from the model-accuracy-efficiency relationship table based on the accuracy level requirements and efficiency level requirements entered by the user.

[0038] S103: Input the preliminary image data into the target deep learning model for target detection and output the recognition result.

[0039] Specifically, the target deep learning model conducts a comprehensive scan and analysis of each input image. The multi-level features of the image are extracted with the help of a convolutional neural network, and the target area in the image is gradually found through a series of operations such as region candidate, bounding box regression, classification and discrimination. These may include workers, machinery and equipment, building components, safety protection facilities and other types of identification targets related to engineering supervision.

[0040] S104: Audit and verify the characteristic data of the target object based on a preset rule engine to obtain a verification result.

[0041] After obtaining the preliminary recognition results output by the deep learning model, the image data management system further reviews and verifies these results based on the preset rule engine to obtain verification results to ensure their accuracy and availability.

[0042] The rule engine is a decision-making system based on expert knowledge and experience, covering various specifications, standards, and procedures in the field of engineering supervision. These rules are expressed in the form of condition-action, for example: "IF workers are working at a height of more than 2 meters AND are not wearing safety helmets THEN determine violation and issue an early warning". The system substitutes the identified target feature data into these rules, automatically determines whether it meets the conditions, and makes corresponding verification decisions.

[0043] For example, in a specific embodiment, a welder is captured by on-site surveillance. Through target detection, the image data management system identifies that the worker is not wearing a protective mask and that there is no obvious isolation fence in the welding area. The safety production rules built into the image data management system immediately check the situation and find that the conditions such as "IF welding operation AND not wearing a protective mask THEN serious violation is determined" are met, so a verification result of "serious violation" is generated and an early warning prompt is issued.

[0044] S105: Generate a final marking result of the original image data according to the verification result.

[0045] After review and verification by the rule engine, the image data governance system has determined the compliance status and risk information of the target objects identified in each original image. In order to further refine and clarify this information, the system generates clear and standardized annotations on the original image to form the final labeling result.

[0046] Specifically, the image data governance system will add text labels near the bounding box of each identified target object, using concise semantics to express its key attributes such as category, status, and hidden dangers. The style and content of these labels are usually planned in a unified manner. For example, for compliance targets, they are marked with a green border and a "normal" label; for non-compliant targets, they are marked with a red border and a specific violation type ("not wearing a safety helmet", "high-altitude work"); for risk hazards, they are marked with an orange border and a risk level ("serious", "large"), etc.

[0047] In the above embodiment, the image data management system collects the original image data of the construction site, performs data cleaning, and then selects the target deep learning model for target detection and verification of the recognition results based on the user's requirements for the accuracy level and efficiency level of model detection. This reduces the impact of unilateral pursuit of model recognition accuracy or efficiency on model recognition, and achieves automatic balancing of accuracy and efficiency in the process of automatic labeling of image data based on user needs.

[0048] The following is a more detailed description of the process of the method provided by this implementation. Figure 2 , is another flow chart of an image data management method based on an engineering supervision platform in an embodiment of the present application.

[0049] S201. Collect original image data of the construction site through a monitoring camera and obtain parameter information corresponding to the original image data.

[0050] The image data management system collects raw image data in real time through surveillance cameras deployed in various areas of the construction site. These cameras usually cover key areas of the construction site, such as large equipment operation areas, high-altitude operation areas, and edge opening areas, to record the behavior of personnel and the status of materials and equipment during the construction process from all directions and angles. The raw image data collected by the camera includes video streams and periodically captured pictures. The data format can be common mp4, avi, jpg, etc., which are not limited here.

[0051] While collecting raw image data, the image data management system obtains parameter information corresponding to each frame of the image. These parameters usually include the image capture time, camera number, image resolution, image brightness, contrast and other key factors that affect image quality and subsequent analysis results. By obtaining these parameters, the image data management system can preliminarily evaluate the availability of raw image data and prepare for subsequent data cleaning.

[0052] S202 : Score the parameter information corresponding to the original image data based on a preset scoring system, and perform data cleaning on the original image data with a score higher than a preset threshold to obtain preliminary image data.

[0053] After obtaining the original image data and its parameters, the image data management system further identifies the quality of each frame of the image. Specifically, the image data management system has a built-in preset image quality scoring system, which mainly gives a score range of 0-100 based on indicators such as image contrast, sharpness, color authenticity, and distortion-free degree. Optionally, images with a score of 80 points or more are determined to be qualified high-quality data, and preliminary image data is obtained after data cleaning. Images with a score of 80 points or more are directly judged as unqualified and discarded or directly handed over to manual processing.

[0054] S203: Collect sample image data at different construction stages and annotate them to obtain a training data set.

[0055] This step is used to construct a training data set for the pre-trained deep learning model used in step S102. Specifically, the image data management system collects representative pictures from different construction stages, such as images of support structures in the foundation pit excavation stage, images of formwork projects in the main construction stage, and images of temporary power lines in the decoration stage. These pictures should cover all aspects of each construction scene as much as possible, and contain enough target categories and postures. After collecting enough pictures, professional personnel from the organization will annotate the key targets in the pictures in the image data management system, outline accurate bounding boxes for each target, and indicate its category, location, status and other attribute information. After the annotation is completed, the image data management system organizes these pictures and their annotation information into a standardized training data set.

[0056] S204. Construct an initial model of the deep learning model based on the convolutional neural network, the region proposal network and the feature pyramid network.

[0057] When the training data set is ready, the image data management system enters the stage of building a deep learning model. Specifically, the image data management system uses a convolutional neural network (CNN) as the backbone network to extract multi-level and multi-scale features in the image, and through further fine-tuning and optimization, it can better adapt to the complex and changeable construction environment images. Then, based on CNN, the image data management system introduces a region proposal network (RPN) to extract candidate regions that may contain targets from the image feature map. Among them, RPN uses a set of rectangular boxes of fixed size and aspect ratio to slide on the feature map, and through binary classification and bounding box regression, it quickly screens out target areas with higher probability, greatly reducing the calculation amount of subsequent detection. In addition, the image data management system can also add a feature pyramid network (FPN) based on CNN. FPN builds a feature pyramid on the multi-scale feature map extracted by CNN, fuses high-level semantic features and low-level detail features, and generates a series of feature maps with different resolutions. During the detection process, targets of different sizes can be effectively covered on feature maps at different levels, thereby achieving adaptive detection.

[0058] S205: Use the training data set to train the initial model, determine the model parameters corresponding to different image recognition granularities, and obtain multiple pre-trained deep learning models.

[0059] Specifically, the image data management system obtains the correspondence between different parameter combinations and model performance by experimentally adjusting the number of network layers, convolution kernel size, number of feature map channels, number of candidate regions, non-maximum suppression threshold and other parameter tests and statistical analysis. For example, increasing the number of network layers and feature channels can improve the model's feature extraction and classification capabilities, but it will also significantly increase the calculation time; while reducing the number of candidate regions and increasing the NMS threshold can speed up target screening, but it may also miss some small targets. Of course, the correspondence between the above-mentioned different parameter combinations and model performance can also be obtained by relevant technical personnel directly uploading the corresponding file data to the image data management system, which is not limited here.

[0060] Next, the image data governance system generates pre-trained models at different image recognition granularities in a targeted manner according to the actual needs of engineering supervision. For some occasions with strict efficiency requirements, such as road vehicle monitoring that requires real-time detection of each frame, the system can appropriately reduce the network scale and cut off some redundant feature channels, maximizing the detection speed at the expense of a certain accuracy rate. For some occasions where accuracy is critical, such as detailed quality inspection of dangerous and large projects, the system will use the largest network parameters, at the expense of longer processing delays, in an effort to achieve extremely high defect recognition accuracy.

[0061] In the above embodiments, the image data management system can improve the target detection capability and accuracy of the pre-trained model by combining multiple deep learning network architectures and optimizing model parameters using training data sets. This provides a more reliable and efficient basis for subsequent model selection and helps to further improve the overall level of image data management.

[0062] S206. According to the test results of the deep learning model on the test set, the deep learning models at different image recognition granularities are divided into corresponding accuracy levels and efficiency levels.

[0063] Specifically, for each picture in the test set, the image data governance system uses the current deep learning model to be evaluated for inference prediction to obtain a set of detection box results. Then, the predicted detection box is compared with the manually annotated true value box to calculate its intersection over union (IoU). If the IoU between a predicted box and the true value box exceeds the preset threshold (such as 0.5), the predicted box is judged to be a true positive (TP), otherwise it is a false positive (FP). After traversing all the prediction boxes, the system can obtain the TP number and FP number of the current picture. Repeat the above process on the entire test set to obtain the accuracy of the model, and perform grade division to obtain the accuracy level corresponding to the model.

[0064] At the same time, the image data management system can also count the time spent by the model during the reasoning process, including the time spent in each stage such as data reading, feature extraction, candidate box generation, bounding box regression, non-maximum suppression, etc. By adding up the time spent in each stage, we can get the average time spent by the model to process a single image, that is, the model efficiency, and divide it into levels to get the corresponding accuracy level of the model.

[0065] In the above embodiment, the image data management system collects sample image data at different stages of construction to construct a training data set. Then, the deep learning model is trained at different network levels to obtain pre-trained deep learning models at different image recognition granularities. Then, according to the test results of these deep learning models on the test set, they are divided into different accuracy levels and efficiency levels. This method can obtain deep learning models with different performances at different image recognition granularities, providing more options for the subsequent flexible selection of appropriate target models according to needs, and further improving the adaptability and practicality of the entire image data management process.

[0066] S207: Use a natural language processing model to analyze the text description information uploaded by the user to determine the key areas for marking.

[0067] Specifically, users can enter a short text in the system interface of the image data management system to express the objects and areas they want to focus on and analyze. These texts can use everyday language and industry terms, and do not require strict format and grammar. For example: "Check whether the tower crane driver is wearing a seat belt", "Watch the excavation slope of the foundation pit", etc.

[0068] When the user submits the text, the image data management system uses a pre-trained Chinese word segmentation model, such as jieba, THULAC, etc., to segment the text into several semantically complete words. Then the part of speech of each word is judged and annotated to distinguish nouns, verbs, adjectives, etc. Combined with professional dictionaries and knowledge graphs in the field of construction, these words are mapped to some predefined engineering supervision concepts, such as "machinery equipment" and "personnel behavior".

[0069] After the above processing, the image data management system can extract some key information from the user's text description and map it to specific parameters in the image understanding and analysis task. For example, in the example of "checking whether the tower crane driver is wearing a seat belt", the system will focus on the "tower crane driver" and determine the preset range area of ​​the crane as the key area.

[0070] S208. Match the corresponding accuracy level requirements and efficiency level requirements from the database according to the key areas.

[0071] Specifically, the image data governance system maintains a demand-performance correspondence database in the background. This database is based on real cases accumulated from various engineering projects in the past, and summarizes the correspondence between the image analysis requirements of different construction areas and links and the performance of the algorithm models actually used. These correspondences are stored in the form of standardized rules to facilitate quick retrieval and matching.

[0072] For example, in a specific embodiment, the database may have the following rules: IF area = "tower crane cab" AND target = "driver personnel" THEN accuracy = "high" AND efficiency = "medium", etc.

[0073] In the above embodiment, the image data management system performs natural language analysis on the text description information uploaded by the user to determine the key areas for marking, and then matches the corresponding accuracy and efficiency requirements. The focus of image data management can be specified more flexibly to meet the personalized needs of different users or projects. At the same time, the use of natural language processing technology to automatically analyze user descriptions reduces the user's workload and makes human-computer interaction more efficient and intelligent. Combining image recognition requirements with user language descriptions helps the engineering supervision system better understand and adapt to the actual situation on site.

[0074] S209, matching the key areas for marking from the database according to the construction phase corresponding to the current project, and the accuracy level requirements and efficiency level requirements corresponding to the key areas.

[0075] Specifically, relevant technical personnel can also build a database based on historical project experience summaries on the image data governance system. The database abstracts each link of construction into several standardized stages, such as "site leveling", "ground and foundation", "main structure", etc. In each construction stage, the database further clarifies the key points and difficulties of image monitoring and analysis, as well as the corresponding model performance requirements, to form matching rules. For example, IF stage = "site leveling" THEN focus = "large earthmoving machinery" AND accuracy = "medium" AND efficiency = "high", etc.

[0076] In the above embodiment, the image data management system matches the corresponding key marking areas and the corresponding accuracy and efficiency requirements according to the current construction stage of the project. The focus of image recognition and data management can be determined more accurately according to the characteristics of different construction stages, avoiding the problem of affecting processing efficiency due to too high requirements or affecting recognition accuracy due to too low requirements. At the same time, matching the construction stage with the accuracy and efficiency requirements can make the entire image data management process more intelligent and adaptive, reduce manual intervention, and improve the efficiency and reliability of project supervision.

[0077] S210. Determine a target deep learning model from a plurality of pre-trained deep learning models of different image recognition granularities based on accuracy level requirements and efficiency level requirements of model detection.

[0078] This step is the same as step S102 and will not be repeated here.

[0079] S211. Input the preliminary image data into the target deep learning model for target detection and output the recognition result.

[0080] This step is the same as step S103 and will not be repeated here.

[0081] S212: Use an optical flow algorithm to estimate the motion state of pixels between consecutive frames in the preliminary image data to obtain optical flow information.

[0082] Specifically, the image data management system calculates the corresponding points of two consecutive frames of images pixel by pixel based on sparse optical flow or dense optical flow methods, and finds their motion vectors. The collection of these vectors forms an optical flow field, which can reflect the moving speed and direction of the target in the picture.

[0083] Taking the Horn-Schunck optical flow method as an example, the image data management system first assumes that the brightness of pixels on the surface of the same object remains unchanged between consecutive frames, that is, the optical flow field is smooth and continuous. Then, by minimizing the global energy function, the velocity component of each pixel is solved to estimate the displacement of the pixel and obtain the optical flow information.

[0084] S213: Track the moving trajectory of each target object in the video frame according to the optical flow information.

[0085] Based on the optical flow field information obtained in step S212, the image data management system further realizes the continuous tracking of the target in the video. Specifically, the image data management system first detects each target object in the first frame of the video and calibrates their initial positions with bounding boxes. Next, the system superimposes the optical flow field on the current frame and predicts the approximate position of the target in the next frame based on the movement trend of the pixels in the bounding box. With this as the center, the system searches for the precise bounding box of the target in the neighboring area of ​​the next frame, and repeats the above prediction and correction process to update the position and trajectory of the target frame by frame.

[0086] In the above embodiment, the image data management system uses the optical flow algorithm to estimate the motion state of pixels between consecutive frames in the preliminary image data and track the movement trajectory of each target object in the video frame. On the basis of static target recognition, it can further realize the tracking and analysis of dynamic targets, obtain more comprehensive and accurate target state information, provide a reliable basis for safety and quality control, and greatly improve the automation level of engineering supervision.

[0087] The image data management system of the embodiment of the present invention is applied to electronic equipment. Figure 3 A schematic diagram of the architecture of an electronic device suitable for implementing an embodiment of the present invention is shown.

[0088] It should be noted that Figure 3 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0089] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments can be completed by instructions (computer programs), or by controlling related hardware through instructions (computer programs), and the instructions can be stored in a computer-readable storage medium and loaded and executed by a processor. The electronic device of this embodiment includes a storage medium and a processor, wherein a plurality of instructions are stored in the storage medium, and the instructions can be loaded by the processor to execute any step of the method provided in the embodiment of the present invention.

[0090] Specifically, the storage medium and the processor are electrically connected directly or indirectly to realize data transmission or interaction. For example, these elements can be electrically connected to each other through one or more signal lines. The storage medium stores computer execution instructions for implementing the data access control method, including at least one software function module that can be stored in the storage medium in the form of software or firmware. The processor executes various functional applications and data processing by running the software program and module stored in the storage medium. The storage medium can be, but is not limited to, random access storage medium (Random Access Memory, referred to as: RAM), read-only storage medium (Read Only Memory, referred to as: ROM), programmable read-only storage medium (Programmable Read-Only Memory, referred to as: PROM), erasable read-only storage medium (Erasable Programmable Read-Only Memory, referred to as: EPROM), electrically erasable read-only storage medium (Electric Erasable Programmable Read-Only Memory, referred to as: EEPROM), etc. Among them, the storage medium is used to store programs, and the processor executes the program after receiving the execution instruction.

[0091] Furthermore, the software programs and modules in the above-mentioned storage medium may also include an operating system, which may include various software components and / or drivers for managing system tasks (such as memory management, storage device control, power management, etc.), and may communicate with various hardware or software components to provide an operating environment for other software components. The processor may be an integrated circuit chip having signal processing capabilities. The above-mentioned processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc., which may implement or execute the various methods, steps, and logic flow diagrams disclosed in this embodiment. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0092] Since the instructions stored in the storage medium can execute the steps in any method provided in the embodiments of the present invention, the beneficial effects of any method provided in the embodiments of the present invention can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0093] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. An image data management method based on an engineering supervision platform, applied to an image data management system, characterized in that: The method comprises: The original image data of the construction site is collected by monitoring cameras and cleaned to obtain preliminary image data; According to the accuracy level and efficiency level requirements of model detection, determine the target deep learning model from multiple pre-trained deep learning models of different image recognition granularities; Inputting the preliminary image data into the target deep learning model for target detection, and outputting a recognition result, wherein the recognition result includes a target object and feature data of the target object; Based on a preset rule engine, the characteristic data of the target object is audited and verified to obtain a verification result; A final marking result of the original image data is generated according to the verification result.

2. The method according to claim 1, characterized in that: Before the step of determining the target deep learning model from a plurality of pre-trained deep learning models of different image recognition granularities according to the accuracy level requirement and efficiency level requirement of the model detection, the method further includes: Collect sample image data at different stages of construction and annotate them to obtain a training data set; Model training is performed at different network levels according to the training data set to obtain the pre-trained deep learning models at different image recognition granularities, and different deep learning models correspond to different network levels; According to the test results of the deep learning model on the test set, the accuracy level and efficiency level corresponding to the deep learning model at different image recognition granularities are divided.

3. The method according to claim 2, characterized in that The step of performing model training at different network levels according to the training data set to obtain the pre-trained deep learning model at different image recognition granularities specifically includes: Constructing an initial model of the deep learning model based on a convolutional neural network, a region proposal network, and a feature pyramid network; Using the training data set to train the initial model, determining model parameters corresponding to different image recognition granularities; The pre-trained deep learning model at different image recognition granularities is determined based on the model parameters.

4. The method according to claim 1, characterized in that: Before the step of determining the target deep learning model from a plurality of pre-trained deep learning models of different image recognition granularities according to the accuracy level requirement and efficiency level requirement of the model detection, the method further includes: Get the construction phase corresponding to the current project; According to the construction stage, key areas for marking are matched from the database, as well as the accuracy level requirements and efficiency level requirements corresponding to the key areas.

5. The method according to claim 1, characterized in that Before the step of determining the target deep learning model from a plurality of pre-trained deep learning models of different image recognition granularities according to the accuracy level requirement and efficiency level requirement of the model detection, the method further includes: Get the text description information uploaded by the user; Using a natural language processing model to analyze the text description information and determine the key areas for marking; The corresponding accuracy level requirements and efficiency level requirements are matched from the database according to the key areas.

6. The method according to claim 1, characterized in that After the step of inputting the preliminary image data into the target deep learning model for target detection and outputting the recognition result, the method further includes: Using an optical flow algorithm to estimate the motion state of pixels between consecutive frames in the preliminary image data to obtain optical flow information; The moving trajectory of each target object in the video frame is tracked according to the optical flow information.

7. The method according to claim 1, characterized in that The step of collecting the original image data of the construction site through the monitoring camera and performing data cleaning to obtain preliminary image data specifically includes: Collect raw image data of the construction site through surveillance cameras; Acquire parameter information corresponding to the original image data, where the parameter information is a picture frame parameter that affects target detection by the deep learning model; The parameter information corresponding to the original image data is scored based on a preset scoring system, and the original image data with a score higher than a preset threshold is cleaned to obtain preliminary image data.

8. An image data management system, characterized in that: The image data management system includes: one or more processors and memory; The memory is coupled to the one or more processors, and the memory is used to store computer program code, wherein the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the image data management system to perform the method according to any one of claims 1-7.

9. A computer-readable storage medium comprising instructions, characterized in that: When the instruction is executed on the image data management system, the image data management system is caused to execute the method according to any one of claims 1 to 7.

10. A computer program product, characterized in that When the computer program product is run on an image data management system, the image data management system is caused to execute the method according to any one of claims 1 to 7.

Images