Operation site object marking system and method based on image segmentation

Abstract

The invention relates to the technical field of image recognition, in particular to a work site object marking system and method based on image segmentation, and the system comprises a video monitoring module, an image processing module, an image segmentation module, a marking classification module, a data management module and a user interaction module. Image features can be automatically learned, and pixel-level accurate segmentation and marking of an object are realized. The deep learning model is trained through a large amount of labeling data, objects in various complex scenes can be recognized, and high-precision segmentation masks are generated, so that the precision and reliability of object labeling are remarkably improved, and the problem that a traditional field operation labeling mode is low in precision is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and particularly to an object marking system and method for a work site based on image segmentation. Background Art

[0002] Image segmentation is a basic task in the field of computer vision, aiming to divide an image into multiple regions with similar attributes. In recent years, with the rise of deep learning technology, significant progress has been made in image segmentation technology. Image segmentation methods based on deep learning can automatically learn image features and achieve accurate segmentation. These methods not only have high segmentation accuracy but also strong adaptability and can handle various complex scenarios.

[0003] Traditional object marking and recognition in a work site mainly rely on manual operations, such as directly writing labels on objects with a marker pen or using measuring tools to determine the size and position of objects. These methods are not only time-consuming and laborious but also easily affected by human factors, resulting in inaccurate marking or recognition errors. Especially in a complex work environment, the difficulty and error rate of manual operations will increase significantly. Summary of the Invention

[0004] The purpose of the present invention is to provide an object marking system and method for a work site based on image segmentation, aiming to solve the problem of low accuracy in traditional on-site work marking methods.

[0005] To achieve the above purpose, in the first aspect, the present invention provides an object marking system for a work site based on image segmentation, including a video monitoring module, an image processing module, an image segmentation module, a marking classification module, a data management module, and a user interaction module, which are connected in sequence;

[0006] The video monitoring module obtains image data of the work site in real time;

[0007] The image processing module performs preprocessing operations on the collected images;

[0008] The image segmentation module is used for pixel-level segmentation of objects in the image;

[0009] The marking classification module is used for marking and classifying the segmented objects;

[0010] The data management module is used for storing the original images, preprocessed images, segmentation results, and object marking and classification information;

[0011] The user interaction module is used to display the object markings and classification results to the safety supervisors in an intuitive manner.

[0012] Among them, the video monitoring module includes a camera array unit, a video encoding and compression unit, a real-time streaming media transmission unit, and a camera management unit. The camera array unit is respectively connected to the video encoding and compression unit and the camera management unit, and the real-time streaming media transmission unit is connected to the video encoding and compression unit;

[0013] The camera array unit is used to comprehensively monitor the operation site to obtain video data;

[0014] The video encoding and compression unit is used to efficiently encode and compress the collected video data;

[0015] The real-time streaming media transmission unit is used to transmit the video data to the image processing module in real time;

[0016] The camera management unit is used for camera calibration, maintenance, fault detection and alarm.

[0017] Among them, the image processing module includes a filtering unit, an enhancement unit, and an image cropping and scaling unit. The filtering unit, the enhancement unit, and the image cropping and scaling unit are connected in sequence;

[0018] The filtering unit is used to remove random noise and interference in the video data;

[0019] The enhancement unit adjusts the brightness and contrast of the video data;

[0020] The image cropping and scaling unit is used to crop the irrelevant areas in the video data and perform scaling processing on the video data.

[0021] Among them, the image segmentation module includes a deep learning model unit, a category recognition unit, and a post-processing optimization unit. The deep learning model unit, the category recognition unit, and the post-processing optimization unit are connected in sequence;

[0022] The deep learning model unit is used to perform pixel-by-pixel analysis on the input image to generate a segmentation mask of the object;

[0023] The category recognition unit combines the segmentation mask and the built-in category recognition ability of the model to assign the correct category label to the segmented object;

[0024] The post-processing optimization unit is used to perform smoothing processing and small area merging operations on the segmentation results.

[0025] Among them, the marking classification module includes a marking rule library, a marking and classification unit, and a marking result verification unit, which are connected in sequence;

[0026] The marking rule library stores preset marking rules and classification criteria;

[0027] The marking and classification unit assigns marking labels to each object according to the output result, marking rules, and classification criteria, and classifies them into corresponding categories;

[0028] The marking result verification unit verifies the correctness of the marking result based on a machine learning algorithm, and discovers and corrects errors in a timely manner.

[0029] Among them, the data management module includes a database management unit, a data access control unit, and a data analysis and visualization unit. The database management unit is respectively connected to the data access control unit and the data analysis and visualization unit;

[0030] The database management unit is used to store image data, segmentation results, and marking classification information;

[0031] The data access control unit is used to set user permissions and data access policies;

[0032] The data analysis and visualization unit provides data analysis reports, trend charts, and statistical charts based on the stored data.

[0033] In a second aspect, a method for marking objects at a job site based on image segmentation is used for the system for marking objects at a job site based on image segmentation described in the first aspect, and includes the following steps:

[0034] Real-time acquisition of image data of the job site through a video monitoring device installed at the job site;

[0035] Perform preprocessing operations on the collected images;

[0036] Input the preprocessed image into a trained image segmentation model, and use the model to perform pixel-level segmentation on the objects in the image;

[0037] Mark and classify the segmented objects according to the output result of the image segmentation module;

[0038] Store the collected original images, preprocessed images, segmentation results, and object marking and classification information into the data management module;

[0039] Through the user interaction module, display the object marking and classification results to the safety supervision personnel in an intuitive manner.

[0040] An object marking system for a work site based on image segmentation according to the present invention includes a video monitoring module, an image processing module, an image segmentation module, a marking classification module, a data management module, and a user interaction module. The video monitoring module, the image processing module, the image segmentation module, the marking classification module, the data management module, and the user interaction module are connected in sequence. The video monitoring module obtains image data of the work site in real time. The image processing module performs preprocessing operations on the collected images. The image segmentation module is used for pixel-level segmentation of objects in the images. The marking classification module is used for marking and classifying the segmented objects. The data management module is used for storing the original images, the preprocessed images, the segmentation results, and the object marking and classification information. The user interaction module is used for presenting the object marking and classification results to safety supervisors in an intuitive manner. Based on the image segmentation technology of deep learning, the present invention can automatically learn image features, achieve pixel-level precise segmentation and marking of objects. The deep learning model is trained with a large amount of labeled data, can recognize objects in various complex scenarios, and generate high-precision segmentation masks, thereby significantly improving the accuracy and reliability of object marking, and further solving the problem of low accuracy of traditional on-site operation marking methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0042] Figure 1 It is a schematic diagram of an object marking system for a work site based on image segmentation provided by the present invention.

[0043] Figure 2 It is a schematic diagram of the video monitoring module.

[0044] Figure 3 It is a schematic diagram of the image processing module.

[0045] Figure 4 It is a schematic diagram of the image segmentation module.

[0046] Figure 5 It is a schematic diagram of the marking classification module.

[0047] Figure 6 It is a schematic diagram of the data management module.

[0048] Figure 7It is a flowchart of a method for marking objects at a job site based on image segmentation provided by the present invention.

[0049] In the figure: 1 - Video monitoring module, 2 - Image processing module, 3 - Image segmentation module, 4 - Marking and classification module, 5 - Data management module, 6 - User interaction module, 11 - Camera array unit, 12 - Video encoding and compression unit, 13 - Real-time streaming media transmission unit, 14 - Camera management unit, 21 - Filtering unit, 22 - Enhancement unit, 23 - Image cropping and scaling unit, 31 - Deep learning model unit, 32 - Category recognition unit, 33 - Post-processing optimization unit, 41 - Marking rule library, 42 - Marking and classification unit, 43 - Marking result verification unit, 51 - Database management unit, 52 - Data access control unit, 53 - Data analysis and visualization unit. Specific implementation manner

[0050] The embodiments of the present invention will be described in detail below. The examples of the embodiments are shown in the drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described by referring to the drawings below are exemplary and are intended to explain the present invention, but should not be construed as limiting the present invention.

[0051] Please refer to Figures 1 to 6 , in the first aspect, the present invention provides an object marking system at a job site based on image segmentation, including a video monitoring module 1, an image processing module 2, an image segmentation module 3, a marking and classification module 4, a data management module 5, and a user interaction module 6. The video monitoring module 1, the image processing module 2, the image segmentation module 3, the marking and classification module 4, the data management module 5, and the user interaction module 6 are connected in sequence;

[0052] The video monitoring module 1 obtains image data of the job site in real time;

[0053] The image processing module 2 performs preprocessing operations on the collected images;

[0054] The image segmentation module 3 is used to perform pixel-level segmentation on the objects in the image;

[0055] The marking and classification module 4 is used to mark and classify the segmented objects;

[0056] The data management module 5 is used to store the original images, preprocessed images, segmentation results, and object marking and classification information;

[0057] The user interaction module 6 is used to display the object marking and classification results to the safety supervision personnel in an intuitive manner.

[0058] In this embodiment, image data of the operation site is obtained in real time through a video monitoring device installed at the operation site; the collected images are preprocessed; the preprocessed images are input into a trained image segmentation model, and the model is used to perform pixel-level segmentation on the objects in the images; according to the output results of the image segmentation module 3, the segmented objects are marked and classified; the collected original images, preprocessed images, segmentation results, and object marking and classification information are stored in the data management module 5; through the user interaction module 6, the object marking and classification results are presented to the safety supervision personnel in an intuitive manner. Based on the deep learning-based image segmentation technology, the present invention can automatically learn image features, realize pixel-level precise segmentation and marking of objects. The deep learning model is trained with a large amount of labeled data, can identify objects in various complex scenarios, and generate high-precision segmentation masks, thereby significantly improving the accuracy and reliability of object marking, and further solving the problem of low accuracy of traditional on-site operation marking methods.

[0059] Further, the video monitoring module 1 includes a camera array unit 11, a video encoding and compression unit 12, a real-time streaming media transmission unit 13, and a camera management unit 14. The camera array unit 11 is respectively connected to the video encoding and compression unit 12 and the camera management unit 14, and the real-time streaming media transmission unit 13 is connected to the video encoding and compression unit 12;

[0060] The camera array unit 11 is used to comprehensively monitor the operation site to obtain video data;

[0061] The video encoding and compression unit 12 is used to efficiently encode and compress the collected video data;

[0062] The real-time streaming media transmission unit 13 is used to transmit the video data to the image processing module 2 in real time;

[0063] The camera management unit 14 is used for calibration, maintenance, fault detection and alarm of the camera.

[0064] In this embodiment, the camera array unit 11 is deployed at key positions in the operation site, with high resolution, wide viewing angle, and night vision function, ensuring all-weather and all-round monitoring of the operation site. The video encoding and compression unit 12 performs efficient encoding and compression on the collected video data, reducing the burden of data transmission and storage while maintaining image quality. The real-time streaming media transmission unit 13 uses protocols such as RTMP (Real Time Messaging Protocol) or HLS (HTTP Live Streaming) to ensure that the video data can be transmitted to the image processing module 2 in real time and stably. The camera management unit 14 is responsible for the calibration, maintenance, fault detection and alarm of the camera, as well as the remote control of the camera (rotation, zoom, focus, etc.).

[0065] Furthermore, the image processing module 2 includes a filtering unit 21, an enhancement unit 22, and an image cropping and scaling unit 23, and the filtering unit 21, the enhancement unit 22, and the image cropping and scaling unit 23 are connected in sequence;

[0066] The filtering unit 21 is used to remove random noise and interference in the video data;

[0067] The enhancement unit 22 adjusts the brightness and contrast of the video data;

[0068] The image cropping and scaling unit 23 is used to crop irrelevant regions in the video data and perform scaling processing on the video data.

[0069] In this embodiment, the filtering unit 21 is a noise removal filter, adopting technologies such as median filtering and Gaussian filtering to effectively remove random noise and interference in the image. The enhancement unit 22 adopts a contrast enhancement algorithm, and automatically adjusts the brightness and contrast of the image according to the image histogram information to improve the visibility of image details. The image cropping and scaling unit 23 crops irrelevant regions in the image and performs scaling processing on the image according to preset rules or user instructions to meet the requirements of subsequent processing. And an image stabilization and anti-shake algorithm is adopted to perform image stabilization processing on the image instability problem caused by camera shake or environmental changes to improve image quality.

[0070] Furthermore, the image segmentation module 3 includes a deep learning model unit 31, a category recognition unit 32, and a post-processing optimization unit 33, and the deep learning model unit 31, the category recognition unit 32, and the post-processing optimization unit 33 are connected in sequence;

[0071] The deep learning model unit 31 is used to perform pixel-by-pixel analysis on the input image to generate a segmentation mask of the object;

[0072] The category recognition unit 32 assigns correct category labels to the segmented objects by combining the segmentation mask and the built-in category recognition ability of the model;

[0073] The post-processing optimization unit 33 is used to perform smoothing processing and small area merging operations on the segmentation result.

[0074] In this embodiment, the deep learning model unit 31 loads a pre-trained image segmentation model, such as U-Net, Mask R-CNN, etc., through a deep learning model loader. The pixel-level segmentation engine analyzes the input image pixel by pixel using the deep learning model to generate a segmentation mask of the object. The category recognition unit 32 assigns correct category labels to the segmented objects by combining the segmentation mask and the built-in category recognition ability of the model. The post-processing optimization unit 33 is a post-processing optimizer that performs operations such as smoothing processing and small area merging on the segmentation result to improve the accuracy and robustness of the segmentation result.

[0075] Further, the marking and classification module 4 includes a marking rule library 41, a marking and classification unit 42, and a marking result verification unit 43, and the marking rule library 41, the marking and classification unit 42, and the marking result verification unit 43 are connected in sequence;

[0076] The marking rule library 41 stores preset marking rules and classification criteria;

[0077] The marking and classification unit 42 assigns marking labels to each object according to the output result, marking rules, and classification criteria, and classifies it into the corresponding category;

[0078] The marking result verification unit 43 verifies the correctness of the marking result based on a machine learning algorithm, and discovers and corrects errors in a timely manner.

[0079] In this embodiment, the marking rule library 41 stores preset marking rules and classification criteria, such as color coding, shape marking, text labels, etc., for guiding the marking and classification of objects. The automatic marking and classification unit 42 automatically assigns marking labels to each object according to the output result of the image segmentation module 3 and the information in the marking rule library 41, and classifies it into the corresponding category. A marking conflict resolution mechanism is adopted: to handle conflict situations in the marking process, such as overlapping markings between different objects, unclear category attribution, etc., to ensure the uniqueness and accuracy of the marking result. The marking result verification unit 43 verifies the correctness of the marking result through a machine learning algorithm, and discovers and corrects errors in a timely manner.

[0080] Further, the data management module 5 includes a database management unit 51, a data access control unit 52, and a data analysis and visualization unit 53. The database management unit 51 is respectively connected to the data access control unit 52 and the data analysis and visualization unit 53;

[0081] The database management unit 51 is used to store image data, segmentation results, and marker classification information;

[0082] The data access control unit 52 is used to set user permissions and data access policies;

[0083] The data analysis and visualization unit 53 provides data analysis reports, trend charts, and statistical charts based on the stored data.

[0084] In this embodiment, the database management unit 51 uses a relational database or a non-relational database to store image data, segmentation results, marker classification information, etc., supporting efficient data query, retrieval, and analysis. And a data backup and recovery mechanism is adopted to back up data regularly to ensure data security and recoverability. In case of data loss or damage, data can be quickly restored. The data access control unit 52 sets user permissions and data access policies to ensure that only authorized users can access and modify data. The data analysis and visualization unit 53 provides visualization tools such as data analysis reports, trend charts, and statistical charts to help users better understand data, discover potential problems, and optimize system performance.

[0085] Please refer to Figure 7 , on the second aspect, a method for marking objects at a job site based on image segmentation, which is used for the system for marking objects at a job site based on image segmentation described in the first aspect, includes the following steps:

[0086] S1: Real-time obtain image data of the job site through a video monitoring device installed at the job site;

[0087] Specifically, image data of the job site is collected in real time through a high-definition camera array, efficiently encoded and compressed by the video encoding and compression unit 12, and the data is transmitted to the image processing module 2 through a real-time streaming media transmission protocol.

[0088] S2: Perform preprocessing operations on the collected images;

[0089] Specifically, in the image processing module 2, preprocessing operations such as noise removal, contrast enhancement, cropping, and scaling are performed on the collected images to improve the segmentability and segmentation accuracy of the images.

[0090] S3: Input the preprocessed image into a trained image segmentation model, and use the model to perform pixel-level segmentation on the objects in the image;

[0091] Specifically, in the image segmentation module 3, a deep learning model is used to perform pixel-level segmentation on the preprocessed image, generate a segmentation mask of the object, and assign the correct class label to the segmented object in combination with the class recognition unit 32.

[0092] S4 marks and classifies the segmented objects according to the output result of the image segmentation module 3;

[0093] Specifically, in the marking and classification module 4, according to the information in the marking rule library 41 and the processing result of the automatic marking and classification engine, a marking label is assigned to each object and it is classified into the corresponding class. At the same time, the marking and classification results are associated with the segmentation mask of the object to form complete object marking information.

[0094] S5 stores the collected original image, the preprocessed image, the segmentation result, and the object marking and classification information in the data management module 5;

[0095] Specifically, in the data management module 5, the collected original image, the preprocessed image, the segmentation result, and the object marking and classification information are stored in the database. The system classifies and stores the data according to dimensions such as time sequence and operation area, and provides data query and management interfaces.

[0096] S6 displays the object marking and classification results to the safety supervision personnel in an intuitive manner through the user interaction module 6.

[0097] Specifically, through the user interaction module 6, the object marking and classification results are displayed to the safety supervision personnel in an intuitive manner. The user can view the monitoring screen of the operation site in real time, query the object marking information, set system parameters, and perform operation control, etc. through this module. When potential safety hazards are found, the system can issue an alarm in time and push the relevant information to the safety supervision personnel so that they can take measures in time for processing.

[0098] Beneficial effects:

[0099] First, significantly improve the accuracy and reliability of object marking: Through the deep learning-based image segmentation technology, the present invention can automatically learn image features to achieve pixel-level precise segmentation and marking of objects. The deep learning model is trained with a large amount of labeled data, can recognize objects in various complex scenarios, and generate high-precision segmentation masks, thereby significantly improving the accuracy and reliability of object marking.

[0100] II. Achieving efficient processing and utilization of on-site monitoring data: Through the collaborative work of components such as the video monitoring module, image processing module, and image segmentation module, the present invention realizes the efficient collection, preprocessing, segmentation, and marking of on-site monitoring data. The data management module provides efficient data storage, access control, and visualization tools to help users better understand the data, discover potential problems, and optimize system performance.

[0101] III. Improving the on-site safety management level: The user interaction module displays the object marking and classification results in an intuitive manner, enabling safety supervisors to view the on-site monitoring images and object marking information in real time. When potential safety hazards are detected, the system can issue an alarm in a timely manner and push the relevant information to the safety supervisors so that they can take timely measures for handling. By improving the accuracy and reliability of the marking results, the present invention helps to reduce false alarms and missed alarms, and improve the overall on-site safety management level.

[0102] IV. Enhancing the flexibility and scalability of the system: The various modules of the present invention adopt a modular design, which is convenient for system maintenance and upgrade. The deep learning model can be fine-tuned or retrained according to actual needs to meet the requirements of different operation scenarios and object types. The data management module supports multiple database types and data access strategies, facilitating the flexible configuration and expansion of the system.

[0103] The above-disclosed is only a preferred embodiment of an object marking system and method for an operation site based on image segmentation of the present invention. Of course, it cannot be used to limit the scope of the rights of the present invention. Those of ordinary skill in the art can understand all or part of the processes of implementing the above embodiments, and the equivalent changes made according to the claims of the present invention still fall within the scope covered by the invention.

Claims

1. An object marking system for a job site based on image segmentation, characterized in that, It includes a video monitoring module, an image processing module, an image segmentation module, a marking and classification module, a data management module, and a user interaction module. The video monitoring module, the image processing module, the image segmentation module, the marking and classification module, the data management module, and the user interaction module are connected in sequence; The video monitoring module obtains image data of the operation site in real time; The image processing module performs preprocessing operations on the collected images; The image segmentation module is used to perform pixel-level segmentation on the objects in the image; The marking and classification module is used to mark and classify the segmented objects; The data management module is used to store the original images, preprocessed images, segmentation results, and object marking and classification information; The user interaction module is used to display the object marking and classification results to the safety supervision personnel in an intuitive manner.

2. The object marking system for an operation site based on image segmentation according to claim 1, characterized in that The video monitoring module includes a camera array unit, a video encoding and compression unit, a real-time streaming media transmission unit, and a camera management unit. The camera array unit is respectively connected to the video encoding and compression unit and the camera management unit, and the real-time streaming media transmission unit is connected to the video encoding and compression unit; The camera array unit is used to comprehensively monitor the operation site to obtain video data; The video encoding and compression unit is used to efficiently encode and compress the collected video data; The real-time streaming media transmission unit is used to transmit the video data to the image processing module in real time; The camera management unit is used for calibration, maintenance, fault detection, and alarm of the camera.

3. The object marking system for an operation site based on image segmentation according to claim 2, characterized in that The image processing module includes a filtering unit, an enhancement unit, and an image cropping and scaling unit. The filtering unit, the enhancement unit, and the image cropping and scaling unit are connected in sequence; The filtering unit is used to remove random noise and interference in the video data; The enhancement unit adjusts the brightness and contrast of the video data; The image cropping and scaling unit is used to crop the irrelevant areas in the video data and perform scaling processing on the video data.

4. The object marking system for an operation site based on image segmentation according to claim 3, characterized in that The image segmentation module includes a deep learning model unit, a category recognition unit, and a post-processing optimization unit. The deep learning model unit, the category recognition unit, and the post-processing optimization unit are connected in sequence; The deep learning model unit is used to perform pixel-by-pixel analysis on the input image to generate a segmentation mask of the object; The category recognition unit combines the segmentation mask and the built-in category recognition ability of the model to assign the correct category label to the segmented object; The post-processing optimization unit is used to perform smoothing processing and small area merging operations on the segmentation results.

5. The object marking system for an operation site based on image segmentation according to claim 4, characterized in that The marking and classification module includes a marking rule library, a marking and classification unit, and a marking result verification unit, which are connected in sequence; The marking rule library stores preset marking rules and classification criteria; The marking and classification unit assigns marking labels to each object according to the output result, marking rules, and classification criteria, and classifies them into corresponding categories; The marking result verification unit verifies the correctness of the marking result based on the machine learning algorithm, and discovers and corrects errors in a timely manner.

6. The object marking system for the work site based on image segmentation according to claim 5, wherein The data management module includes a database management unit, a data access control unit, and a data analysis and visualization unit, and the database management unit is respectively connected to the data access control unit and the data analysis and visualization unit; The database management unit is used to store image data, segmentation results, and marking classification information; The data access control unit is used to set user permissions and data access policies; The data analysis and visualization unit provides data analysis reports, trend charts, and statistical charts based on the stored data.

7. A method for marking objects at a job site based on image segmentation, which is used for the system for marking objects at a job site based on image segmentation according to any one of claims 1-6, characterized in that, It includes the following steps: Real-time acquisition of image data of the work site through video monitoring devices installed at the work site; Perform preprocessing operations on the collected images; Input the preprocessed images into the trained image segmentation model, and use the model to perform pixel-level segmentation on the objects in the images; Mark and classify the segmented objects according to the output result of the image segmentation module; Store the collected original images, preprocessed images, segmentation results, and object marking and classification information in the data management module; Through the user interaction module, display the object marking and classification results to the safety supervision personnel in an intuitive manner.

Images