Factory environment 3D space intelligent reconstruction and risk identification method and identification system

Abstract

The invention relates to the technical field of industrial safety monitoring, in particular to a factory environment 3D space intelligent reconstruction and risk identification method and system, and the method comprises the steps: 1, obtaining factory environment data, and carrying out the preprocessing of the factory environment data; 2, generating an initial factory 3D model based on the preprocessed factory environment data; 3, acquiring equipment label image data in a factory, and processing the equipment label image data by using a deep learning algorithm to form a local three-dimensional space structure; 4, matching and registering the local three-dimensional space structure with the initial factory 3D model to obtain a factory 3D space model containing equipment information; and step 5, obtaining operation state data of the operation personnel, and performing identification and early warning on operation risks according to the operation state data and the factory 3D space model. According to the invention, 3D space intelligent reconstruction of a complex environment of a factory is realized, deep reasoning and generation are carried out on a local space by using a 3D Gaussian splashing technology, matching is carried out in combination with overall 3D point cloud data of the factory, a high-precision and high-detail 3D space model is generated, and accurate space basic data is provided for subsequent risk identification.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of industrial safety monitoring, in particular to a 3D space intelligent reconstruction and risk identification method and system for a factory environment. BACKGROUND

[0002] With the increasing industrialization, the modern factory environment is becoming more and more complex, and the layout and frequent changes of production facilities and equipment. In the production process of the factory, especially in the complex environment such as pipeline valves, high radiation areas, and closed spaces, the traditional risk identification method mainly relies on manual inspection and experience judgment, which is difficult to accurately identify and timely warn the operation risk. For example, in the high radiation area, manual inspection has the risk of radiation exposure and low efficiency; when working in a closed space, due to the narrow space, poor ventilation and other factors, the traditional method is difficult to monitor the safety status of the workers in real time.

[0003] In the prior art, the risk identification method based on two-dimensional plan or simple sensor data cannot comprehensively consider the position relationship, posture, size and other factors of equipment and personnel in three-dimensional space, and it is difficult to effectively identify the risks in complex environments such as high-altitude operation and equipment collision which cannot be found by traditional methods. SUMMARY

[0004] The present application provides a 3D space intelligent reconstruction and risk identification method and system for a factory environment, which is used to solve the problem that the risk identification method in the prior art cannot consider the risks in complex environments such as high-altitude operation and equipment collision in three-dimensional space.

[0005] The technical scheme of the present application is as follows:

[0006] The present application provides a 3D space intelligent reconstruction and risk identification method and system for a factory environment, which is used to solve the problem that the risk identification method in the prior art cannot consider the risks in complex environments such as high-altitude operation and equipment collision in three-dimensional space.

[0007] Step 1: Obtain factory environment data and pre-process the factory environment data;

[0008] Step 2: Generate an initial factory 3D model based on the pre-processed factory environment data;

[0009] Step 2.1: Generate overall 3D point cloud data of the factory environment based on the pre-processed factory environment data;

[0010] Step 2.2: Convert the 3D point cloud data into a Gaussian distribution using 3D Gaussian splashing technology to generate an initial factory 3D model;

[0011] Step 2.3: Check and correct the initial factory 3D model;

[0012] Step three: obtain equipment tag image data in the factory, process the equipment tag image data using a deep learning algorithm to form a local three-dimensional spatial structure;

[0013] Step 3.1: Obtain equipment tag image data in the factory using a shooting device, focusing on obtaining equipment tag image data in key areas;

[0014] Step 3.2: Preprocess the equipment tag image data

[0015] Step 3.3: Convert character information in the extracted image into text information;

[0016] Step 3.4: Use a deep learning algorithm to perform local spatial depth reasoning based on the equipment tag image data to obtain local 3D point cloud data of the equipment tag and its surrounding environment, forming a local three-dimensional spatial structure;

[0017] Step four: match and register the local three-dimensional spatial structure with the initial factory 3D model to obtain a factory 3D spatial model containing equipment information;

[0018] Step five: Obtain work state data of workers, and identify and warn work risks based on work state data and factory 3D spatial model.

[0019] In some embodiments, the factory environment data in step one is collected by multi-view cameras and laser scanners, focusing on complex environments such as pipeline valves, high radiation areas, and closed spaces; preprocessing the factory environment data specifically includes cleaning, filtering, and completing the factory environment data.

[0020] In some embodiments, in step 2.1, the overall 3D point cloud data of the factory environment is generated based on the preprocessed factory environment data, specifically including: for multi-view cameras, using matching points in image data and camera pose information, calculating the position of each pixel point in three-dimensional space based on multi-view camera image data, generating three-dimensional coordinates corresponding to each pixel point, thereby obtaining 3D point cloud data of the entire factory; for laser scanners, by emitting laser beams and receiving returned signals, using time difference or phase difference of signals to measure distance of object surface, obtaining three-dimensional coordinates of objects, obtaining three-dimensional coordinates of each pixel point in laser scanner image.

[0021] In some embodiments, in step 2.1, in generating the overall 3D point cloud data of the factory environment based on the preprocessed factory environment data, the data of multi-view cameras and laser scanners are combined, the precise geometry of the factory environment is obtained using laser scanning data, the details of the 3D point cloud data are enhanced using visual information of multi-view cameras, and the 3D point cloud data is converted into a three-dimensional mesh model and added with texture.

[0022] In some embodiments, the 3D point cloud data is converted into a Gaussian distribution in step 2.2 and an initial factory 3D model is generated using a 3D Gaussian splatting technique, specifically including: converting each point data of the 3D point cloud data into a 3D Gaussian function through the Gaussian splatting technique, defining the position μ, covariance matrix Σ, color RGB and opacity α of each point data; when converting the Gaussian function using the Gaussian splatting technique, a Gaussian function density control strategy is performed on the preset key area to increase the number of Gaussian functions in the key area.

[0023] In some embodiments, the key area is a pipeline valve dense area, a high radiation area and an airtight space entrance; step two generates an initial factory 3D model, specifically including: when converting the 3D point cloud data into a Gaussian distribution, for different view 3D point cloud data generated by multi-view cameras and laser scanners, different view Gaussian functions are registered and fused through a multi-view fusion algorithm to eliminate differences and redundant information between views and form a consistent initial factory 3D model; after generating the initial factory 3D model, a consistency check is performed on the initial factory 3D model to ensure that the fused model has consistency in geometry, color and texture, and the initial factory 3D model is refined and edited using a 3D Gaussian splatting technique, the surface details of the model are enhanced or smoothed by adjusting the parameters of the Gaussian function; the local structure of the model is modified by adding or deleting Gaussian functions.

[0024] In some embodiments, in step 3.1, a handheld or stand-mounted camera is used to capture equipment tag image data in the factory, and feature matching of multi-view images is used to enhance the consistency of image data point clouds; in step 3.2, the equipment tag image data is preprocessed, including image enhancement, denoising and edge detection; in step 3.3, an OCR algorithm is used to identify and extract the equipment number and model number in the tag image; in step 3.4, a local three-dimensional spatial structure is formed, specifically including:

[0025] A deep learning model for predicting a pixel-level depth map from a single image is constructed, and the model output is the depth value Z(u,v) of each pixel, and the model is specifically as formula (1):

[0026] Z=f θ (I) (1)

[0027] Where (u,v) is the horizontal and vertical coordinates of the pixel, Z(u,v) is the depth value of the pixel (u,v), is a depth estimation network, and I represents the input RGB image. The depth learning model is trained using the depth map in the factory environment data as a data set, and the depth estimation accuracy is enhanced by a geometric consistency loss, and the loss function of the geometric consistency loss is as formula (2):

[0028] L vn =∑ p Pn(p)-n gt (p)P2 (2)

[0029] Among them, L vn Let be the geometric consistency loss, be the normal vector of the predicted point p, and be the true normal vector. Then, the equipment sign image data is input into the deep learning model to obtain the equipment sign image data depth map. Based on the camera intrinsic parameters, the equipment sign image data depth map is converted into a 3D point cloud to form a local three-dimensional spatial structure. The conversion formula is as shown in formula (3):

[0030]

[0031] Where X, Y, Z are the 3D point cloud coordinates corresponding to pixel coordinates (u, v), is the optical center of the camera, and is the focal length of the camera.

[0032] In some embodiments, step four involves matching and registering the local 3D spatial structure with the initial factory 3D model to achieve precise alignment of the local and overall point cloud data, resulting in a factory 3D spatial model containing equipment information. Specifically, this includes: using an iterative nearest-point algorithm to perform initial and fine matching of the point cloud data of the local 3D spatial structure and the initial factory 3D model to determine the equipment location information of the local 3D spatial structure; associating and storing the equipment location information of the 3D spatial structure with equipment operating parameters, maintenance records, and other data to obtain the factory 3D spatial model. The iterative nearest-point algorithm specifically includes: matching each point cloud data of the local 3D spatial structure with the nearest point in the point cloud data of the initial factory 3D model based on the initial matching results; for each pair of matched points, calculating the rigid transformation adjustment and recalculating the distance error; then adjusting the rigid transformation parameter based on the distance error result, iterating until the distance error converges to a threshold.

[0033] In some embodiments, the operational risks in step five include: fall from height risk, high radiation risk, path error risk, equipment contact risk, and electromagnetic interference risk. For fall from height risk, real-time monitoring of sensor data from personal protective equipment (PPE) is conducted when workers are working at heights. Based on the 3D spatial model and sensor data from the PPE, it is determined whether the worker is at risk of a fall from height. For high radiation risk, the location and intensity information of radiation sources in the 3D spatial model, combined with the worker's location data, is used to calculate the radiation dose at the worker's location in real time. Intensity data of radiation sources is obtained through radiation monitoring sensors, and combined with the geometric relationships in the 3D spatial model, the radiation dose at different locations is calculated. When the radiation dose exceeds a safety threshold, a high radiation risk warning is issued, and workers are guided to evacuate the danger zone. For path error risk, the movement path of workers is monitored in real time through the equipment layout and work area division in the 3D spatial model, and the personnel positioning system is used to obtain the location and intensity data of radiation sources. The system uses real-time location data of operators, combined with area division information in the 3D spatial model, to determine whether operators have mistakenly entered undesignated areas or gone to the wrong intervals. For the risk of accidental equipment contact, it uses equipment location data from the 3D spatial model and operator location data to monitor the distance between operators and equipment in real time. Through the personnel positioning system and equipment location data, it calculates the real-time distance between operators and equipment. When an operator approaches the safe distance of the equipment, it issues a risk warning of accidental equipment contact and reminds the operator to pay attention to safety. For the risk of electromagnetic interference, it uses the location and intensity information of electromagnetic equipment in the 3D spatial model, combined with operator location data, to calculate the electromagnetic interference intensity at the operator's location in real time. It uses electromagnetic monitoring sensors to obtain the intensity data of electromagnetic equipment and combines it with the geometric relationships in the 3D spatial model to calculate the electromagnetic interference intensity at different locations of the operator. When the electromagnetic interference intensity exceeds the safety threshold, it issues an electromagnetic interference risk warning and reminds the operator to take protective measures.

[0034] This invention proposes a 3D spatial intelligent reconstruction and risk identification system for factory environment. The system includes an environmental data acquisition module, a factory model generation module, an equipment embedding module, a registration optimization module, and a risk warning module. The environmental data acquisition module is used to acquire factory environmental data and preprocess the factory environmental data.

[0035] The factory model generation module is used to generate overall 3D point cloud data of the factory environment based on the preprocessed factory environment data, and to convert the 3D point cloud data into a Gaussian distribution using 3D Gaussian splashing technology to generate an initial 3D factory model.

[0036] The device embedding module is used to acquire equipment nameplate image data in the factory. It uses deep learning algorithms to perform local spatial depth reasoning based on the equipment nameplate image data to obtain local 3D point cloud data of the equipment nameplate and its surrounding environment, forming a local three-dimensional spatial structure.

[0037] The registration optimization module is used to match and register the local 3D spatial structure with the initial factory 3D model, so as to achieve accurate alignment between local and overall point cloud data and obtain a factory 3D spatial model containing equipment information.

[0038] The risk warning module is used to acquire the work status data of the operators and to identify and warn of work risks based on the work status data and the 3D spatial model of the factory.

[0039] The implementation of this invention has the following beneficial effects:

[0040] 1. This invention proposes a method and system for intelligent 3D spatial reconstruction and risk identification of factory environments. Based on AIGC technology, this invention realizes intelligent 3D spatial reconstruction of complex factory environments. It uses 3D Gaussian splashing technology to perform deep reasoning and generation of local spaces, and combines it with the overall 3D point cloud data of the factory to generate a high-precision, high-detail 3D spatial model, providing accurate spatial basis data for subsequent risk identification.

[0041] 2. This invention proposes a 3D spatial intelligent reconstruction and risk identification method and system for factory environments. This invention utilizes image recognition technology to calibrate equipment positions. By photographing equipment labels and identifying key information therein, combined with local 3D point cloud data generated by 3D Gaussian splashing technology, the equipment position is accurately matched and anchored to 3D space. This can accurately obtain the spatial position and related attribute information of the equipment, providing a key basis for risk assessment.

[0042] 3. This invention proposes a 3D spatial intelligent reconstruction and risk identification method and system for factory environments. This invention enables risk identification and early warning for high-risk operations, including risks such as falls from heights, high radiation, misalignment of work areas, accidental equipment contact, and electromagnetic interference. By real-time monitoring of the operator's position and posture, combined with equipment layout and risk source information in the 3D spatial model, timely risk warnings are issued to ensure operator safety. (See attached figures.)

[0043] Figure 1 The figures are those presented in the embodiments of the present invention;

[0044] Figure 2 The figure is provided as an embodiment of the present invention. Detailed Implementation

[0045] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] like Figure 1 As shown, this invention proposes a method for intelligent 3D spatial reconstruction and risk identification of factory environments, which includes:

[0047] Step 1: Acquire and preprocess factory environmental data. This data is collected using multi-view cameras or laser scanners. Special attention is paid to complex environments such as pipelines and valves, high-radiation areas, and confined spaces to ensure the collected data accurately reflects the characteristics of these areas. For example, in high-radiation areas, data showing the location and range of radiation sources is needed; in confined spaces, data demonstrating the internal structure and equipment layout is required.

[0048] Preprocessing of the collected data mainly includes data cleaning, filtering, and completion. Data cleaning removes noise and outliers to improve data quality; filtering smooths the data and reduces high-frequency noise; and completion fills in missing parts of the data to make it more complete. For example, outliers in point cloud data can be removed using statistical analysis methods; and blurred parts in image data can be processed using image enhancement algorithms.

[0049] Step 2: Generate an initial 3D model of the factory based on the preprocessed factory environment data.

[0050] Step 2.1: Generate overall 3D point cloud data of the factory environment based on the preprocessed factory environment data. Specifically, this includes: When obtaining factory environment data through multi-view cameras, different cameras need to be calibrated to ensure that the camera's intrinsic and extrinsic parameters, such as focal length, principal point position, and the positional relationship between cameras, are known, providing a data foundation for subsequent calculations. Using matching points in the images and camera pose information, the position of each pixel in 3D space is calculated based on the multi-view image data, generating the corresponding 3D coordinates for each pixel, thus obtaining the overall 3D point cloud data of the factory.

[0051] When using a laser scanner to acquire 3D point cloud data, a laser beam is emitted and the returned signal is received. The distance to the object's surface is measured using time difference or phase difference to obtain the object's 3D coordinates. This yields the 3D coordinates of each pixel in the laser scanner image, generating accurate 3D data. Multiple scans are performed to register data from different locations (aligning point clouds from different scan positions) to form a complete 3D dataset. Furthermore, multi-view cameras and laser scanners can be combined to acquire even more accurate environmental data. Laser scan data can be used to obtain the precise geometry of the factory environment, while camera visual information enhances the details of the point cloud. By converting the point cloud data into a 3D mesh model and adding textures, the 3D reconstruction effect is improved.

[0052] Step 2.2: Utilize 3D Gaussian splashing technology to convert 3D point cloud data into a Gaussian distribution, generating an initial 3D factory model. Specifically, this involves converting each point data point in the 3D point cloud data into a 3D Gaussian function using Gaussian splashing technology, defining its position (μ), covariance matrix (Σ), color (RGB), and opacity (α). 3D Gaussian splashing technology controls the Gaussian shape through the covariance matrix, covering unsampled areas of the point cloud and achieving continuous surface interpolation. This effectively overcomes the limitations of point clouds and overcomes the lack of surface continuity and geometric details through surface interpolation, enabling detailed processing such as smooth transitions at pipe edges in the image.

[0053] During the 3D point cloud data conversion process, a density control strategy is implemented for preset key areas. While maintaining the overall scene effect, the number of Gaussian functions in key areas is increased to improve the detail representation of these areas. In the complex environment of a factory, to balance rendering quality and computational efficiency, the number and distribution of Gaussian functions need to be dynamically adjusted. Through the density control strategy, the number of Gaussian functions in key areas (such as densely populated areas of pipes and valves, high-radiation areas, and entrances to enclosed spaces) can be increased to improve the detail representation of these areas while maintaining the overall scene effect. For example, in densely populated areas of pipes and valves, increasing the number of Gaussian functions and decreasing the value of the covariance matrix can make the outlines and details of the pipes and valves clearer; in high-radiation areas, adjusting the distribution of Gaussian functions can better represent the location and range of radiation sources; and at the entrance of an enclosed space, increasing the number of Gaussian functions can more accurately display the shape and size of the entrance.

[0054] Due to the complexity of factory environments, data is typically collected from multiple perspectives. To fuse data from different perspectives and generate a consistent 3D spatial model, a multi-view fusion algorithm is used to register and fuse Gaussian functions from different perspectives, eliminating differences and redundant information between perspectives, and ultimately generating an initial 3D factory model.

[0055] Step 2.3: Check and correct the initial factory 3D model. This includes: after generating the initial factory 3D model, performing a consistency check to ensure the merged model has consistency in geometry, color, and texture. For example, when dealing with large equipment in a factory, data collected from different perspectives may show differences in equipment shape and location. Through multi-view fusion and consistency checks, an accurate and complete equipment model can be generated. After generating the initial factory 3D model, 3D Gaussian splashing technology can be used to refine and edit the model. By adjusting the parameters of the Gaussian function, the surface details of the model can be enhanced or smoothed; or by adding or deleting Gaussian functions, the local structure of the model can be modified. Furthermore, other technologies, such as deep learning algorithms, can be combined to perform semantic segmentation and annotation on the model, further improving its usability and practicality. For example, automatic identification and annotation of pipeline valves can provide accurate equipment information for subsequent risk assessments.

[0056] Compared to traditional risk identification methods based on manual inspection and experience-based judgment, step two of this invention utilizes advanced AIGC, 3D vision, and image recognition technologies to automate and intelligently identify risks, greatly improving identification efficiency and accuracy while reducing errors and omissions caused by human factors.

[0057] Step 3: Acquire equipment signage image data in the factory, and use deep learning algorithms to process the equipment signage image data to form a local three-dimensional spatial structure.

[0058] Step 3.1: Acquire equipment label image data in the factory using imaging equipment, focusing on key areas. This includes using handheld or tripod cameras to capture images of equipment labels in complex environments such as pipes, valves, high-radiation areas, and enclosed spaces. Ensure image clarity and integrity during imaging for subsequent image recognition processing. For example, choose appropriate lighting conditions and shooting angles when photographing equipment labels, avoiding factors that affect image quality such as reflections and shadows. Feature matching of multi-view images (such as ORB or LoFTR features) can also enhance the consistency of the image data point cloud to address occlusion issues. Data acquisition can be completed using handheld or tripod 3D scanners and cameras, eliminating the need for installing numerous sensors within the factory, reducing implementation costs and complexity, and offering wider applicability.

[0059] Step 3.2: Preprocess the equipment sign image data, specifically including: preprocessing the captured equipment sign images, including image enhancement, noise reduction, edge detection, etc. Image enhancement aims to improve the brightness and contrast of the image, making it clearer; noise reduction aims to remove noise points in the image, improving image quality; edge detection aims to highlight edge information in the image, facilitating subsequent character recognition.

[0060] Step 3.3: Convert the extracted character information in the image into text information. Specifically, this includes: using optical character recognition (OCR) algorithms to recognize and extract characters in the image, and converting the character information in the image into text data. For example, using OCR algorithms to accurately recognize and extract characters such as equipment numbers and models from signage images.

[0061] Step 3.4: Utilize deep learning algorithms to perform local spatial depth inference based on the equipment signage image data, obtaining local 3D point cloud data of the equipment signage and its surrounding environment, forming a local three-dimensional spatial structure, specifically including:

[0062] Construct a deep learning model (such as MiDaS or Transformer architecture) for predicting pixel-level depth maps from a single RGB image. The model outputs the depth value Z(u,v) for each pixel, as shown in formula (1):

[0063] Z = f θ (I) (1)

[0064] Where (u,v) are the x and y coordinates of the pixel, Z(u,v) is the depth value representing pixel (u,v), and f θ For the depth estimation network, I represents the input RGB image. The deep learning model is trained using the depth map from the factory environment data as the dataset. (Is this correct?) The depth estimation accuracy is enhanced by geometric consistency loss, and the loss function of geometric consistency loss is as shown in formula (2):

[0065] L vn =∑ p Pn(p)-n gt (p)P2 (2)

[0066] Where Lvn is the geometric consistency loss, n(p) is the normal vector of the predicted point p, and n gt (p) is the true normal vector. Then, the equipment sign image data is input into the deep learning model to obtain the equipment sign image data depth map. Based on the camera intrinsic parameters, the equipment sign image data depth map is converted into a 3D point cloud to form a local three-dimensional spatial structure. The conversion formula is as shown in formula (3):

[0067]

[0068] Where X, Y, Z are the 3D point cloud coordinates corresponding to pixel coordinates (u, v), (c x ,c y f is the optical center of the camera. x and f y This refers to the camera's focal length.

[0069] Step 4: Match and register the local 3D spatial structure with the initial factory 3D model to achieve precise alignment of local and overall point cloud data, resulting in a factory 3D spatial model containing equipment information. Specifically, this includes: matching and anchoring the actual position information of the equipment based on the identified equipment text data and the local 3D point cloud data generated using 3D Gaussian splashing technology. By comparing and calibrating the actual position of the equipment with its position in the factory 3D spatial model, the precise position coordinates, orientation, and size information of the equipment in the 3D spatial model are determined. The equipment's position information is then associated and stored with data such as operating parameters and maintenance records to achieve full lifecycle management of the equipment. For example, for a piece of equipment located in a high-radiation area, image recognition and 3D point cloud matching can accurately determine its position in the 3D spatial model, and associate it with data such as operating parameters and maintenance records for convenient subsequent equipment management and maintenance. In the above process, key feature points are extracted from the local and global point cloud data (i.e., the local 3D spatial structure and the initial factory 3D model). Based on the correspondence of feature points in different point cloud data, the alignment and matching operations of the local and global point cloud data are completed. During the alignment and matching process, the iterative nearest neighbor algorithm is used to optimize the registration accuracy of the point cloud data. The above process includes two steps: initial matching and fine matching. In the initial matching, some simple geometric constraints (such as nearest neighbor matching, preliminary projection of feature points, etc.) can be used to obtain a rough alignment result. The fine matching process maximizes the overlap between the two sets of point cloud data by continuously adjusting the transformation parameters (including rotation, translation, etc.).

[0070] This invention uses the Iterative Closest Point (ICP) algorithm. Specific steps include: Initial matching: Based on feature points or the initial coarse registration results, select one point in each point cloud to match the nearest point in another point cloud.

[0071] Calculate the rigid transformation: For each pair of matched points, calculate a rigid transformation (including rotation matrix and translation vector).

[0072] Apply transformation: Apply a rigid transformation to all points in the point cloud data so that the matching points overlap as much as possible in the two sets of point clouds.

[0073] Calculation error: Calculate the error after matching all point cloud data pairs after rigid transformation. The distance error between two sets of point clouds is usually calculated using the least squares method.

[0074] Iteration: Adjust the rigid transformation parameters based on the error results, and repeat steps 2 to 4 until the error converges to a preset threshold.

[0075] Step 5: Obtain the operational status data of the workers. Based on the operational status data and the 3D spatial model of the factory, identify and issue warnings for operational risks. Operational risks include: fall from height risk, high radiation risk, path error risk, equipment collision risk, and electromagnetic interference risk.

[0076] To address the risk of falls from heights, the system monitors real-time sensor data from workers' personal protective equipment (PPE) while they are working at heights. Based on a 3D spatial model and sensor data from the PPE, it determines whether workers are at risk of falling from heights. For example, by monitoring the tension of safety belts and the wearing of safety helmets, the system automatically triggers a fall risk warning when workers are near an edge or not wearing protective equipment correctly. Warning information is promptly sent to workers and site managers via audible and visual alarms, SMS notifications, and mobile app push notifications, reminding them to take appropriate safety measures to prevent accidents.

[0077] For high radiation risks, in high radiation risk identification scenarios, the radiation source location and intensity information in the 3D spatial model, combined with the worker's location data, is used to calculate the radiation dose at the worker's location in real time. Intensity data of the radiation source is acquired through radiation monitoring sensors, and combined with the geometric relationships in the 3D spatial model, the radiation dose to the worker at different locations is calculated. When the radiation dose exceeds the safety threshold, the system automatically issues a high radiation risk warning and guides the worker to evacuate the danger zone. Warning information can be sent to workers and on-site management personnel via audible and visual alarms, SMS notifications, and mobile app push notifications, reminding them to take protective measures or evacuate the danger zone. The 3D spatial model diagram shows when the equipment radiation dose exceeds the safety threshold; the red areas in the diagram represent high radiation areas.

[0080] To address path error risk, the system monitors worker movement in real-time using equipment layout and work area divisions within a 3D spatial model during the path error risk identification process. Real-time worker location data is acquired using a personnel positioning system and combined with area division information from the 3D spatial model to determine if workers have mistakenly entered undesignated areas or gone to the wrong interval. When a worker mistakenly enters an undesignated area or goes to the wrong interval, the system automatically issues a warning and guides the worker back to the correct path. Warning information can be sent to workers via on-site broadcasts, mobile app push notifications, etc., reminding them to adjust their path and avoid entering dangerous areas.

[0081] To address the risk of accidental equipment contact, the system utilizes equipment location data from a 3D spatial model and operator location data to monitor the distance between operators and equipment in real time during risk identification. The system calculates the real-time distance using personnel positioning data and equipment location data. When an operator approaches the safe distance from the equipment, the system automatically issues a risk warning and reminds the operator to be cautious. Warning information can be sent to operators and on-site managers via audible and visual alarms, SMS notifications, and mobile app push notifications, reminding them to maintain a safe distance and avoid accidental equipment contact.

[0082] To address electromagnetic interference (EMI) risks, the system calculates the EMI intensity at the worker's location in real time by combining the location and intensity information of electromagnetic equipment in a 3D spatial model with the worker's location data. Electromagnetic monitoring sensors acquire the intensity data of electromagnetic equipment, and, combined with the geometric relationships in the 3D spatial model, calculate the EMI intensity at different worker locations. When the EMI intensity exceeds a safety threshold, the system automatically issues an EMI risk warning and reminds the worker to take protective measures. Warning information can be sent to workers and site managers via audible and visual alarms, SMS notifications, and mobile app push notifications, reminding them to take protective measures to avoid EMI harming their health.

[0083] By monitoring the position and posture of workers in real time, and combining the equipment layout and risk source information in the 3D spatial model, timely risk warnings are issued, which improves the safety awareness and response capabilities of workers, effectively reduces the possibility of accidents and the losses caused by accidents, and improves the safety and production continuity of the factory.

[0084] This embodiment provides a method for intelligent 3D spatial reconstruction and operational risk identification of complex factory environments based on AIGC. Utilizing 3D Gaussian splashing technology and overall 3D point cloud data of the factory, it achieves intelligent reconstruction of complex factory environments. Through deep reasoning and generation of local spaces, the spatial model achieves higher accuracy and detail, accurately matching the precise locations of equipment into the 3D space, thus providing crucial data support for equipment management and risk assessment. By real-time monitoring of the position and posture of workers, as well as risk sources in the factory environment (such as falls from heights, radiation, electromagnetic interference, etc.), combined with the equipment layout in the 3D spatial model, various risk warnings can be issued promptly, ensuring the safety of workers.

[0085] like Figure 2 As shown, this embodiment of the invention also provides a 3D spatial intelligent reconstruction and risk identification system for factory environments, including:

[0086] The environmental data acquisition module is used to acquire factory environmental data and preprocess the factory environmental data;

[0087] The factory model generation module is used to generate overall 3D point cloud data of the factory environment based on the preprocessed factory environment data, and to convert the 3D point cloud data into a Gaussian distribution using 3D Gaussian splashing technology to generate an initial 3D factory model.

[0088] The device embedding module 403 is used to acquire equipment nameplate image data in the factory, and use deep learning algorithms to perform local spatial depth inference based on the equipment nameplate image data to obtain local 3D point cloud data of the equipment nameplate and its surrounding environment, forming a local three-dimensional spatial structure.

[0089] The registration optimization module 404 is used to match and register the local three-dimensional spatial structure with the initial factory 3D model, so as to achieve accurate alignment between local and overall point cloud data and obtain a factory 3D spatial model containing equipment information.

[0090] The risk warning module 405 is used to acquire the work status data of the operators and identify and warn of work risks based on the work status data and the 3D spatial model of the factory.

[0091] This embodiment provides a method for intelligent 3D spatial reconstruction and operational risk identification of complex factory environments based on AIGC. Utilizing overall 3D point cloud data of the factory and 3D Gaussian splashing technology, it achieves intelligent reconstruction of the complex factory environment. Through image recognition technology combined with equipment label information, it accurately identifies the spatial location and attributes of identified equipment. By real-time monitoring of the position and posture of workers and risk sources in the factory environment (such as falls from heights, radiation, electromagnetic interference, etc.), and combining this with the equipment layout in the 3D spatial model, it issues timely risk warnings to ensure worker safety, thereby significantly improving the safety of the factory operating environment and reducing the probability of potential accidents. This method not only achieves high-precision spatial modeling and equipment management but also effectively improves the safety of the operating environment, possessing broad application prospects, especially suitable for complex industrial scenarios such as nuclear power plants with large scale and hazardous operating environments requiring precise management and risk monitoring.

[0092] The above embodiments merely illustrate several implementation methods of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this patent should be determined by the appended claims.

Claims

1. A method for intelligent 3D spatial reconstruction and risk identification of a factory environment, characterized in that, The method includes: Step 1: Acquire factory environmental data and preprocess the factory environmental data; Step 2: Generate an initial 3D model of the factory based on the preprocessed factory environment data; Step 2.1: Generate overall 3D point cloud data of the factory environment based on the preprocessed factory environment data; Step 2.2: Use 3D Gaussian splashing technology to convert 3D point cloud data into a Gaussian distribution to generate an initial 3D factory model; Step 2.3: Inspect and correct the initial 3D factory model; Step 3: Acquire equipment signage image data in the factory, process the equipment signage image data using deep learning algorithms, and form a local three-dimensional spatial structure; Step 3.1: Use imaging equipment to acquire image data of equipment nameplates in the factory, focusing on acquiring image data of equipment nameplates in key areas; Step 3.2: Preprocess the equipment nameplate image data Step 3.3: Convert the extracted character information from the image into text information; Step 3.4: Use deep learning algorithms to perform local spatial depth reasoning based on the equipment sign image data to obtain local 3D point cloud data of the equipment sign and its surrounding environment, forming a local three-dimensional spatial structure; Step 4: Match and register the local 3D spatial structure with the initial factory 3D model to obtain a factory 3D spatial model containing equipment information; Step 5: Obtain the work status data of the operators, and identify and warn of work risks based on the work status data and the 3D spatial model of the factory.

2. The method for intelligent 3D spatial reconstruction and risk identification of a factory environment according to claim 1, characterized in that, In step one, the factory environment data is collected using multi-view cameras and laser scanners, with a focus on complex environments such as pipelines and valves, high-radiation areas, and enclosed spaces. The preprocessing of the factory environment data specifically includes cleaning, filtering, and completing the data.

3. The method for intelligent 3D spatial reconstruction and risk identification of a factory environment according to claim 2, characterized in that, Step 2.1, which generates overall 3D point cloud data of the factory environment based on the preprocessed factory environment data, specifically includes: for multi-view cameras, using the matching points in the image data of the multi-view cameras and the pose information of the cameras, calculating the position of each pixel in three-dimensional space based on the image data of the multi-view cameras, generating the three-dimensional coordinates corresponding to each pixel, thereby obtaining the overall 3D point cloud data of the factory; for laser scanners, emitting a laser beam and receiving the returned signal by the laser scanner, using the time difference or phase difference of the signal to measure the distance to the object surface, obtaining the three-dimensional coordinates of the object, and obtaining the three-dimensional coordinates of each pixel in the laser scanner image.

4. The method for intelligent 3D spatial reconstruction and risk identification of a factory environment according to claim 3, characterized in that, In step 2.1, in generating the overall 3D point cloud data of the factory environment based on the preprocessed factory environment data, the data from multi-view cameras and laser scanners are combined. The precise geometric shape of the factory environment is obtained using the laser scanning data, and the details of the 3D point cloud data are enhanced using the visual information from the multi-view cameras. The 3D point cloud data is then converted into a three-dimensional mesh model and textures are added.

5. The method for intelligent 3D spatial reconstruction and risk identification of a factory environment according to claim 4, characterized in that, In step 2.2, the 3D point cloud data is converted into a Gaussian distribution using 3D Gaussian splashing technology to generate an initial factory 3D model. Specifically, this includes: converting each point data of the 3D point cloud data into a 3D Gaussian function using Gaussian splashing technology, defining the position μ, covariance matrix Σ, color RGB, and opacity α of each point data; when converting the Gaussian function using Gaussian splashing technology, a Gaussian function density control strategy is implemented for preset key areas to increase the number of Gaussian functions in key areas.

6. The method for intelligent 3D spatial reconstruction and risk identification of a factory environment according to claim 5, characterized in that, The key areas are densely populated areas of pipes and valves, high-radiation areas, and entrances to enclosed spaces. Step two, generating the initial 3D factory model, specifically includes: when converting 3D point cloud data into a Gaussian distribution, for 3D point cloud data from different perspectives generated by multi-view cameras and laser scanners, a multi-view fusion algorithm is used to register and fuse the Gaussian functions from different perspectives, eliminating differences and redundant information between perspectives to form a consistent initial 3D factory model; after generating the initial 3D factory model, a consistency check is performed on the initial 3D factory model to ensure that the fused model has consistency in geometry, color, and texture, and the initial 3D factory model is refined and edited using 3D Gaussian splashing technology, by adjusting the parameters of the Gaussian function to enhance or smooth the surface details of the model; and by adding or deleting Gaussian functions, the local structure of the model is modified.

7. The method for intelligent 3D spatial reconstruction and risk identification of a factory environment according to claim 1, characterized in that, In step 3.1, handheld or stand-up cameras are used to photograph equipment signs to obtain image data of equipment signs in the factory, and feature matching of multi-view images is used to enhance the consistency of point cloud of image data. Step 3.2 preprocesses the equipment sign image data, including image enhancement, noise reduction, and edge detection. In step 3.3, the OCR algorithm is used to identify and extract the device number and model from the sign image; Step 3.4, which involves forming a local three-dimensional spatial structure, specifically includes: A deep learning model is constructed to predict pixel-level depth maps from a single image. The model outputs the depth value Z(u,v) for each pixel, as shown in formula (1): Z=f θ (I) (1) Where (u,v) are the x and y coordinates of the pixel, Z(u,v) is the depth value representing pixel (u,v), and f θ For the depth estimation network, I represents the input RGB image. The deep learning model is trained using the depth map from the factory environment data as the dataset. The depth estimation accuracy is enhanced by geometric consistency loss, the loss function of which is shown in formula (2): L vn =∑ p Pn(p)-n gt (p)P2 (2) Among them, L vn The loss is the geometric consistency loss, where n(p) is the normal vector of the predicted point p. gt (p) is the true normal vector. Then, the equipment sign image data is input into the deep learning model to obtain the equipment sign image data depth map. Based on the camera intrinsic parameters, the equipment sign image data depth map is converted into a 3D point cloud to form a local three-dimensional spatial structure. The conversion formula is as shown in formula (3): Where X, Y, Z are the 3D point cloud coordinates corresponding to pixel coordinates (u, v), (c x ,c y f is the optical center of the camera. x and f y This refers to the camera's focal length.

8. The method for intelligent 3D spatial reconstruction and risk identification of a factory environment according to claim 1, characterized in that, Step four involves matching and registering the local 3D spatial structure with the initial factory 3D model to achieve precise alignment of the local and overall point cloud data, resulting in a factory 3D spatial model containing equipment information. Specifically, this includes: using an iterative nearest-point algorithm to perform initial and fine-grained matching of the point cloud data of the local 3D spatial structure and the initial factory 3D model to determine the equipment location information of the local 3D spatial structure; associating and storing the equipment location information with equipment operating parameters, maintenance records, and other data to obtain the factory 3D spatial model. The iterative nearest-point algorithm specifically includes: matching each point cloud data of the local 3D spatial structure with the nearest point in the point cloud data of the initial factory 3D model based on the initial matching results; for each pair of matched points, calculating the rigid transformation adjustment and recalculating the distance error; then adjusting the rigid transformation parameters based on the distance error result, iterating until the distance error converges to a threshold.

9. The method for intelligent 3D spatial reconstruction and risk identification of a factory environment according to claim 1, characterized in that, The operational risks in step five include: fall from height risk, high radiation risk, path error risk, equipment contact risk, and electromagnetic interference risk. For fall from height risk, real-time monitoring of sensor data from personal protective equipment (PPE) is conducted when workers are working at heights. Based on the 3D spatial model and the sensor data from the PPE, it is determined whether workers are at risk of falling from heights. For high radiation risk, the location and intensity information of radiation sources in the 3D spatial model, combined with the worker's location data, is used to calculate the radiation dose at the worker's location in real time. Intensity data of radiation sources is obtained through radiation monitoring sensors, and combined with the geometric relationships in the 3D spatial model, the radiation dose at different locations is calculated. When the radiation dose exceeds the safety threshold, a high radiation risk warning is issued, and workers are guided to evacuate the danger zone. For path error risk, the movement path of workers is monitored in real time through the equipment layout and work area division in the 3D spatial model, and the personnel positioning system is used to obtain the worker's location information. The system uses real-time location data, combined with area division information in the 3D spatial model, to determine whether workers have mistakenly entered undesignated areas or gone to the wrong intervals. For the risk of accidental equipment contact, it uses equipment location data and worker location data in the 3D spatial model to monitor the distance between workers and equipment in real time. Through the personnel positioning system and equipment location data, it calculates the real-time distance between workers and equipment. When a worker approaches the safe distance of the equipment, it issues a risk warning of accidental equipment contact and reminds the worker to pay attention to safety. For the risk of electromagnetic interference, it uses the location and intensity information of electromagnetic equipment in the 3D spatial model, combined with worker location data, to calculate the electromagnetic interference intensity at the worker's location in real time. It uses electromagnetic monitoring sensors to obtain the intensity data of electromagnetic equipment and combines it with the geometric relationships in the 3D spatial model to calculate the electromagnetic interference intensity at different worker locations. When the electromagnetic interference intensity exceeds the safety threshold, it issues an electromagnetic interference risk warning and reminds the worker to take protective measures.

10. A 3D spatial intelligent reconstruction and risk identification system for a factory environment, wherein the system employs the 3D spatial intelligent reconstruction and risk identification method for a factory environment as described in any one of claims 1-9, characterized in that, The system includes an environmental data acquisition module, a factory model generation module, an equipment embedding module, a registration optimization module, and a risk warning module. The environmental data acquisition module is used to acquire factory environmental data and preprocess the factory environmental data. The factory model generation module is used to generate overall 3D point cloud data of the factory environment based on the preprocessed factory environment data, and to convert the 3D point cloud data into a Gaussian distribution using 3D Gaussian splashing technology to generate an initial 3D factory model. The device embedding module is used to acquire equipment sign image data in the factory, and uses deep learning algorithms to perform local spatial depth reasoning based on the equipment sign image data to obtain local 3D point cloud data of the equipment sign and its surrounding environment, forming a local three-dimensional spatial structure. The registration optimization module is used to match and register the local three-dimensional spatial structure with the initial factory 3D model, so as to achieve accurate alignment of local and overall point cloud data and obtain a factory 3D spatial model containing equipment information. The risk warning module is used to acquire the work status data of the operators and, based on the work status data and the 3D spatial model of the factory, identify and warn of work risks.

Images