Deep dataset construction methods, security monitoring methods and related equipment

Abstract

This invention discloses a method for constructing a deep dataset, a safety monitoring method, and related equipment. The method for constructing a deep dataset includes: acquiring scene configuration parameters corresponding to an industrial safety scenario; constructing a parameterized simulation scenario based on the scene configuration parameters; responding to a preset safety event script, controlling the interaction between human and equipment models in the simulation scenario to generate a corresponding safety event scenario sequence; performing depth rendering on the safety event scenario sequence based on a sensor physical noise model to obtain depth data; and generating labeled data corresponding to the depth data based on scene geometric information and object attribute information to construct a deep dataset corresponding to the industrial safety scenario. By constructing an industrial safety scenario deep dataset through depth rendering using a sensor physical noise model, the depth data can simulate real sensor measurement errors, thereby improving the training effect and monitoring accuracy of the industrial safety monitoring model.

Description

Technical Field

[0001] This invention relates to the field of industrial safety, and in particular to a method for constructing a deep dataset, a safety monitoring method, and related equipment. Background Technology

[0002] In the field of industrial safety monitoring, to effectively prevent personnel from accidentally entering dangerous areas or continuing to operate equipment before it has been completely shut down, it is usually necessary to monitor the industrial environment, equipment location, and personnel status in real time using various sensors. For example, laser rangefinders, stereo safety sensors, Time-of-Flight (TOF) depth cameras, or 3D structured light cameras can be used to perceive the spatial location of personnel, equipment, and objects in industrial scenes and generate corresponding depth information to assist in safety control or early warning. Existing methods mostly rely on real data collected by a single sensor. However, in complex industrial scenarios, due to environmental obstructions, diverse personnel movements, and complex equipment layouts, a single sensor cannot cover all safety-critical areas, easily leading to monitoring blind spots or incomplete data.

[0003] To compensate for insufficient monitoring data, some existing technologies have begun to employ simulation or digital scene construction methods. These methods use virtual human and equipment models to simulate various safety incidents in industrial settings, such as personnel entering dangerous areas, falling objects, and equipment collisions, generating corresponding depth maps. However, the depth information generated by these simulations differs from the real environment, making it difficult to directly use it to train safety monitoring models.

[0004] Therefore, how to efficiently generate deep data and labeled data that match the characteristics of real industrial environments in industrial safety scenarios, so as to build high-quality industrial safety scenario deep datasets and improve the monitoring accuracy of safety monitoring, has become a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0005] Therefore, it is necessary to provide methods for constructing deep datasets, safety monitoring methods, and related equipment to address the aforementioned technical issues. These methods and equipment have the advantage of efficiently generating deep data and labeled data that match the characteristics of real industrial environments in industrial safety scenarios, thereby constructing high-quality deep datasets for industrial safety scenarios and improving the accuracy of safety monitoring.

[0006] A method for constructing a deep dataset, the method comprising: Obtain the scenario configuration parameters corresponding to the industrial safety scenario; Based on the scenario configuration parameters, a parameterized simulation scenario is constructed; In response to a preset safety event script, the human body model and device model in the simulation scene are controlled to interact to generate a corresponding safety event scene sequence, which includes scene geometric information and object attribute information. Based on the sensor physical noise model, the security event scene sequence is rendered with depth data to obtain depth data. The sensor physical noise model is a noise model that characterizes the measurement error of the sensor in the depth imaging process. Based on the scene geometry information and the object attribute information, labeled data corresponding to the depth data is generated to construct a depth dataset corresponding to the industrial safety scenario.

[0007] Optionally, the safety event scenario sequence includes at least one of the following: a hazardous area scenario sequence, an object falling scenario sequence, and a device collision scenario sequence. The step of responding to a preset safety event script and controlling the human and device models in the simulation scenario to interact to generate the corresponding safety event scenario sequence includes: In response to the safety event script characterizing a person entering a dangerous area, the human body model is controlled to approach the device model along a movement trajectory that deviates from the safe path, so as to generate a corresponding sequence of scenes of a person entering a dangerous area. In response to the safety event script representing an object falling event, the target object is controlled to move relative to the support body under external force disturbance, so as to generate a corresponding object falling scene sequence; In response to the safety event script characterizing a device collision event, the human model and / or device model are controlled to approach or come into contact according to preset motion parameters to generate a corresponding device collision scene sequence.

[0008] Optionally, after responding to a preset security event script and controlling the human body model and device model in the simulation scene to interact to generate a corresponding security event scene sequence, the method further includes: The interaction results between the human body model and the device model are subjected to physical constraint verification to obtain the verification results; If the verification results indicate that there are abnormalities such as clipping, collision, or abnormal motion trajectory, the motion parameters of the human body model and / or device model are adjusted, and the safety event scenario sequence is regenerated.

[0009] Optionally, the step of performing depth rendering on the security event scene sequence based on the sensor physical noise model to obtain depth data includes: Based on the scene geometry information and the object attribute information, determine the ideal depth value corresponding to each pixel in the security event scene sequence; Based on the sensor physical noise model and the object attribute information, the initial noise intensity corresponding to each pixel is determined; Based on the sensor physical noise model and the scene geometric information, the initial noise distribution corresponding to each pixel is determined; The initial noise intensity and initial noise distribution are perturbed to obtain the perturbed noise parameters; Based on the noise parameters after the disturbance, a measurement error is introduced into the ideal depth value to obtain the depth data.

[0010] Optionally, the initial noise distribution includes a mixed pixel noise distribution and a multipath interference noise distribution. Determining the initial noise distribution for each pixel based on the sensor physical noise model and the scene geometric information includes: When the scene geometry information characterizes the target pixel as being in a depth discontinuous region, the mixed pixel noise distribution corresponding to the target pixel is determined based on the area ratio of the foreground region to the background region and / or the echo intensity ratio. When characterizing the target pixel corresponding to multiple reflection paths using the scene geometry information, the multipath interference noise distribution corresponding to the target pixel is determined based on the contribution relationship between the direct path and the indirect path.

[0011] Optionally, the annotation data includes event-type annotation data and visual-type annotation data. The event-type annotation data includes security event labels and corresponding event level labels. Generating the annotation data corresponding to the depth data based on the scene geometry information and the object attribute information includes: Based on the scene geometry information and the object attribute information, the visual class annotation data is generated, which includes semantic segmentation labels, instance segmentation masks, target bounding boxes, and / or depth ground truth. Based on the event type corresponding to the security event script, generate security event tags corresponding to the depth data; Based on the event parameters corresponding to the security event script, an event level label corresponding to the security event label is generated.

[0012] Optionally, generating labeled data corresponding to the depth data based on the scene geometry information and the object attribute information to construct a depth dataset corresponding to the industrial safety scenario includes: Collect real-world depth data from industrial sites; The real depth data is labeled based on the pre-labeling model to obtain labeled data of the real depth data. The pre-labeling model is trained based on the depth data. Based on the depth data and the corresponding labeled data, a first depth dataset is constructed; Based on the actual depth data and the corresponding labeled data, a second depth dataset is constructed; Based on the first depth dataset and the second depth dataset, feature alignment processing is performed to obtain the target first depth dataset and the target second depth dataset; The target first depth dataset and the target second depth dataset are fused according to a preset ratio to obtain the depth dataset corresponding to the industrial safety scenario.

[0013] A safety monitoring method, the method comprising: Obtain the safety monitoring model to be trained and the deep dataset corresponding to the industrial safety scenario; The security monitoring model to be trained is trained based on the deep dataset to obtain the target security monitoring model. Based on the target security monitoring model, security monitoring is performed on the target area to obtain security monitoring results.

[0014] A deep dataset construction apparatus, the apparatus comprising: The first acquisition module is used to acquire the scenario configuration parameters corresponding to the industrial safety scenario; The construction module is used to construct a parameterized simulation scenario based on the scenario configuration parameters; The interaction module is used to respond to a preset safety event script and control the human body model and device model in the simulation scene to interact in order to generate a corresponding safety event scene sequence, the safety event scene sequence including scene geometric information and object attribute information; The rendering module is used to perform depth rendering on the security event scene sequence based on the sensor physical noise model to obtain depth data. The sensor physical noise model is a noise model that characterizes the measurement error of the sensor in the depth imaging process. The generation module is used to generate labeled data corresponding to the depth data based on the scene geometry information and the object attribute information, so as to construct a depth dataset corresponding to the industrial safety scene.

[0015] A safety monitoring device, the device comprising: The second acquisition module is used to acquire the safety monitoring model to be trained and the deep dataset corresponding to the industrial safety scenario. The training module is used to train the security monitoring model to be trained based on the deep dataset to obtain the target security monitoring model. The monitoring module is used to perform security monitoring processing on the target area based on the target security monitoring model to obtain security monitoring results.

[0016] To achieve the purpose of the invention, this solution also provides a computer device, including a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, wherein the processor implements the above-described deep dataset construction method or security monitoring method when executing the computer-readable instructions.

[0017] To achieve the purpose of the invention, this solution also provides a readable storage medium storing computer-readable instructions, which, when executed by a processor, implement the above-mentioned deep dataset construction method or security monitoring method.

[0018] The aforementioned deep dataset construction method, safety monitoring method, and related equipment acquire scene configuration parameters corresponding to industrial safety scenarios; based on the scene configuration parameters, a parameterized simulation scenario is constructed; responding to a preset safety event script, the human body model and equipment model in the simulation scenario are controlled to interact to generate a corresponding safety event scenario sequence, the safety event scenario sequence including scene geometric information and object attribute information; based on a sensor physical noise model, the safety event scenario sequence is subjected to depth rendering to obtain depth data, the sensor physical noise model being a noise model characterizing the measurement error of the sensor during the depth imaging process; based on the scene geometric information and the object attribute information, labeled data corresponding to the depth data is generated to construct a deep dataset corresponding to the industrial safety scenario. By constructing a deep dataset for industrial safety scenarios based on a parameterized simulation scenario, generating an interaction sequence of human body models and equipment models in response to a safety event script, performing depth rendering using a sensor physical noise model, and simultaneously generating labeled data based on scene geometric information and object attributes, the deep dataset can simulate real sensor measurement errors. This provides high-quality, fully labeled samples for training safety monitoring models, significantly improving the training effect, monitoring accuracy, and safety and reliability of industrial equipment operation. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating a method for constructing a deep dataset according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating the second method for constructing a deep dataset in one embodiment of the present invention; Figure 3 This is a flowchart illustrating the third method for constructing a deep dataset in one embodiment of the present invention; Figure 4 This is a flowchart illustrating the fourth deep dataset construction method in one embodiment of the present invention; Figure 5This is a flowchart illustrating a security monitoring method according to one embodiment of the present invention; Figure 6 This is a schematic diagram of a deep dataset construction device according to an embodiment of the present invention; Figure 7 This is a schematic diagram of a safety monitoring device according to one embodiment of the present invention; Figure 8 This is a schematic diagram of a computer device according to an embodiment of the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] In one embodiment, such as Figure 1 As shown, a method for constructing a deep dataset is provided, including the following steps: 101. Obtain the scenario configuration parameters corresponding to the industrial safety scenario.

[0023] In this embodiment of the invention, the industrial safety scenario may include typical industrial environments such as workshops, warehouses, production lines, loading and unloading areas, robotic arm operation areas, and AGV transport channels. To generate depth data corresponding to these scenarios, configuration parameters for each scenario need to be obtained. These parameters are used to accurately describe the layout and attributes of various elements within the scenario. For example, scenario configuration parameters may include the specific locations of equipment and workstations, equipment height and dimensions, material type, reflectivity, ground roughness, lighting conditions, and the distribution of potential obstructions. Through parametric modeling, a configurable industrial scenario template library can be constructed. Each template can support the randomization of equipment layout, material properties, lighting conditions, and other environmental factors, thereby covering diverse situations in different industrial environments. This not only allows for the rapid generation of a large amount of simulation data for different scenarios but also ensures that the generated scenarios have diversity in spatial layout and environmental characteristics, meeting the needs of subsequent depth data rendering and safety event simulation.

[0024] As one possible implementation, adjustable dynamic elements, such as movable shelves, variable-height robotic arms, or temporary obstructions, can be added to the template library to further enrich the complexity of the simulation scene, improve the coverage of the dataset, and enhance the generalization ability of the training model.

[0025] 102. Construct a parameterized simulation scenario based on scenario configuration parameters.

[0026] In this embodiment of the invention, after obtaining the configuration parameters of the industrial safety scenario, a parameterized simulation scenario can be constructed based on these parameters. In the simulation scenario, the spatial layout, equipment placement, material properties, lighting conditions, and distribution of obstructions are all accurately simulated according to the configuration parameters.

[0027] The parameterized industrial scene template library pre-sets the basic structures of typical industrial environments such as workshops, warehouses, assembly lines, loading and unloading areas, robotic arms, and AGV transport channels. Each template contains a complete parameter table, which can be configured with factory area, aisle width, equipment spacing, robotic arm model, conveyor belt length, shelf height, etc., and can randomly select different combinations to generate diverse scenes. The reflectivity of floor and wall materials, light intensity, dust, water mist, temperature distribution, etc. can all be randomly configured to simulate actual industrial safety scenarios under different environmental conditions.

[0028] During the generation process, various layout combinations can be set according to the characteristics of different industrial safety scenarios, such as warehouse shelf arrangement, production line equipment spacing, AGV channel path planning, etc., to ensure that the simulation scenario covers various typical industrial operating environments. Each set of parameter configurations corresponds to a complete 3D scene model, including geometric models, material maps, lighting settings, and all the information required for rendering, eliminating the need for manual modeling.

[0029] As one possible implementation, the simulation scenario can incorporate time-dynamic simulation, such as simulating the periodic movement of a robotic arm, the movement of an AGV vehicle, or changes in lighting over time. This allows the deep data generation to reflect the real characteristics of the dynamic industrial environment, improving the diversity of the generated data and the applicability of the trained model. Furthermore, the template library can be integrated with a script engine to drive human and equipment models to interact with safety events, providing controllable and diverse inputs for subsequent generation of safety event scenario sequences, further enhancing the richness and representativeness of the dataset in the safety event dimension.

[0030] 103. Responding to a preset safety event script, control the human body model and device model in the simulation scene to interact in order to generate a corresponding safety event scene sequence.

[0031] In this embodiment of the invention, after the simulation scene is constructed, a preset safety event script can be invoked to control the interaction between the human body model and the device model in the scene, thereby generating a corresponding safety event scene sequence. The safety event scene sequence includes scene geometric information and object attribute information. The scene geometric information reflects the position, posture, and motion trajectory of the human body model, device model, and other objects in three-dimensional space, while the object attribute information includes the size, shape, material, reflection characteristics, and possible motion parameters of each object, such as velocity, acceleration, and force state.

[0032] Safety event scripts are automatically generated by the script engine and can include sequences of behaviors such as normal work actions, personnel entering hazardous areas, falling objects, equipment collisions, or other abnormal events. During generation, the physics engine verifies the rationality of the actions of the human and equipment models, ensuring that the actions do not clip through, float, or violate the laws of physics. For example, when a person deviates from the safe path and enters the robotic arm's work area, the physics engine ensures that their movement trajectory maintains a reasonable safe distance from the robotic arm's position, while recording data on the proximity or potential collision points.

[0033] As one possible implementation, the script engine can randomly combine human and equipment motion parameters according to different industrial safety event types, such as adjusting personnel walking speed, deviation angle, object falling direction, or equipment movement amplitude, thereby generating rich event variations to enhance the diversity of deep datasets in the safety event dimension.

[0034] 104. Based on the sensor physical noise model, perform depth rendering on the sequence of security event scenarios to obtain depth data.

[0035] In this embodiment of the invention, after generating a sequence of safety event scenarios, depth rendering can be performed on the sequence based on a sensor physical noise model to obtain depth data that approximates a real industrial environment. The sensor physical noise model characterizes various measurement errors that may occur during depth imaging, including multipath interference, shot noise, ambient light noise, motion blur, and flying point effects. These errors affect the depth value of each pixel. By introducing a noise model during rendering, the generated depth image can more closely resemble the data collected by the sensor in a real industrial setting.

[0036] To further enhance the diversity and realism of the data, physical noise models can incorporate domain randomization strategies to randomly perturb sensor parameters, noise intensity, lighting conditions, obstruction distribution, and other environmental factors, thereby generating multiple depth data variants covering various industrial conditions. For example, injecting multipath interference at metal reflective surfaces or depth edges, simulating motion blur at fast-moving human or equipment models, and adding environmental noise under strong or complex lighting conditions can generate depth maps that simultaneously reflect measurement deviations under various real-world industrial safety scenarios.

[0037] As one possible implementation, the noise model can also be adjusted according to the imaging characteristics of different types of sensors (such as structured light cameras, TOF cameras, stereo security sensors, or stereo vision cameras) to adapt to the depth acquisition needs of various sensors in industrial environments.

[0038] 105. Based on scene geometry information and object attribute information, generate labeled data corresponding to depth data to construct a depth dataset corresponding to industrial safety scenarios.

[0039] In this embodiment of the invention, after generating depth data and its corresponding labeled data, the data can be organized into a depth dataset corresponding to an industrial safety scenario based on scene geometry information and object attribute information. This depth dataset contains depth images of various types of safety events and corresponding visual and event labeling information, which can be directly used to train a safety monitoring model and improve the model's ability to identify personnel, equipment, and abnormal events.

[0040] Furthermore, to enhance the realism and generalization ability of the dataset, real depth data and its annotations can be collected from industrial sites, and the simulated depth data can be aligned with the real data using feature alignment. By fusing the simulated and real data according to a preset ratio, a more comprehensive and realistic industrial safety scenario depth dataset can be obtained, thereby further improving the applicability and reliability of the trained model in actual industrial environments.

[0041] As one possible implementation, different types of event data can be weighted or enhanced according to the security risk level of different industrial security scenarios to ensure that the deep dataset has higher sensitivity to critical security events during training.

[0042] In this embodiment of the invention, scene configuration parameters corresponding to an industrial safety scenario are obtained; a parameterized simulation scenario is constructed based on the scene configuration parameters; in response to a preset safety event script, the human body model and equipment model in the simulation scenario are controlled to interact to generate a corresponding safety event scenario sequence, the safety event scenario sequence including scene geometric information and object attribute information; based on a sensor physical noise model, the safety event scenario sequence is subjected to depth rendering to obtain depth data, the sensor physical noise model being a noise model characterizing the measurement error of the sensor in the depth imaging process; based on the scene geometric information and the object attribute information, labeled data corresponding to the depth data is generated to construct a depth dataset corresponding to the industrial safety scenario. By constructing an industrial safety scenario depth dataset based on a parameterized simulation scenario, generating an interaction sequence of the human body model and equipment model in response to a safety event script, performing depth rendering using a sensor physical noise model, and simultaneously generating labeled data based on scene geometric information and object attributes, the depth data can simulate real sensor measurement errors, thereby providing high-quality, fully labeled samples when training a safety monitoring model, significantly improving the training effect, monitoring accuracy, and safety and reliability of industrial equipment operation of the industrial safety monitoring model.

[0043] Optionally, the safety event scenario sequence includes at least one of the following: a hazardous area scenario sequence, an object falling scenario sequence, and an equipment collision scenario sequence. In the step of controlling the interaction between the human body model and the equipment model in the simulation scenario in response to a preset safety event script to generate the corresponding safety event scenario sequence, it is also possible to: respond to a safety event script representing a person entering a hazardous area by controlling the human body model to approach the equipment model along a trajectory deviating from the safe path to generate the corresponding human body entering a hazardous area scenario sequence; respond to a safety event script representing an object falling by controlling the target object to displace relative to the load under external force disturbance to generate the corresponding object falling scenario sequence; and respond to a safety event script representing an equipment collision by controlling the human body model and / or the equipment model to approach or contact each other according to preset motion parameters to generate the corresponding equipment collision scenario sequence.

[0044] In this embodiment of the invention, when generating a sequence of safety event scenarios, a preset safety event script can be invoked to control the interaction between human models and equipment models in the simulation scenario, thereby constructing a sequence of hazardous area scenarios, a sequence of falling objects scenarios, and a sequence of equipment collision scenarios. Each scenario sequence contains scene geometry information and object attribute information. The scene geometry information describes the position, posture, and trajectory of the human model, equipment model, and other objects in three-dimensional space, while the object attribute information includes size, shape, material, reflection characteristics, and motion parameters, such as velocity and acceleration.

[0045] To efficiently generate rich and realistic event sequences, a script engine can be introduced, its core consisting of three layers. The basic motion library provides common human motion segments in industrial safety scenarios, such as walking, bending, carrying, climbing, and waving commands. Each motion is stored in skeletal animation form, with adjustable speed, amplitude, and direction. The behavior logic layer defines the motion scripts for industrial operations. For example, a complete sequence of "operator working next to the robotic arm" includes walking to the workstation, starting the equipment, picking up materials, placing materials, inspecting, and leaving the workstation; a script for "personnel accidentally entering a hazardous area" includes a sequence of actions such as normal walking, deviating from the safe path, entering the warning zone, and entering the hazardous area. The physical constraint layer verifies the rationality of all actions through a physics engine, ensuring that the human body does not cross obstacles, that the movement trajectory conforms to ergonomics, and that interactive behaviors conform to physical laws, such as posture changes when carrying heavy objects or reactions after collisions.

[0046] Based on this three-layer architecture, the script engine can automatically generate simulation sequences for six typical industrial safety events. The "Personnel Entering a Dangerous Area" event uses a behavior script that deviates from the safety passage to drive a human model towards operating hazardous equipment. The physics engine performs collision detection to ensure the trajectory does not cross physical objects. The "Personnel Too Close to Moving Equipment" event uses a minimum distance parameter set and real-time calculation and recording to ensure reasonable interaction between human actions and equipment operation. The "Object Falling or Tilting" event uses rigid body dynamics simulation to model external force disturbances causing objects to fall along the direction of gravity; the object's mass, friction coefficient, and initial position are configurable. The "Personnel Falling" event uses a contact mechanics module to calculate the falling posture of a person losing balance on slippery or uneven ground and constrains the physiological rationality of skeletal movements. The "Not Wearing Protective Equipment" event generates different missing combination samples by randomly removing 3D models of safety helmets, goggles, etc., from equipment slots. The "Multi-Agent Collaboration Anomaly" event introduces action timing deviations through multi-agent collaboration logic to simulate possible abnormal situations in handling or assembly tasks.

[0047] By parametrically controlling variables such as personnel body size, movement speed, approach angle, and equipment operating status, a large number of differentiated event variants can be automatically generated for each type of safety event, thereby significantly enriching the diversity of the dataset in the dimension of safety events and providing a sufficient scene foundation for subsequent deep data rendering and annotation generation.

[0048] As one possible implementation, environmental disturbances, such as dynamic lighting changes, temporary obstacles, or the introduction of slippery ground areas, can be added to the script engine to generate more complex industrial safety event sequences and improve the adaptability of training data to real-world scenarios.

[0049] Optionally, after responding to a preset safety event script and controlling the interaction between the human body model and the device model in the simulation scene to generate a corresponding safety event scene sequence, the interaction results between the human body model and the device model can be physically constrained to obtain the verification results. If the verification results indicate that there is clipping, collision anomaly, or abnormal motion trajectory, the motion parameters of the human body model and / or the device model are adjusted, and the safety event scene sequence is regenerated.

[0050] In this embodiment of the invention, after generating the safety event scenario sequence, physical constraint verification can be performed on the interaction results between the human body model and the device model to ensure the rationality of the action sequence and the accuracy of the simulation data. Physical constraint verification calculates the contact relationship, collision situation, and the rationality of the motion trajectory between the human body model and the device model using a physics engine. If clipping, collision anomalies, or motion trajectories that do not conform to ergonomics or physical laws are found, the motion parameters of the human body model or device model can be adjusted, such as modifying the walking path, speed, angle, or range of motion, and the safety event scenario sequence can be regenerated.

[0051] Through the aforementioned dynamic verification and adjustment mechanism, it can be ensured that the generated security event sequence closely resembles the real industrial security scenario in terms of spatial layout, action continuity, and physical constraints, thereby improving the accuracy of deep data rendering and the reliability of labeled data.

[0052] As one possible implementation, constraint priorities can be introduced during the verification process, such as prioritizing the minimum safe distance between personnel and high-speed operating equipment, or adjusting the action sequence in multi-person collaborative scenarios to avoid object collisions, thereby generating a simulated event sequence that meets the safety requirements of complex industries.

[0053] Optionally, in the step of performing depth rendering on the security event scene sequence based on the sensor physical noise model to obtain depth data, the ideal depth value corresponding to each pixel in the security event scene sequence can be determined based on scene geometric information and object attribute information; the initial noise intensity corresponding to each pixel can be determined based on the sensor physical noise model and object attribute information; the initial noise distribution corresponding to each pixel can be determined based on the sensor physical noise model and scene geometric information; the initial noise intensity and initial noise distribution can be perturbed to obtain perturbed noise parameters; and measurement errors can be introduced into the ideal depth value based on the perturbed noise parameters to obtain depth data.

[0054] In this embodiment of the invention, when performing depth rendering of a security event scene sequence based on a sensor physical noise model, it is first necessary to establish a physical noise model that reflects the characteristics of the TOF sensor. This model is used to simulate various measurement errors that the sensor may generate during depth imaging, and considers the coupling relationship between noise and the materials and geometry in the scene, so that the generated depth image is as close as possible to the depth data acquired by the TOF camera in the actual industrial environment.

[0055] The core idea of ​​the model is to first obtain the ideal depth value through ray tracing, and then calculate and inject differentiated noise pixel by pixel based on the material properties of the ray collision points and the surrounding geometry. The noise model includes five main error sources: shot noise and dark current noise, multipath interference, flying point effect, motion blur, and ambient light interference. Shot noise is related to the number of echo photons; different material reflectivities lead to differences in signal-to-noise ratio. Dark current noise is related to pixel integration time and operating temperature, constituting the measurement substrate noise. Multipath interference originates from the arrival delay of infrared light after multiple reflections; the error of metal surfaces or grooves in industrial safety scenarios is simulated by calculating the ratio of direct and indirect path photons. The flying point effect occurs in areas of depth discontinuity; a single pixel covering both the foreground and background causes mixed echoes. The model injects mixed noise based on the foreground and background area ratio and echo intensity. Motion blur calculates the degree and direction of depth blur based on object velocity and sensor integration time. Ambient light interference simulates the superposition effect of industrial light sources on the infrared sensing band; the additional noise level is calculated through spectral power distribution and sensor spectral response.

[0056] To expand the range of variation in simulation data and narrow the gap with real data, a domain randomization strategy was introduced. This strategy sets randomization parameters based on the actual variation range of the industrial environment, including noise intensity randomization (fluctuations of 0.5 to 2 times), lighting condition randomization (illuminance variations from 50 lux to 2000 lux and localized strong light sources), random generation of obstructions (dust, water mist, oil particles), and sensor parameter randomization (slight perturbations in intrinsic parameters and changes in installation pose). During depth rendering, these five types of noise are processed by domain randomization and then superimposed onto the ideal depth value, generating a depth image that physically approximates the output of a real TOF camera, while ensuring that the depth value of each pixel reflects the combined effects of scene materials, geometry, and dynamic environmental factors.

[0057] As one possible implementation, the noise model parameters can also be adjusted according to the imaging characteristics of different TOF camera models, such as changing the distribution function of shot noise intensity or the light path simulation method of multipath interference, to adapt to the depth acquisition needs of various industrial field sensors.

[0058] Optionally, the initial noise distribution includes a mixed pixel noise distribution and a multipath interference noise distribution. In the step of determining the initial noise distribution corresponding to each pixel based on the sensor physical noise model and scene geometry information, when the scene geometry information characterizes the target pixel as being in a depth discontinuous region, the mixed pixel noise distribution corresponding to the target pixel can be determined based on the area ratio of the foreground region to the background region and / or the echo intensity ratio; when the scene geometry information characterizes the target pixel as corresponding to multiple reflection paths, the multipath interference noise distribution corresponding to the target pixel can be determined based on the contribution relationship between the direct path and the indirect path.

[0059] In this embodiment of the invention, when determining the initial noise distribution corresponding to each pixel, the initial noise can mainly include two categories: mixed pixel noise distribution and multipath interference noise distribution. These two types of noise reflect the error characteristics in the depth image caused by pixels covering multiple surfaces or complex light paths. Mixed pixel noise mainly occurs in areas of depth discontinuity, such as the boundary between human contours and background, device edges, or areas of object stacking. In these areas, the photosensitive area of ​​a pixel may simultaneously cover surfaces at two different depths, the foreground and the background. The echo signal received by the sensor is a mixed echo, resulting in a measured depth value between the foreground and the background. By analyzing the area ratio of the foreground to the background and the echo intensity ratio, the mixed pixel noise that should be introduced by the pixel can be calculated, thereby more realistically simulating flying spots or depth edge deviation phenomena.

[0060] Multipath interference noise originates from the light pulses from the TOF camera reaching the sensor after multiple reflections, causing the measured depth to deviate from the true value. In industrial safety scenarios, areas such as metal surfaces, corners, and pipe recesses are prone to strong multipath reflections. The model calculates the multipath interference noise to be applied to each pixel based on the contribution ratio of photons from direct and indirect paths, arrival delays, and the relationship between light path intensity, thereby reproducing this systematic measurement error in the depth map.

[0061] By jointly modeling mixed pixel noise and multipath interference noise, the initial noise distribution of each pixel can fully reflect the complex characteristics of depth measurement in industrial safety scenarios. As a possible implementation, the intensity and distribution parameters of mixed pixel noise and multipath interference noise can also be dynamically adjusted according to different material types and surface roughness to generate richer simulated depth data, thereby improving the realism and applicability of depth data when training industrial safety monitoring models.

[0062] Optionally, the labeled data includes event-type labeled data and visual labeled data. Event-type labeled data includes security event labels and corresponding event level labels. In the step of generating labeled data corresponding to depth data based on scene geometry information and object attribute information, visual labeled data can also be generated based on scene geometry information and object attribute information. Visual labeled data includes semantic segmentation labels, instance segmentation masks, target bounding boxes, and / or depth ground truth values. Based on the event type corresponding to the security event script, security event labels corresponding to the depth data are generated. Based on the event parameters corresponding to the security event script, event level labels corresponding to the security event labels are generated.

[0063] In this embodiment of the invention, based on the generated depth data, labeled data can be constructed, including event-type labeled data and visual labeled data. Event-type labeled data is used to identify the type of safety event and its corresponding severity level, such as normal operation, personnel entering a dangerous area, object falling, or equipment collision, etc. Each event can also be labeled with an event level to reflect the degree of risk or hazard. Visual labeled data is generated based on scene geometric information and object attribute information, including semantic segmentation labels, instance segmentation masks, target bounding boxes, and accurate depth ground truth. Semantic segmentation labels assign the semantic category of the object to each pixel, covering core elements in industrial safety scenarios, such as personnel (operators, visitors, managers), robotic arms, conveyor belts, AGVs / forklifts, shelves and storage units, operating tables, fences / guardrails, ground, walls and columns, safety signs, protective equipment (safety helmets, reflective vests, goggles, safety gloves, safety shoes), doors / entrances, pipelines / cables, tools and parts, fallen objects, and background areas.

[0064] Instance segmentation masks and object bounding boxes generate uniquely numbered segmentation masks for each individual object and compute minimum bounding rectangles for instance-level object detection and segmentation tasks. Accurate depth ground truth records the noise-free ideal depth value for each pixel, serving as a supervisory signal for depth estimation and completion tasks. Security event labels annotate the event type and its severity in each frame of data based on the event type and parameters corresponding to the security event script, used to train the security event detection and classification model.

[0065] As one possible implementation, the labeled data can be further expanded to include the timing information of events or action trajectories, which can be used to train models that predict human behavior or event development trends. In addition, visual annotations can be enhanced by combining different lighting, occlusion, or sensor noise conditions to generate richer and more realistic training samples, thereby improving the generalization ability of industrial safety monitoring models in complex environments.

[0066] Optionally, in the step of generating labeled data corresponding to the depth data based on scene geometric information and object attribute information to construct a depth dataset corresponding to the industrial safety scenario, real depth data in the industrial site can also be collected; the real depth data can be labeled based on a pre-labeling model to obtain labeled data of the real depth data, and the pre-labeling model is trained based on the depth data; a first depth dataset can be constructed based on the depth data and the corresponding labeled data; a second depth dataset can be constructed based on the real depth data and the corresponding labeled data; feature alignment processing can be performed on the first depth dataset and the second depth dataset to obtain a target first depth dataset and a target second depth dataset; the target first depth dataset and the target second depth dataset can be fused according to a preset ratio to obtain a depth dataset corresponding to the industrial safety scenario.

[0067] In this embodiment of the invention, when constructing an industrial safety scenario depth dataset, simulated depth data can be combined with real-world depth data to form a virtual-real collaborative dataset. Based on the simulation data, a pre-annotated model trained on the depth data can be used to annotate the real-world depth data, thereby obtaining corresponding annotated data. In this way, simulated depth datasets and real-world depth datasets can be constructed separately. Before fusing the two types of data, multi-granularity feature alignment technology is needed to eliminate the distribution deviation between the simulation domain and the real domain.

[0068] Feature alignment comprises three levels: global feature alignment, depth interval conditional alignment, and semantic alignment of safety-critical targets. In global feature alignment, a domain discriminator module is added to the feature extraction network, and gradient inversion adversarial training is used to achieve coarse alignment between simulated and real data in terms of overall statistical properties (such as mean and variance). In depth interval conditional alignment, based on the characteristics of TOF sensor noise variation with measurement distance, depth values ​​are divided into several intervals, such as near-field, mid-field, and far-field. The maximum mean difference loss (MMD) is calculated in each interval, and the network is optimized after weighted summation to adapt to noise patterns at different distances. Semantic alignment of safety-critical targets utilizes semantic labels generated from simulated data to extract feature subsets of core target areas such as personnel and moving equipment, assigning them higher loss weights. Adversarial training prioritizes eliminating feature differences between the simulated and real domains for key targets, thereby ensuring detection performance of core safety targets. The combination of these three methods effectively narrows the gap between simulated and real data in the feature space, improving the reliability of the trained model in real-world industrial safety scenarios.

[0069] Real-world data collection employs an active acquisition strategy, guiding sampling based on scene changes, detection of personnel entering hazardous areas, and sample distribution diversity. Priority is given to capturing the most valuable data for training, reducing redundant acquisition. Depth information is acquired pixel-by-pixel using a Time-of-Flight (TOF) depth camera, and time-stamping ensures inter-frame consistency. For annotation, pre-annotated results are automatically generated using a simulation-pre-trained model and processed according to confidence levels. High-confidence samples are directly adopted, medium-confidence samples are quickly manually verified, and low-confidence samples are re-annotated manually. Simultaneously, closed-loop incremental training continuously improves model annotation accuracy, achieving an efficient and low-cost annotation process.

[0070] Finally, the simulation data and real data are fused according to a preset ratio (e.g., approximately 80% simulation data and 20% real data), and after feature alignment processing, the target dataset is obtained, which is used to train the industrial safety monitoring model. This virtual-real collaborative construction method not only reduces data acquisition costs and safety risks, and improves the realism and annotation efficiency of simulation data, but also effectively narrows the gap between simulation and real data through domain adaptation and multi-granularity alignment techniques, forming a systematic closed loop for constructing deep industrial safety datasets. This can be extended to different types of TOF sensors and diverse industrial safety scenarios, providing high-quality training data for safety monitoring models.

[0071] As one possible implementation, different weights can be assigned to different types of data samples during the fusion process based on event category or risk level to enhance the dataset's sensitivity to critical security events, or the collection density can be increased in specific areas to improve the model's ability to detect local dangerous areas.

[0072] like Figure 2 As shown in the figure, this embodiment of the invention also provides a flowchart of a second method for constructing a deep dataset, through... Figure 2 As can be seen, the process on the left illustrates the generation path of simulation data (i.e., the aforementioned depth data): First, a configurable scene template (i.e., a template for a parametric simulation scene) is constructed through parametric modeling of the scene configuration. Then, driven by the interaction action engine of the human body model and the device model, the interaction actions between the human body and the device are arranged through safety event scripts, thereby generating a sequence of safety event scenes. Next, the scene sequence is simulated and annotated using a TOF sensor physical noise model for depth rendering, and data diversity is enhanced through domain randomization, thus obtaining simulation depth data with precise annotations (i.e., the aforementioned depth data).

[0073] The flowchart on the right illustrates the acquisition path of real data (i.e., real depth data in the industrial field): real scene data is acquired in the industrial field through a TOF depth camera, combined with an active acquisition strategy based on information gain, prioritizing the acquisition of key frames, event frames, and diverse samples; then, through a simulation-guided semi-automatic annotation process, the real data is pre-annotated (i.e., the model trained using simulation data automatically generates semantic labels, instance masks, target bounding boxes, and ground truth depth values ​​for each frame of the depth map), and combined with a small amount of manual verification, to obtain fully annotated real depth data.

[0074] The middle part is the virtual-real fusion stage: the simulation data and real data are feature aligned and domain adapted and mixed to eliminate the distribution differences between the two in the feature space, and to ensure that the dataset is consistent with the real industrial safety scenario in terms of statistical characteristics and key objectives.

[0075] Finally, the fused data is divided according to the proportions for training, validation, and testing to form the final industrial safety protection TOF deep dataset, which is used to train, validate, and test industrial safety monitoring models. The entire process organically combines simulation generation, real-world data acquisition, semi-automatic annotation, and virtual-real fusion to achieve efficient, low-cost, and high-quality deep dataset construction.

[0076] like Figure 3 As shown in the figure, this embodiment of the invention also provides a flowchart of a third method for constructing a deep dataset, through... Figure 3 As can be seen from top to bottom, the process first involves selecting or generating suitable industrial environment templates based on the industrial scene template library, including workshops, warehouses, production lines, robotic arm work areas, AGV transport channels, etc. Then, based on the parameterized scene configuration, the equipment layout, material properties, lighting conditions, and other environmental factors in the templates are specified to form an industrial safety scene that can be directly used for simulation (i.e., a parameterized simulation scene).

[0077] Subsequently, the human and device models, under the control of the script engine through safety event scripts, orchestrate interactive actions to execute various safety event behavior sequences, such as personnel entering dangerous areas, objects falling, equipment collisions, or abnormal multi-person collaboration, generating a safety event scene sequence. This sequence is then simulated and rendered using a TOF sensor, combined with a physical noise model to simulate measurement errors during depth imaging, yielding depth data that approximates real-world acquisition.

[0078] Based on scene parameters (i.e., scene geometry) and object attributes (i.e., object attribute information), ground truth annotations are automatically generated, including semantic segmentation labels, instance segmentation masks, target bounding boxes, and depth ground truths. Finally, the above data is aggregated to form a simulated depth dataset (i.e., a depth dataset), serving as high-quality samples for training industrial safety monitoring models. This process organically combines parametric scene configuration, an action script engine, sensor physical noise simulation, and automatic annotation to achieve efficient, controllable, and scalable generation of depth datasets (i.e., simulated datasets).

[0079] like Figure 4 As shown in the figure, this embodiment of the invention also provides a flowchart of a fourth method for constructing a deep dataset, through... Figure 4 This demonstrates an efficient acquisition and semi-automatic annotation method based on real-world depth data (i.e., real-world depth data from industrial sites). The process begins with real-time data acquisition from a TOF camera, and simultaneously incorporates information such as scene changes, detected personnel or hazardous events, and the sparsity of feature distribution to determine trigger conditions for acquisition. The system determines whether the current frame contains valuable information; if the acquisition conditions are not met, the frame is discarded; if the conditions are met, the frame is saved as key data for subsequent annotation processing.

[0080] The saved keyframe data is fed into the simulation pre-training model to generate pre-annotated results, including target bounding boxes, semantic categories, and instance masks. Simultaneously, the model outputs a confidence score for each prediction to reflect the reliability of the pre-annotation results. The annotation results are categorized based on confidence level: high-confidence samples are directly adopted without manual intervention; medium-confidence samples are quickly verified or fine-tuned manually; and low-confidence samples are re-annotated manually. The manual annotation results can be fed back into the pre-training model for incremental training, achieving continuous optimization of annotation accuracy.

[0081] After this process, a fully labeled real training dataset (i.e., the second deep dataset) is formed, providing high-quality, low-redundancy samples for industrial safety monitoring models. This method significantly reduces the cost of real data collection and labeling through proactively triggered acquisition, simulation-guided semi-automatic labeling (i.e., labeling processing achieved through pre-labeled models), and a confidence grading strategy, while ensuring that the data covers key safety events and sparse scene features, thereby enhancing the diversity and representativeness of the deep dataset.

[0082] In one embodiment, such as Figure 5 As shown, a safety monitoring method is provided, including the following steps: 501. Obtain the safety monitoring model to be trained and the deep dataset corresponding to the industrial safety scenario; 502. The security monitoring model to be trained is trained based on the deep dataset to obtain the target security monitoring model; 503. Based on the target security monitoring model, perform security monitoring processing on the target area to obtain security monitoring results.

[0083] In this embodiment of the invention, the safety monitoring method uses a target safety monitoring model trained on a deep dataset to perform real-time monitoring and analysis of personnel, equipment, and potential safety events in industrial safety scenarios. First, the model is trained using the safety monitoring model to be trained and the corresponding deep dataset for the industrial safety scenario. The model learns the characteristic patterns of different types of safety events by studying simulated and real-world collected deep data and their annotation information. After training, the target safety monitoring model can be applied to actual industrial environments to monitor target areas in real time, identify events such as whether personnel have entered dangerous areas, whether objects have fallen, or whether equipment has experienced abnormal collisions, and output the corresponding safety monitoring results.

[0084] As one possible implementation, time series analysis or behavior prediction modules can be further integrated to perform motion trajectory analysis on depth data of continuous frames, thereby predicting potential safety events in advance and improving the early warning capabilities and accident prevention efficiency of industrial environments. Furthermore, the safety monitoring model can be fine-tuned or incrementally trained for different industrial safety scenarios or different sensor types to adapt to scenario characteristics and sensor noise features, ensuring accurate and reliable safety monitoring performance even in complex environments.

[0085] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0086] In one embodiment, a deep dataset construction apparatus is provided, which corresponds one-to-one with the deep dataset construction method described in the above embodiments. For example... Figure 6 As shown, the deep dataset construction device includes a first acquisition module 601, a construction module 602, an interaction module 603, a rendering module 604, and a generation module 605. Detailed descriptions of each functional module are as follows: The first acquisition module 601 is used to acquire the scenario configuration parameters corresponding to the industrial safety scenario; Module 602 is used to construct a parameterized simulation scenario based on the scenario configuration parameters; The interaction module 603 is used to respond to a preset safety event script and control the human body model and device model in the simulation scene to interact in order to generate a corresponding safety event scene sequence, wherein the safety event scene sequence includes scene geometric information and object attribute information. The rendering module 604 is used to perform depth rendering on the security event scene sequence based on the sensor physical noise model to obtain depth data. The sensor physical noise model is a noise model that characterizes the measurement error of the sensor in the depth imaging process. The generation module 605 is used to generate labeled data corresponding to the depth data based on the scene geometry information and the object attribute information, so as to construct a depth dataset corresponding to the industrial safety scene.

[0087] Optionally, the safety event scenario sequence includes at least one of the following: a hazardous area scenario sequence, an object falling scenario sequence, and a device collision scenario sequence. The interaction module 603 is further configured to: In response to the safety event script characterizing a person entering a dangerous area, the human body model is controlled to approach the device model along a movement trajectory that deviates from the safe path, so as to generate a corresponding sequence of scenes of a person entering a dangerous area. In response to the safety event script representing an object falling event, the target object is controlled to move relative to the support body under external force disturbance, so as to generate a corresponding object falling scene sequence; In response to the safety event script characterizing a device collision event, the human model and / or device model are controlled to approach or come into contact according to preset motion parameters to generate a corresponding device collision scene sequence.

[0088] Optionally, the device further includes: The verification module is used to perform physical constraint verification on the interaction results between the human body model and the device model, and obtain the verification results. The adjustment module is used to adjust the motion parameters of the human body model and / or device model and regenerate the safety event scene sequence when the verification results indicate that there are clipping, collision anomalies or motion trajectory anomalies.

[0089] Optionally, the rendering module 604 is further configured to: Based on the scene geometry information and the object attribute information, determine the ideal depth value corresponding to each pixel in the security event scene sequence; Based on the sensor physical noise model and the object attribute information, the initial noise intensity corresponding to each pixel is determined; Based on the sensor physical noise model and the scene geometric information, the initial noise distribution corresponding to each pixel is determined; The initial noise intensity and initial noise distribution are perturbed to obtain the perturbed noise parameters; Based on the noise parameters after the disturbance, a measurement error is introduced into the ideal depth value to obtain the depth data.

[0090] Optionally, the initial noise distribution includes a mixed pixel noise distribution and a multipath interference noise distribution, and the rendering module 604 is further configured to: When the scene geometry information characterizes the target pixel as being in a depth discontinuous region, the mixed pixel noise distribution corresponding to the target pixel is determined based on the area ratio of the foreground region to the background region and / or the echo intensity ratio. When characterizing the target pixel corresponding to multiple reflection paths using the scene geometry information, the multipath interference noise distribution corresponding to the target pixel is determined based on the contribution relationship between the direct path and the indirect path.

[0091] Optionally, the annotation data includes event-type annotation data and visual-type annotation data. The event-type annotation data includes security event labels and corresponding event level labels. The generation module 605 is further used for: Based on the scene geometry information and the object attribute information, the visual class annotation data is generated, which includes semantic segmentation labels, instance segmentation masks, target bounding boxes, and / or depth ground truth. Based on the event type corresponding to the security event script, generate security event tags corresponding to the depth data; Based on the event parameters corresponding to the security event script, an event level label corresponding to the security event label is generated.

[0092] Optionally, the generation module 605 is further configured to: Collect real-world depth data from industrial sites; The real depth data is labeled based on the pre-labeling model to obtain labeled data of the real depth data. The pre-labeling model is trained based on the depth data. Based on the depth data and the corresponding labeled data, a first depth dataset is constructed; Based on the actual depth data and the corresponding labeled data, a second depth dataset is constructed; Based on the first depth dataset and the second depth dataset, feature alignment processing is performed to obtain the target first depth dataset and the target second depth dataset; The target first depth dataset and the target second depth dataset are fused according to a preset ratio to obtain the depth dataset corresponding to the industrial safety scenario.

[0093] In one embodiment, a safety monitoring device is provided, which corresponds one-to-one with the safety monitoring methods described in the above embodiments. For example... Figure 7 As shown, the safety monitoring device includes a second acquisition module 701, a training module 702, and a monitoring module 703. Detailed descriptions of each functional module are as follows: The second acquisition module 701 is used to acquire the safety monitoring model to be trained and the deep dataset corresponding to the industrial safety scenario. Training module 702 is used to train the security monitoring model to be trained based on the deep dataset to obtain the target security monitoring model; The monitoring module 703 is used to perform security monitoring processing on the target area based on the target security monitoring model to obtain security monitoring results.

[0094] Each module in the aforementioned deep dataset construction device or security monitoring device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0095] In one embodiment, a computer device is provided, which may be a terminal device, and its internal structure diagram may be as follows: Figure 8 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computational and control capabilities. The memory includes a readable storage medium storing computer-readable instructions. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer-readable instructions implement a deep dataset construction method or a security monitoring method. The readable storage medium provided in this embodiment includes both non-volatile and volatile readable storage media.

[0096] In this application embodiment, a computer device is provided, including a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor. When the processor executes the computer-readable instructions, it implements the steps of the deep dataset construction method or security monitoring method described above.

[0097] In one embodiment of the application, a readable storage medium is provided, which stores computer-readable instructions. When executed by a processor, the computer-readable instructions implement the steps of the deep dataset construction method or security monitoring method described above.

[0098] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware with computer-readable instructions. These computer-readable instructions can be stored in a non-volatile readable storage medium or a volatile readable storage medium. When executed, these computer-readable instructions can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0099] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0100] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for constructing a deep dataset, characterized in that, The method includes: Obtain the scenario configuration parameters corresponding to the industrial safety scenario; Based on the scenario configuration parameters, a parameterized simulation scenario is constructed; In response to a preset safety event script, the human body model and device model in the simulation scene are controlled to interact to generate a corresponding safety event scene sequence, which includes scene geometric information and object attribute information. Based on the sensor physical noise model, the security event scene sequence is rendered with depth data to obtain depth data. The sensor physical noise model is a noise model that characterizes the measurement error of the sensor in the depth imaging process. Based on the scene geometry information and the object attribute information, labeled data corresponding to the depth data is generated to construct a depth dataset corresponding to the industrial safety scenario. The method of performing depth rendering on the security event scene sequence based on the sensor physical noise model to obtain depth data includes: Based on the scene geometry information and the object attribute information, determine the ideal depth value corresponding to each pixel in the security event scene sequence; Based on the sensor physical noise model and the object attribute information, the initial noise intensity corresponding to each pixel is determined; Based on the sensor physical noise model and the scene geometric information, the initial noise distribution corresponding to each pixel is determined; The initial noise intensity and initial noise distribution are perturbed to obtain the perturbed noise parameters; Based on the disturbed noise parameters, a measurement error is introduced into the ideal depth value to obtain the depth data; The initial noise distribution includes a mixed pixel noise distribution and a multipath interference noise distribution. Determining the initial noise distribution for each pixel based on the sensor's physical noise model and the scene geometry information includes: When the scene geometry information characterizes the target pixel as being in a depth discontinuous region, the mixed pixel noise distribution corresponding to the target pixel is determined based on the area ratio of the foreground region to the background region and / or the echo intensity ratio. When characterizing the target pixel corresponding to multiple reflection paths using the scene's geometric information, the multipath interference noise distribution corresponding to the target pixel is determined based on the contribution relationship between the direct path and the indirect path. The intensity parameters and distribution parameters of the mixed pixel noise distribution and the multipath interference noise distribution are adjusted based on the material type and surface roughness of each pixel.

2. The deep dataset construction method as described in claim 1, characterized in that, The safety event scenario sequence includes at least one of three: a hazardous area scenario sequence, an object falling scenario sequence, and an equipment collision scenario sequence. The step of responding to a preset safety event script, controlling the human and equipment models in the simulation scenario to interact, and generating the corresponding safety event scenario sequence, includes: In response to the safety event script characterizing a person entering a dangerous area, the human body model is controlled to approach the device model along a movement trajectory that deviates from the safe path, so as to generate a corresponding sequence of scenes of a person entering a dangerous area. In response to the safety event script representing an object falling event, the target object is controlled to move relative to the support body under external force disturbance, so as to generate a corresponding object falling scene sequence; In response to the safety event script characterizing a device collision event, the human model and / or device model are controlled to approach or come into contact according to preset motion parameters to generate a corresponding device collision scene sequence.

3. The deep dataset construction method as described in claim 2, characterized in that, After responding to a preset security event script and controlling the human body model and device model in the simulation scenario to interact in order to generate a corresponding security event scenario sequence, the method further includes: The interaction results between the human body model and the device model are subjected to physical constraint verification to obtain the verification results; If the verification results indicate that there are abnormalities such as clipping, collision, or abnormal motion trajectory, the motion parameters of the human body model and / or device model are adjusted, and the safety event scenario sequence is regenerated.

4. The deep dataset construction method as described in claim 1, characterized in that, The labeled data includes event-type labeled data and visual-type labeled data. The event-type labeled data includes security event labels and corresponding event level labels. Generating labeled data corresponding to the depth data based on the scene geometry information and the object attribute information includes: Based on the scene geometry information and the object attribute information, the visual class annotation data is generated, which includes semantic segmentation labels, instance segmentation masks, target bounding boxes, and / or depth ground truth. Based on the event type corresponding to the security event script, generate security event tags corresponding to the depth data; Based on the event parameters corresponding to the security event script, an event level label corresponding to the security event label is generated.

5. The method for constructing a deep dataset as described in any one of claims 1 to 4, characterized in that, The step of generating labeled data corresponding to the depth data based on the scene geometry information and the object attribute information to construct a depth dataset corresponding to the industrial safety scenario includes: Collect real-world depth data from industrial sites; The real depth data is labeled based on the pre-labeling model to obtain labeled data of the real depth data. The pre-labeling model is trained based on the depth data. Based on the depth data and the corresponding labeled data, a first depth dataset is constructed; Based on the actual depth data and the corresponding labeled data, a second depth dataset is constructed; Based on the first depth dataset and the second depth dataset, feature alignment processing is performed to obtain the target first depth dataset and the target second depth dataset; The target first depth dataset and the target second depth dataset are fused according to a preset ratio to obtain the depth dataset corresponding to the industrial safety scenario.

6. A safety monitoring method, characterized in that, The method includes: Obtain the safety monitoring model to be trained and the deep dataset corresponding to the industrial safety scenario as described in any one of claims 1 to 5; The security monitoring model to be trained is trained based on the deep dataset to obtain the target security monitoring model. Based on the target security monitoring model, security monitoring is performed on the target area to obtain security monitoring results.

7. A deep dataset construction apparatus, characterized in that, The device includes: The first acquisition module is used to acquire the scenario configuration parameters corresponding to the industrial safety scenario; The construction module is used to construct a parameterized simulation scenario based on the scenario configuration parameters; The interaction module is used to respond to a preset safety event script and control the human body model and device model in the simulation scene to interact in order to generate a corresponding safety event scene sequence, the safety event scene sequence including scene geometric information and object attribute information; The rendering module is used to perform depth rendering on the security event scene sequence based on the sensor physical noise model to obtain depth data. The sensor physical noise model is a noise model that characterizes the measurement error of the sensor in the depth imaging process. The generation module is used to generate labeled data corresponding to the depth data based on the scene geometry information and the object attribute information, so as to construct a depth dataset corresponding to the industrial safety scene. The rendering module is also used for: Based on the scene geometry information and the object attribute information, determine the ideal depth value corresponding to each pixel in the security event scene sequence; Based on the sensor physical noise model and the object attribute information, the initial noise intensity corresponding to each pixel is determined; Based on the sensor physical noise model and the scene geometric information, the initial noise distribution corresponding to each pixel is determined; The initial noise intensity and initial noise distribution are perturbed to obtain the perturbed noise parameters; Based on the disturbed noise parameters, a measurement error is introduced into the ideal depth value to obtain the depth data; The initial noise distribution includes a mixed pixel noise distribution and a multipath interference noise distribution. The rendering module is further configured to: When the scene geometry information characterizes the target pixel as being in a depth discontinuous region, the mixed pixel noise distribution corresponding to the target pixel is determined based on the area ratio of the foreground region to the background region and / or the echo intensity ratio. When characterizing the target pixel corresponding to multiple reflection paths using the scene's geometric information, the multipath interference noise distribution corresponding to the target pixel is determined based on the contribution relationship between the direct path and the indirect path. The intensity parameters and distribution parameters of the mixed pixel noise distribution and the multipath interference noise distribution are adjusted based on the material type and surface roughness of each pixel.

8. A safety monitoring device, characterized in that, The device includes: The second acquisition module is used to acquire the safety monitoring model to be trained and the deep dataset corresponding to the industrial safety scenario as described in any one of claims 1 to 5. The training module is used to train the security monitoring model to be trained based on the deep dataset to obtain the target security monitoring model. The monitoring module is used to perform security monitoring processing on the target area based on the target security monitoring model to obtain security monitoring results.

Images