Safety adjustment control system for theater performance stage equipment

Abstract

The invention discloses a theater performance stage equipment safety adjustment control system, which relates to an intelligent automatic control system and comprises a map construction module for monitoring objects and personnel on a stage in real time, constructing a three-dimensional map of the stage and collecting stage environment data and image data, a preprocessing module for preprocessing the stage image data and sending the preprocessed stage image data to a server. The system comprises a preprocessing module for preprocessing stage image data to obtain preprocessed stage image data, a path generation module for generating an optimal path of stage equipment according to a three-dimensional map of a stage, and an avoidance module for detecting obstacles on the stage in real time and re-planning the optimal path after the obstacles are detected. By establishing the human body recognition model, the position of the actor can be accurately recognized in a complex stage environment, the Gaussian weight is introduced to calculate the human body recognition probability, judgment is performed according to the set human body detection threshold, and the probability of false detection and missing detection is effectively reduced.

Description

Technical Field

[0001] The present invention relates to the field of intelligent automation control systems, and particularly to a safety adjustment control system for theater performance stage equipment. Background Art

[0002] With the development of theater performance art, the safety and intelligence level of stage equipment have gradually become the focus of attention in the industry. The control of traditional theater performance stage equipment mainly relies on manual operation and simple automation control. In recent years, with the introduction of advanced technologies such as computer vision, robotics, automatic control theory, and multi-sensor fusion, the control mode of stage equipment has been significantly improved.

[0003] The existing technologies have obvious deficiencies in dealing with the safety adjustment of theater performance stage equipment. First of all, the current path planning algorithms are mostly based on the assumption of a static environment. When temporary obstacles appear on the stage or the positions of actors change suddenly, the equipment cannot adjust the path in time, resulting in potential safety hazards. In addition, most traditional human detection models adopt two-dimensional image analysis from a single perspective, ignoring the three-dimensional space information, which not only limits the detection accuracy but also leads to misjudgment or missed detection. Summary of the Invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a safety adjustment control system for theater performance stage equipment to solve the problem that the path planning algorithms are mostly based on the assumption of a static environment. When temporary obstacles appear on the stage or the positions of actors change suddenly, the equipment cannot adjust the path in time, resulting in potential safety hazards.

[0006] To solve the above technical problems, the present invention provides the following technical solutions:

[0007] The present invention provides a safety adjustment control system for theater performance stage equipment, which includes,

[0008] A map construction module, which monitors the objects and personnel on the stage in real time, constructs a three-dimensional map of the stage, and collects stage environment data and image data;

[0009] A preprocessing module, which preprocesses the stage image data to obtain the preprocessed stage image data;

[0010] A path generation module, which generates the optimal path of the stage equipment according to the three-dimensional map of the stage;

[0011] An avoidance module, which detects the obstacles on the stage in real time and replans the optimal path after detecting the obstacles;

[0012] The human body recognition module establishes a human body detection model based on the pre - processed stage image data, identifies the positions of all actors, and tracks the actor positions in real - time;

[0013] The safety assessment module sets safety areas and dangerous areas based on the 3D map of the stage, and evaluates whether an actor is in a dangerous area according to the actor's position;

[0014] The human body safety module issues an alarm when an actor is in a dangerous area and generates a human body early - warning report;

[0015] The emergency module introduces dual - power supply technology, sets a redundant mode, and formulates an emergency response plan.

[0016] Preferably, it includes: real - time monitoring of objects and people on the stage, constructing a 3D map of the stage, collecting stage environment data and image data. The specific steps are as follows:

[0017] Deploy lidar and cameras on the stage, start the lidar to scan the stage, and obtain point - cloud data about the stage;

[0018] Take pictures of the stage area through the camera to obtain stage image data;

[0019] Set an initial point as a reference position, and record the lidar point - cloud data at the reference position and the image at the corresponding moment;

[0020] Perform Iterative Closest Point (ICP) algorithm matching on the lidar point - cloud data between two adjacent frames to obtain the relative pose transformation matrix between the two frames. At the same time, use the Random Sample Consensus (RANSAC) algorithm to estimate the relative motion between the two frames;

[0021] Convert the point - cloud data of each frame to the global coordinate system according to the current pose transformation matrix and accumulate it into the global point - cloud map;

[0022] Use a segmentation algorithm to divide the point - cloud data into different object categories;

[0023] Apply a multi - target tracking algorithm to track the position changes of dynamic objects and complete the construction of the 3D map of the stage;

[0024] The 3D map of the stage includes the stage layout, obstacle positions, and actor positions.

[0025] Preferably, it includes: pre - processing the stage image data to obtain pre - processed stage image data. The specific steps are as follows:

[0026] Use a Gaussian filter to remove noise in the stage image data;

[0027] Use histogram equalization to enhance the contrast of the stage image data;

[0028] Color correction is performed on the stage image data using color space conversion and white balance adjustment.

[0029] Preferably, wherein: an optimal path for stage equipment is generated based on a three-dimensional map of the stage, specifically including the following steps

[0030] According to the performance requirements, mark all the key positions that the stage equipment needs to reach in the three-dimensional map;

[0031] Convert the coordinates of the marked key positions from the image coordinate system to the global coordinate system;

[0032] Discretize the three-dimensional map into a grid map;

[0033] Project the discretized grid map to obtain a two-dimensional grid map, where the corresponding grids represent passable and non-passable areas,

[0034] Define each grid as a path node and set the movement cost between each node;

[0035] Set the starting point and the target point in the two-dimensional grid map and define an open list and a closed list;

[0036] The open list is to record the nodes to be explored, and the closed list is to record the nodes that have been visited;

[0037] Find the node with the minimum total cost in the open list, denote it as the current node. If the current node is the target point, stop the search, generate a path, and move the current node to the closed list;

[0038] Based on the current node and the target point, use the Euclidean distance to define a heuristic function, and its expression is:

[0039] ;

[0040] Wherein, represents the Euclidean distance value, represents the current node, represents the abscissa of the current node, represents the abscissa of the target node, represents the ordinate of the current node, represents the ordinate of the target node;

[0041] Accumulate based on the movement cost of the nodes to obtain the actual cost of node movement;

[0042] According to the heuristic function and the actual cost of node movement, define a path cost function to measure the total cost of the path, and its expression is:

[0043] ;

[0044] Among them, represents the path cost function value, represents the actual cost of node movement;

[0045] Find all adjacent nodes of the current node, and skip the impassable nodes and the nodes that are already in the closed list among the adjacent nodes;

[0046] Calculate the actual cost from the current node to the adjacent node;

[0047] If the adjacent node is not in the open list, add it to the open list and record the parent node of the adjacent node as the current node;

[0048] If the adjacent node is in the open list and the new actual cost is smaller, update the actual cost and the path cost function value;

[0049] When the target point is selected as the current node, the search is completed;

[0050] Starting from the target point, trace back along the parent node of each node until returning to the starting point;

[0051] Reverse the order of the nodes obtained by backtracking to generate the optimal path from the starting point to the target point.

[0052] Preferably, among them: Obstacles on the stage are detected in real time, and the optimal path is re-planned after detecting the obstacles. Specifically, it includes the following steps.

[0053] Use lidar and cameras to detect the obstacle information of the stage in real time, and map the obstacle information to a two-dimensional grid map;

[0054] Set the maximum linear velocity and maximum angular velocity of the stage equipment, and define the velocity sampling interval;

[0055] Set the prediction time step and sampling period;

[0056] For each speed combination, predict the future trajectory of the stage equipment, and its expression is:

[0057] ;

[0058] ;

[0059] ;

[0060] Among them, represents the abscissa of the stage equipment at time , represents the abscissa of the stage equipment at time , represents the linear velocity, represents the cosine value, represents the orientation angle of the stage device at time ; represents the time step, represents the ordinate of the stage device at time ; represents the ordinate of the stage device at time ; represents the orientation angle of the stage device at time ; represents the angular velocity of the device;

[0061] Optimize the speed of the stage device within the dynamic window;

[0062] Calculate the orientation score through the deviation angle between the current orientation and the target direction of the stage device;

[0063] Obtain the obstacle avoidance score through the minimum distance between the stage device trajectory and the obstacle;

[0064] Calculate the ratio of the current speed to the maximum speed to obtain the speed score;

[0065] Based on the orientation score, obstacle avoidance score, and speed score, select the speed combination that maximizes the total score function, and its expression is:

[0066] ;

[0067] where, represents the total score value of the current speed combination, represents the orientation score, represents the weight of the orientation score, represents the obstacle avoidance score, represents the weight of the obstacle avoidance score, represents the speed score, represents the weight of the speed score;

[0068] Based on the speed combination with the maximum total score, update the position and orientation of the stage device, and generate the optimal path from the current position to the target position.

[0069] Preferably, wherein: establish a human detection model according to the preprocessed stage image data, identify the positions of all actors, and track the actor positions in real time. The specific steps are as follows:

[0070] Use the YOLO model to extract the shallow feature map and deep feature map of the stage image, and use the detection head of the YOLO model to output the class scores and grid cell bounding box parameters of each feature map coordinate point;

[0071] Use the activation function to convert the class scores into class probabilities;

[0072] Perform a non - linear activation function processing on the feature map, and weight the activated feature values by the class probabilities;

[0073] Accumulate the weighted features of the shallow - layer feature map and the deep - layer feature map to obtain multi - scale fusion features, and perform normalization processing;

[0074] Apply the Sigmoid function to the result of the normalization processing;

[0075] Take the center point of the grid cell bounding box parameters of the feature map as the detection center point, calculate the Gaussian weight for the detection center point, and obtain the calculation result of the Gaussian weight;

[0076] Set the human detection threshold according to the requirements of the stage performance;

[0077] Calculate the output of the Sigmoid function and the calculation result of the Gaussian weight to obtain the recognition probability of the human body, and its expression is:

[0078] ;

[0079] where, represents the probability that point belongs to the human body at time , represents the Sigmoid function, represents the normalization factor, represents the scale of the feature, represents the non - linear activation function, represents the feature map scale at point value, represents point at scale and time belongs to the class probability of the human target, represents point at time spatial weight;

[0080] Compare the recognition probability of the human body with the human detection threshold. If it is greater than the human detection threshold, then determine that this point is a human target, and track and capture the position of the human body in real - time.

[0081] Preferably, among them: Set the safe area and the dangerous area based on the three - dimensional map of the stage, and evaluate whether it is in the dangerous area according to the position of the actor. The specific steps are as follows,

[0082] Define the edge of the rotating stage, the edge of the lifting platform, and the movement range of the robotic arm as the dangerous area;

[0083] Define other areas outside the danger zone as safe areas and set safety thresholds based on the distance from the danger zone.

[0084] After projecting the stage onto a two-dimensional plane, divide it into small cells and mark the status of the corresponding cells as danger zones and safe areas.

[0085] When the detected distance of the current position of the human body from the danger zone is lower than the safety threshold, it is determined that the actor is in the danger zone and a warning message is generated.

[0086] Preferably, when an actor is in the danger zone, an alarm is issued and a human body warning report is generated, which specifically includes the following steps.

[0087] When an actor enters the danger zone, immediately send a warning message to the on-site staff and trigger the protection mechanism at the same time.

[0088] Generate a personnel warning report, which includes the location of the danger zone, the actor number, and the time.

[0089] Preferably, introduce a dual-power supply technology, which specifically includes the following steps.

[0090] Configure a main power supply and a backup power supply for the stage, both of which are independently powered and connected to a power monitoring sensor.

[0091] Select redundant sensors for the temperature sensor and the vibration sensor and install the redundant sensors at different positions.

[0092] Compare the data of the sensor and the redundant sensor. When the data of one sensor shows a significant deviation, it is considered that the sensor has failed and the data of the other sensor is switched.

[0093] Preferably, set a redundant mode and formulate an emergency response plan, which specifically includes the following steps.

[0094] When performing an operation, require the user to send the same instruction twice. Only when the two instructions are exactly the same will the operation be actually executed.

[0095] For major operations, approval from the management staff is also required.

[0096] When a fault is detected, an alarm will be immediately triggered, the fault location will be isolated, the maintenance staff will be notified, and the fault information will be automatically recorded.

[0097] After the fault is repaired, a self-check is performed. If there is no error, a success signal is sent.

[0098] The beneficial effects of the present invention are as follows: By real-time monitoring of obstacle information on the stage and mapping this information onto a two-dimensional grid map. And predicting future trajectories and selecting the optimal speed combination to ensure the safe operation of stage equipment even in case of emergencies. The introduction of orientation scoring, obstacle avoidance scoring, and speed scoring enables path adjustment to not only consider physical distance but also take into account the motion state of the equipment and environmental changes, achieving a more intelligent and efficient obstacle avoidance strategy. In addition, by establishing a human body recognition model, the position of actors can be accurately recognized in a complex stage environment, and Gaussian weights are introduced to calculate the human body recognition probability, and judgment is made according to the set human body detection threshold, effectively reducing the probability of false detection and missed detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0099] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention, and those of ordinary skill in the art can obtain other drawings without creative efforts based on these drawings.

[0100] Figure 1 It is a schematic diagram of the safety adjustment control system for theater performance stage equipment in Embodiment 1.

[0101] Figure 2 It is a schematic diagram of the generation of the optimal path in Embodiment 1. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0102] To make the above objects, features, and advantages of the present invention more obvious and understandable, the following will provide a detailed description of the specific embodiments of the present invention with reference to the accompanying drawings of the specification.

[0103] Embodiment 1, referring to Figure 1 and Figure 2 This is the first embodiment of the present invention, which provides a safety adjustment control system for theater performance stage equipment, including the following steps:

[0104] A map construction module that real-time monitors objects and personnel on the stage, constructs a three-dimensional map of the stage, and collects stage environment data and image data.

[0105] Deploy lidar and cameras in the stage area.

[0106] The lidar is used to obtain accurate three-dimensional point cloud data and capture the spatial position information of the surface of stage objects. The lidar has high precision and all-weather working ability and can provide distance measurements at the millimeter level.

[0107] The camera is used to obtain two-dimensional image data of the stage, providing rich texture, color, and visual features for subsequent object segmentation, classification, and tracking.

[0108] Start the lidar to scan the stage area and generate three-dimensional point cloud data about the stage. These point cloud data contain the position and geometric shape information of the objects (such as obstacles, props, actors, etc.) on the stage.

[0109] Take pictures of the stage area through the camera to generate high-resolution two-dimensional image data. These image data contain the texture, color and other visual features of the stage surface, and can provide supplementary information for the point cloud data.

[0110] Select a certain position on the stage as a reference point, and record the point cloud data of the reference point and the corresponding image at the corresponding moment.

[0111] Efficiently calculate the relative motion between two adjacent frames through the Iterative Closest Point (ICP) algorithm and the Random Sample Consensus (RANSAC) algorithm.

[0112] Transform each frame of point cloud data to the global coordinate system according to the pose transformation matrix of the current frame to form a unified three-dimensional map of the stage. And use the point cloud segmentation algorithm to divide the point cloud data into different object categories.

[0113] Use data association to associate the trajectories of dynamic objects through the point cloud segmentation results and the camera images to ensure the continuity of tracking.

[0114] The finally generated three-dimensional map of the stage contains the following information: the stage layout, the positions of obstacles and the positions of actors.

[0115] The stage layout refers to the global geometric structure of the stage.

[0116] The positions of obstacles refer to the precise positions and distributions of the obstacles (such as tables, chairs, props) on the stage.

[0117] The preprocessing module preprocesses the stage image data to obtain the preprocessed stage image data.

[0118] Gaussian filtering is a classic image smoothing tool. The stage images obtained from the camera may contain noise (such as sensor noise, ambient light interference). Apply a Gaussian kernel to perform a convolution operation on the image to smooth the random noise in the image while retaining the edge and structure information of the objects. It can effectively remove the high-frequency noise in the stage images, such as random bright spots, color anomalies, etc.

[0119] Histogram equalization is a technique for adjusting the grayscale distribution of an image. It makes the brightness and contrast of an image more uniform by redistributing the grayscale values ​​of image pixels. The number of pixels of each grayscale value in the image is counted to obtain the grayscale histogram of the image. The cumulative distribution function is generated using the histogram, and the grayscale value of each pixel is redistributed to output the image with enhanced contrast. This step can make objects on the stage more clearly visible, especially the details of dark and highlight areas.

[0120] Color space is a mathematical model for representing colors in images. White balance is an important step in image color correction. Its purpose is to eliminate the influence of light source color on the image and make the white area appear truly white.

[0121] Specifically, the RGB image is converted into the Lab color space, the white balance of the image is adjusted according to the stage lighting conditions to eliminate the color cast of the light source, and the corrected image is converted back to the RGB space.

[0122] Further explanation: Correcting the color distortion caused by stage lighting makes the colors in the image closer to the real scene.

[0123] The path generation module generates the optimal path for stage equipment based on the three-dimensional map of the stage.

[0124] Key positions are specific locations that stage equipment (such as lighting equipment, sound equipment, props, etc.) need to reach. The marked key positions are converted from the image coordinate system (2D pixel coordinates) to the global coordinate system of the 3D map (real physical coordinates).

[0125] It is further explained that global positioning of key positions of the equipment is provided to ensure that the equipment can accurately reach the specified location.

[0126] Discretize the 3D map and divide the continuous 3D point cloud into voxels (3D grid units) of fixed size. Each voxel represents a small cubic space.

[0127] Project the three-dimensional grid map onto a two-dimensional plane to generate a two-dimensional grid map.

[0128] Each grid in the two-dimensional grid map represents a plane area of ​​a fixed size. The area is divided into passable areas and impassable areas. The passable areas are marked as 0 and the impassable areas are marked as 1.

[0129] It is further explained that simplifying a complex three-dimensional map into a two-dimensional grid map reduces the computational complexity.

[0130] Define each grid in the two-dimensional grid map as a path node, and define the movement cost from one grid to the adjacent grid, for example, the horizontal or vertical movement cost is 1. The oblique movement cost is .

[0131] Use the Euclidean distance as the heuristic function to estimate the cost from the current node to the target node, and its expression is:

[0132] ;

[0133] where represents the Euclidean distance value, represents the current node, represents the abscissa of the current node, represents the abscissa of the target node, represents the ordinate of the current node, represents the ordinate of the target node;

[0134] Accumulate based on the movement cost of the node to obtain the actual cost of node movement

[0135] According to the heuristic function and the actual cost of node movement, define the path cost function to measure the total cost of the path, and its expression is:

[0136] ;

[0137] where represents the path cost function value, represents the actual cost of node movement;

[0138] Specifically, set the starting point and the target point on the grid map, and define the open list and the closed list.

[0139] The open list is used to record the nodes to be explored, and the closed list is used to record the nodes that have been visited.

[0140] Find the node with the minimum total cost from the open list and set it as the current node.

[0141] Check whether the current node is the target point. If it is the target point, stop the search and generate the path. If not, move the current node to the closed list.

[0142] Find all adjacent nodes of the current node, and skip the impassable nodes and the nodes that are already in the closed list.

[0143] For each adjacent node, calculate the actual cost and the total path cost.

[0144] If the adjacent node is not in the open list, add it to the open list and record its parent node as the current node. If the adjacent node is already in the open list and the new actual cost is smaller, update the cost and re-record the parent node. When the target point is selected as the current node, the search is completed.

[0145] Furthermore, based on the discretized grid map and the heuristic search algorithm, the optimal path from the starting point to the target point is quickly generated for the device, ensuring the lowest path cost while avoiding obstacles and impassable areas.

[0146] The avoidance module detects obstacles on the stage in real time and replans the optimal path after detecting an obstacle.

[0147] According to the physical performance of the stage device, the maximum linear velocity and the maximum angular velocity of the device are set. Ensure that the device moves within a safe range and avoid out-of-control caused by too high speed. Discretely sample the linear velocity and the angular velocity within the dynamic window.

[0148] The total time range of the prediction time step is usually from 1 to 2 seconds.

[0149] The sampling period is the time step of each prediction, which determines the time resolution of the trajectory.

[0150] For each speed combination, predict the future trajectory of the stage device, and its expression is:

[0151] ;

[0152] ;

[0153] ;

[0154] Among them, represents the abscissa of the stage device at time , represents the abscissa of the stage device at time , represents the linear velocity, represents the cosine value, represents the orientation angle of the stage device at time , represents the time step, represents the ordinate of the stage device at time , represents the ordinate of the stage device at time , represents the orientation angle of the stage device at time , represents the angular velocity of the device;

[0155] By predicting the trajectories of each speed combination, the movement path of the stage device in the future for a period of time is obtained.

[0156] Optimize the speed of the stage device within the dynamic window. The specific steps are as follows:

[0157] Calculated based on the deviation angle between the current orientation of the device and the target direction. The smaller the deviation, the higher the score, which is expressed as:

[0158] ;

[0159] Among them, represents the orientation score, represents the current orientation angle of the device, represents the orientation angle of the target direction of the device;

[0160] Calculated by the minimum distance between the device trajectory and the obstacle. The larger the distance, the higher the score. The expression is:

[0161] ;

[0162] Among them, represents the obstacle avoidance score, represents the minimum distance between the trajectory and the nearest obstacle.

[0163] The ratio of the current speed to the maximum speed is used to encourage the device to move forward at a higher speed. The expression is:

[0164] ;

[0165] Among them, represents the speed score, represents the maximum speed.

[0166] Based on the orientation score, obstacle avoidance score, and speed score, select the speed combination that maximizes the total score function. The expression is:

[0167] ;

[0168] Among them, represents the total score value of the current speed combination, represents the weight of the orientation score,, represents the weight of the obstacle avoidance score, represents the weight of the speed score;

[0169] Based on the selected optimal speed combination, update the current position and orientation of the stage device. Starting from the updated device position, re-plan the optimal path to the target point. If the target point remains unchanged, only locally adjust the path to avoid newly emerging obstacles.

[0170] The human body recognition module establishes a human body detection model based on the preprocessed stage image data, identifies the positions of all actors, and real-time tracks the actor positions.

[0171] The shallow feature map is extracted by the first few convolutional layers of the YOLO model and mainly contains low-level features of the image, such as texture, edges, colors, etc. It is suitable for detecting small objects or targets with rich details (such as gestures or small props on the stage). The deep feature map is extracted by the last few convolutional layers and mainly contains high-level semantic features of the image, such as overall shape, structure, and complex semantic information. It is suitable for detecting large objects or targets with clear semantics (such as human bodies or large props).

[0172] The detection head of the YOLO model analyzes each grid cell of different feature maps and outputs class scores and bounding box parameters.

[0173] The class parameter is the predicted score of different classes in each grid cell, indicating the possibility that the cell belongs to a certain class (such as human body, prop, etc.).

[0174] The bounding box parameter is the bounding box parameter predicted by each grid cell, which contains the coordinates of the center point of the bounding box.

[0175] The activation function is used to convert the class scores output by the detection head into class probabilities. The class probability represents the normalized predicted value that the grid cell belongs to a certain class (such as human body), and the range is between [0,1].

[0176] The feature map output by YOLO is processed by a non-linear activation function to activate the feature values and highlight the key features.

[0177] The weighted features of the shallow feature map and the deep feature map are accumulated to obtain the fused multi-scale features.

[0178] The accumulated multi-scale features are normalized to ensure that the feature values are within a reasonable range and avoid instability caused by overly large or small numerical values. The Sigmoid function is applied to the normalized result to map the feature values to the range of [0,1], further enhancing the numerical stability.

[0179] Furthermore, multi-scale fusion improves the detection ability for targets of different scales. The use of normalization and the Sigmoid function ensures the unified range of feature values and avoids numerical fluctuations affecting the detection performance.

[0180] The center point coordinates of the bounding box parameter of the feature map grid cell are taken as the detection center point. According to the detection center point, the Gaussian weight is calculated. The Gaussian weight reflects the spatial influence range of the target point, and the weight value decreases as the distance from the center point increases.

[0181] Combining the output of the Sigmoid function and the Gaussian weight, calculate the point At time The probability of belonging to a human body, and its expression is:

[0182] ;

[0183] Among them, represents the probability that point belongs to the human body at time . represents the Sigmoid function, represents the normalization factor, represents the scale of the feature, represents the non-linear activation function, represents the feature map scale at point . represents point at the feature map scale and time belongs to the category probability of the human body target, represents point at time spatial weight of

[0184] According to the requirements of stage performances, set the human detection threshold. For example, set the human detection threshold to 0.7,

[0185] If is less than 0.7, it is ignored, and if it is greater than 0.7, it is determined as a human body. For the detected human body target, track its position change in real time.

[0186] Safety assessment module, based on the 3D map of the stage, set the safe area and the dangerous area, and evaluate whether the actor is in the dangerous area according to the position of the actor.

[0187] The dangerous area refers to the area that may pose a potential threat to the actor or other personnel during the stage performance. Define the edge of the rotating stage, the edge of the lifting platform, and the movement range of the robotic arm as the dangerous area. Define the area beyond a certain distance from the dangerous area as the safe area.

[0188] Project the 3D stage environment onto a 2D plane for calculation and analysis of the stage area.

[0189] Divide the stage into small cells at a fixed resolution.

[0190] Mark the cells in the dangerous areas such as the edge of the rotating stage, the edge of the lifting platform, and the movement range of the robotic arm, and set the status as the dangerous area.

[0191] The cells that do not belong to the dangerous area are marked as the safe area, but the safety level needs to be further calculated according to the distance from the dangerous area.

[0192] The safety threshold refers to the minimum safe distance between the actor and the dangerous area.

[0193] The actor’s current position is detected in real time and the distance between the actor and the nearest dangerous area cell is calculated. When the distance is less than the safety threshold, the actor is determined to be in a dangerous area and a warning message is generated.

[0194] The human safety module sounds an alarm when an actor is in a dangerous area and generates a human early warning report.

[0195] When an actor enters a dangerous area, an early warning message is immediately sent to the on-site staff. The early warning message is an audible warning, such as the actor is approaching a dangerous area, please pay attention to safety! There is also a visual warning, such as a red warning light on the stage area (such as an LED indicator light at the edge of the dangerous area).

[0196] The protection mechanism is to immediately reduce the speed of the rotating stage or stop the rotation to prevent the actors from sliding down or being drawn into the edge of the stage due to inertia. The height change of the lifting platform is stopped to prevent the actors from falling and causing injuries. The movement of the robotic arm is paused to prevent collision or pinching of the actors.

[0197] After each warning event, a personnel warning report is automatically generated, which records the information related to the event in detail. The content of the report includes the location of the danger zone, actor number, event time, event description, protection mechanism status and processing results.

[0198] Emergency module, introduces dual power supply technology, sets redundancy mode, and formulates emergency response plan.

[0199] The main power supply provides regular power supply for stage equipment and is connected to the main power-consuming equipment on the stage (such as lighting, sound, robotic arms, lifting platforms, etc.). It is usually connected to the mains or a stable industrial power grid to ensure the normal operation of the equipment.

[0200] The backup power supply is independently powered and serves as an emergency power supply when the main power supply fails. The backup power supply can be a UPS (uninterruptible power supply), a diesel generator or a stage-specific energy storage device, which can immediately switch power supply when the main power supply is interrupted to prevent the stage equipment from stopping operation.

[0201] Install a power monitoring sensor between the main power supply and the backup power supply to detect parameters such as voltage, current, and frequency in real time.

[0202] When the main power supply is abnormal (such as low voltage, excessive current or power failure), the monitoring sensor will immediately send a signal to trigger the backup power supply to switch. The switching time should be in milliseconds to ensure seamless operation of stage equipment.

[0203] Each type of sensor is equipped with a set of redundant sensors to ensure reliable data even when a single sensor fails.

[0204] Redundant sensors are installed at different positions to avoid failures caused by single-point failures. For example, temperature sensors are installed on different sides of the power supply equipment respectively. Vibration sensors are installed at different support points of the mechanical equipment respectively.

[0205] Data from the sensors and redundant sensors are collected in real time and compared. If the data difference between the two sensors is within the allowable error range (such as 5%), the sensors are considered to be working properly. If the data of a certain sensor shows an obvious deviation (such as exceeding 10%), the sensor is considered to have failed.

[0206] When the main sensor fails is detected, it automatically switches to the redundant sensor.

[0207] When the user performs an operation, two completely identical instructions need to be sent. The two instructions are compared, and the operation will only be executed when the two instructions are completely identical.

[0208] For example, double-instruction confirmation is required when starting major equipment such as rotating stages and lifting platforms.

[0209] For operations involving stage safety or major equipment (such as equipment reset, emergency stop, etc.), approval from the management personnel is required.

[0210] When a fault is detected, an alarm is issued in various ways such as sound, light, and screen to alert the on-site staff. And the faulty equipment or area is automatically isolated to prevent the spread of the fault.

[0211] A fault notification is automatically sent to the maintenance personnel, including information such as the fault type, location, and time.

[0212] The detailed information of the fault is recorded, including the fault type, fault location, fault time, and fault status.

[0213] After the fault is repaired, a self-check is performed to confirm whether the equipment operation status is normal. After the self-check passes, a success signal is issued to confirm that the fault has been repaired and the equipment resumes normal operation.

[0214] In summary, the present invention: By real-time monitoring of the obstacle information on the stage and mapping this information onto a two-dimensional grid map. And predicting the future trajectory and selecting the optimal speed combination to ensure the safe operation of the stage equipment even in case of emergencies. The introduction of the orientation score, obstacle avoidance score, and speed score makes the path adjustment not only consider the physical distance but also take into account the motion state of the equipment and environmental changes, realizing a more intelligent and efficient obstacle avoidance strategy. In addition, by establishing a human body recognition model, the position of the actor can be accurately recognized in a complex stage environment, and the Gaussian weight is introduced to calculate the human body recognition probability, and judgment is made according to the set human body detection threshold, effectively reducing the probability of false detection and missed detection.

[0215] Example 2. Referring to Table 1, this is the second example of the present invention. To further verify the technical solution of the present invention, experimental simulation data of the safety adjustment control system for theater performance stage equipment is given.

[0216] To verify the effectiveness of the safety adjustment control system for theater performance stage equipment, an experiment was designed. The experiment was carried out in a simulated theater environment. The experimental site was set as a stage area of 20 meters × 15 meters, including a rotating stage, a lifting platform, and multiple fixed obstacles (such as lighting brackets, prop boxes) and dynamic obstacles (such as actors and mobile devices). The specific experimental steps are as follows:

[0217] Three-dimensional map construction and real-time monitoring.

[0218] Two lidar sensors (point cloud resolution of 0.2 meters) and four high-definition cameras (resolution 1920×1080) were deployed in the stage area. The stage point cloud data was obtained by lidar scanning. Combining with the camera image data, the stage environment was monitored in real time. The iterative closest point algorithm (ICP) was used to match adjacent point cloud frames to generate a global three-dimensional map, and the point cloud update frequency was 10 frames per second. The random sample consensus algorithm (RANSAC) was used to filter out the noise and outliers in the point cloud data.

[0219] Image data preprocessing and human detection model

[0220] The Gaussian filter was used to remove image noise, and the histogram equalization was used to enhance the image contrast. The processed image was input into the YOLO model to detect the dynamic human targets on the stage, and the positions of the actors were recognized in real time. The detection results were output in the form of the center point coordinates. The confidence threshold of the model was set to 0.8, and all detected targets were tracked and updated to the three-dimensional map in real time.

[0221] Path planning and obstacle avoidance.

[0222] According to the performance requirements, the positions of key equipment were marked in the three-dimensional map, including the lifting platform (target point 1) and the center area of the stage (target point 2). The three-dimensional map was discretized into a two-dimensional grid map with a grid resolution of 0.5 meters × 0.5 meters, and the passable area and the obstacle area were marked. The A* algorithm was used to calculate the optimal path from the current equipment position to the target point, and a dynamic obstacle detection module was set. When an obstacle was detected, the grid map was updated in real time and the path was re-planned. A dynamic obstacle detection module was set, and the maximum linear velocity and maximum angular velocity of the stage equipment were set, the speed sampling interval was defined, and the prediction time step and sampling period were set, as shown in Table 1.

[0223] Table 1 Stage equipment constraint table

[0224] Parameter Name Value Maximum Linear Velocity (v_max) 1.5 m / s Maximum Angular Velocity (ω_max) 1.0 rad / s Linear Velocity Sampling Interval 0.1 m / s Angular Velocity Sampling Interval 0.05 rad / s Prediction Time Step 2.0 s Sampling Period 0.1 s Target Distance Weight (w_goal) 0.6 Obstacle Distance Weight (w_obstacle) 0.3 Velocity Weight (w_speed) 0.1

[0225] Predict the future trajectory of the stage device based on each speed combination, and set the initial parameters of the stage device, as shown in Table 2 specifically.

[0226] Table 2 Initial Parameter Table of Stage Device

[0227] Parameter Name Value Unit Initial Abscissa (x) 0 m Initial Ordinate (y) 0 m Initial Orientation Angle (θ) 0 rad Linear Velocity (v) 1 m / s Angular Velocity (ω) 0.5 rad / s Time Step (Δt) 0.1 s Total Time (T) 2 s Total Number of Steps (N) 20 Step

[0228] The expression for predicting the future trajectory of the stage device based on each speed combination is:

[0229] ;

[0230] ;

[0231] ;

[0232] Taking 0 seconds to 0.2 seconds as an example, it specifically includes the following steps.

[0233] At time step 0, the abscissa of the stage device is 0, the ordinate is 0, and the orientation angle is 0.

[0234] The calculation process of the future trajectory of the ordinate from 0 seconds to 0.1 seconds is:

[0235] ;

[0236] ; ;

[0237] ;

[0238] The calculation process of the future trajectory of the abscissa from 0 seconds to 0.1 seconds is:

[0239] ;

[0240] ;

[0241] ;

[0242] ;

[0243] The calculation process of the future trajectory of the orientation angle from 0 seconds to 0.1 seconds is:

[0244] ;

[0245] ;

[0246] ;

[0247] At this time, the abscissa of the stage equipment is , the ordinate is , and the orientation angle is ;

[0248] The calculation process of the future trajectory of the ordinate from 0.1 second to 0.2 second is as follows:

[0249] ;

[0250] ;

[0251] ;

[0252] ;

[0253] ;

[0254] The calculation process of the future trajectory of the abscissa from 0.1 second to 0.2 second is as follows:

[0255] ;

[0256] ;

[0257] ;

[0258] ;

[0259] ;

[0260] The calculation process of the future trajectory of the orientation angle from 0.1 second to 0.2 second is as follows:

[0261] ;

[0262] ;

[0263] ;

[0264] ;

[0265] The following Table 3 shows the calculation results from 0.0 second to 0.5 second. Repeat the above steps for the calculation from 0.2 second to 2.0 seconds.

[0266] Table 3 Stage Equipment Prediction Table

[0267] Time t (s) Abscissa x (m) Ordinate y (m) Orientation Angle θ (rad) 0 0 0 0 0.1 0.1 0 0.05 0.2 0.1999 0.005 0.1 0.3 0.2997 0.015 0.15 0.4 0.3994 0.03 0.2 0.5 0.4988 0.05 0.25

[0268] Through the above steps and data tables, based on the maximum linear velocity, maximum angular velocity, velocity sampling interval, prediction time step, and sampling period of the stage device, the future trajectory of the stage device can be predicted.

[0269] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered within the scope of the claims of the present invention.

Claims

1. A theater performance stage equipment safety adjustment and control system, characterized in that: include: The map construction module monitors objects and people on the stage in real time, builds a three-dimensional map of the stage, and collects stage environment data and image data; A preprocessing module preprocesses the stage image data to obtain preprocessed stage image data; The path generation module generates the optimal path for stage equipment based on the three-dimensional map of the stage; The avoidance module detects obstacles on the stage in real time and replans the optimal path after detecting obstacles; The human body recognition module builds a human body detection model based on the pre-processed stage image data, identifies the positions of all actors, and tracks the positions of actors in real time; The safety assessment module sets safe and dangerous areas based on the three-dimensional map of the stage, and assesses whether the actors are in dangerous areas based on their positions; The human safety module will sound an alarm when an actor is in a dangerous area and generate a human early warning report; Emergency module, introduces dual power supply technology, sets redundancy mode, and formulates emergency response plan.

2. The theater performance stage equipment safety adjustment and control system according to claim 1, characterized in that: Real-time monitoring of objects and people on the stage, building a three-dimensional map of the stage, and collecting stage environment data and image data include the following steps: Deploy lidar and cameras on the stage, start the lidar to scan the stage, and obtain point cloud data about the stage; Shoot the stage area through a camera to obtain stage image data; Set the initial point as the reference position, and record the LiDAR point cloud data of the reference position and the image at the corresponding time; The LiDAR point cloud data between two adjacent frames are matched by iterative closest point algorithm to obtain the relative pose transformation matrix between the two frames. At the same time, the random sampling consistency algorithm is used to estimate the relative motion between the two frames. The point cloud data of each frame is converted to the global coordinate system according to the current pose transformation matrix, and accumulated into the global point cloud map; Use segmentation algorithms to classify point cloud data into different object categories; Apply multi-target tracking algorithms to track the position changes of dynamic objects and complete the construction of a three-dimensional map of the stage; The three-dimensional map of the stage includes the stage layout, obstacle positions and actor positions.

3. The theater performance stage equipment safety adjustment and control system as claimed in claim 2, characterized in that: Preprocessing the stage image data to obtain preprocessed stage image data specifically includes the following steps: Use a Gaussian filter to remove noise from the stage image data; Use histogram equalization to enhance the contrast of stage image data; Color correct stage image data using color space conversion and white balance adjustments.

4. The theater performance stage equipment safety adjustment and control system as claimed in claim 3, characterized in that: Generate the optimal path for stage equipment based on the three-dimensional map of the stage, which specifically includes the following steps: According to the performance requirements, mark the key locations where all stage equipment needs to reach on the 3D map; Convert the coordinates of the key positions of the markers from the image coordinate system to the global coordinate system; Discretize the three-dimensional map into a raster map; The discretized grid map is projected to obtain a two-dimensional grid map, and the corresponding grid represents a passable and impassable area. Define each grid as a path node and set the movement cost between each node; Set the starting and target points on the 2D grid map and define the open list and closed list; The open list records the nodes to be explored, and the closed list records the nodes that have been visited; Find the node with the smallest total cost in the open list and record it as the current node. If the current node is the target point, stop searching, generate a path, and move the current node to the closed list; Based on the current node and the target point, the heuristic function is defined using the Euclidean distance, and its expression is: ； in, represents the Euclidean distance value, Represents the current node, Indicates the horizontal coordinate of the current node. represents the horizontal coordinate of the target node, Indicates the vertical coordinate of the current node. Indicates the vertical coordinate of the target node; Accumulate the movement cost of the node to obtain the actual cost of the node movement; According to the heuristic function and the actual cost of node movement, the path cost function is defined to measure the total cost of the path, and its expression is: ； in, represents the path cost function value, Indicates the actual cost of node movement; Find all adjacent nodes of the current node, skipping the inaccessible nodes and the nodes that are already in the closed list; Calculate the actual cost from the current node to the adjacent node; If the adjacent node is not in the open list, add it to the open list and record the parent node of the adjacent node as the current node; If the adjacent node is in the open list and the new actual cost is smaller, update the actual cost and path cost function values; When the target point is selected as the current node, the search is completed; Starting from the target point, trace back along the parent node of each node until you return to the starting point; Arrange the nodes obtained by backtracking in reverse order to generate the optimal path from the starting point to the target point.

5. The theater performance stage equipment safety adjustment and control system according to claim 4, characterized in that: Detect obstacles on the stage in real time and re-plan the optimal path after detecting obstacles. The specific steps include the following: Use laser radar and cameras to detect obstacle information on the stage in real time and map the obstacle information to a two-dimensional grid map; Set the maximum linear speed and maximum angular speed of the stage equipment and define the speed sampling interval; Set the prediction time step and sampling period; For each speed combination, the future trajectory of the stage equipment is predicted, and its expression is: ； ； ； in, Indicates the stage equipment at time The horizontal axis of time, Indicates the stage equipment at time The horizontal axis of time, Indicates the line speed, represents the cosine value, Indicates the stage equipment at time The orientation angle at represents the time step, Indicates the stage equipment at time The vertical coordinate of time, Indicates the stage equipment at time The vertical coordinate of time, Indicates the stage equipment at time The orientation angle at Indicates the angular velocity of the device; Optimize the speed of stage equipment within dynamic windows; The orientation score is obtained by calculating the deviation angle between the current orientation of the stage equipment and the target orientation; Obstacle avoidance score is obtained by the minimum distance between the stage equipment trajectory and obstacles; Calculate the ratio of the current speed to the maximum speed to obtain a speed score; Based on the orientation score, obstacle avoidance score and speed score, the speed combination that maximizes the total score function is selected, and its expression is: ； in, Indicates the total score of the current speed combination. represents the orientation score, represents the weight of the orientation score, represents the obstacle avoidance score, represents the weight of obstacle avoidance score, Indicates the speed rating, represents the weight of the speed score; Based on the speed combination with the largest total score, the stage equipment position and orientation are updated, and the optimal path from the current position to the target position is generated.

6. The theater performance stage equipment safety adjustment and control system according to claim 5, characterized in that: According to the preprocessed stage image data, a human body detection model is established to identify the positions of all actors and track the positions of actors in real time. The specific steps include: Use the YOLO model to extract shallow feature maps and deep feature maps of the stage image, and use the detection head of the YOLO model to output the category score and grid unit bounding box parameters of each feature map coordinate point; Use activation functions to convert class scores into class probabilities; Perform nonlinear activation function processing on the feature map and weight the activated feature value by category probability; Accumulate the weighted features of the shallow feature map and the deep feature map to obtain multi-scale fusion features, and perform normalization; Apply the Sigmoid function to the normalized result; The center point of the grid unit bounding box parameters of the feature map is taken as the detection center point, and the Gaussian weight is calculated for the detection center point to obtain the calculation result of the Gaussian weight; Set human detection threshold according to stage performance requirements; The output of the Sigmoid function and the calculation result of the Gaussian weight are calculated to obtain the recognition probability of the human body, which is expressed as: ； in, Indicate point In time The probability of belonging to the human body is represents the Sigmoid function, represents the normalization factor, The scale of the feature, represents a nonlinear activation function, Represents the scale of the feature map At the point The value of Indicate point In scale and time The following is the probability of belonging to the category of human target, Indicate point In time The spatial weight of The recognition probability of the human body is compared with the human body detection threshold. If it is greater than the human body detection threshold, the point is determined to be a human target, and the position of the human body is tracked and captured in real time.

7. The theater performance stage equipment safety adjustment and control system according to claim 6, characterized in that: The safe area and dangerous area are set based on the three-dimensional map of the stage, and the actor's position is evaluated to see whether he is in the dangerous area. The specific steps include: The edge of the rotating stage, the edge of the lifting platform and the range of motion of the robotic arm are defined as dangerous areas; Define other areas outside the danger zone as safe areas and set safety thresholds based on the distance from the danger zone; The stage is projected onto a two-dimensional plane and divided into small cells, and the states of the corresponding cells are marked as dangerous areas and safe areas; When the distance between the current position of the human body and the danger zone is lower than the safety threshold, the actor is judged to be in the danger zone and a warning message is generated.

8. The theater performance stage equipment safety adjustment and control system according to claim 7, characterized in that: When an actor is in a dangerous area, an alarm is issued and a human early warning report is generated, which specifically includes the following steps: When an actor enters a dangerous area, an early warning message is immediately sent to the on-site staff, and the protection mechanism is triggered at the same time; A personnel warning report is generated, wherein the generated personnel warning report includes the location of the danger zone, the actor number and the time.

9. The theater performance stage equipment safety adjustment and control system according to claim 8, characterized in that: Introducing dual power supply technology, specifically including the following steps: The stage is equipped with a main power supply and a backup power supply, both of which are independently powered and connected to power monitoring sensors; Select redundant sensors for temperature sensors and vibration sensors and install the redundant sensors in different locations; The data of the sensor and the redundant sensor are compared. When the data of one sensor shows obvious deviation, the sensor is considered to be failed and the data of another sensor is switched.

10. The theater performance stage equipment safety adjustment and control system according to claim 9, characterized in that: Set up redundancy mode and develop an emergency response plan, which includes the following steps: When executing an operation, the user is required to send the same instruction twice. The operation will only be executed if the two instructions are exactly the same. For major operations, approval from management personnel is also required; When a fault is detected, an alarm will be triggered immediately, the fault location will be isolated, maintenance personnel will be notified, and the fault information will be automatically recorded; After the fault is repaired, a self-check is performed and a success signal is issued if no errors are found.

Images