A safety adjustment control system for a theatrical stage device

Abstract

The application discloses a kind of theatre stage equipment safety adjustment control systems, it is related to intelligent automation control system, including, map construction module, real-time monitoring object and personnel on stage, constructs the three-dimensional map of stage, collects stage environment data and image data, preprocessing module, stage image data is preprocessed, obtain the stage image data after preprocessing, path generation module, generates stage equipment optimal path according to the three-dimensional map of stage, avoid module, real-time detection barrier on stage, after detecting barrier, re-planning optimal path.The application can accurately identify the position of actor in complex stage environment by establishing human body recognition model, and introduces Gaussian weight to calculate human body recognition probability, judges according to set human body detection threshold, effectively reduces the probability of false detection and missed detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of intelligent automation control systems, and in particular to a theater performance stage equipment safety adjustment control system. BACKGROUND

[0002] With the development of theater performance art, the safety and intelligence level of stage equipment have gradually become the focus of the industry. Traditional theater performance stage equipment control mainly relies on manual operation and simple automation control. In recent years, with the introduction of advanced technologies such as computer vision, robotics, automatic control theory, and multi-sensor fusion, the control method of stage equipment has been significantly improved.

[0003] The prior art has obvious deficiencies in handling the safety adjustment of theater performance stage equipment. First, current path planning algorithms are mostly based on static environment assumptions. When temporary obstacles appear on the stage or the positions of actors change suddenly, the equipment cannot adjust the path in time, leading to potential safety hazards. In addition, traditional human detection models mostly use single-view two-dimensional image analysis, ignoring stereo spatial information, which not only limits the detection accuracy but also leads to false positives or missed detections. SUMMARY

[0004] In view of the above existing problems, the present application is proposed.

[0005] Therefore, the present application provides a theater performance stage equipment safety adjustment control system to solve the problem that path planning algorithms are mostly based on static environment assumptions, and when temporary obstacles appear on the stage or the positions of actors change suddenly, the equipment cannot adjust the path in time, leading to potential safety hazards.

[0006] To solve the above technical problems, the present application provides the following technical solutions:

[0007] The present application provides a theater performance stage equipment safety adjustment control system, which comprises,

[0008] a map construction module that monitors objects and personnel on the stage in real time, constructs a three-dimensional map of the stage, and collects stage environment data and image data;

[0009] a preprocessing module that preprocesses the stage image data to obtain preprocessed stage image data;

[0010] a path generation module that generates an optimal path for stage equipment according to the three-dimensional map of the stage;

[0011] an avoidance module that detects obstacles on the stage in real time and re-plans an optimal path after detecting an obstacle;

[0012] The human body recognition module establishes a human body detection model according to the pre-processed stage image data, recognizes the positions of all actors, and tracks the actor positions in real time;

[0013] The safety evaluation module sets a safety area and a dangerous area based on the three-dimensional map of the stage, and evaluates whether the actor is in the dangerous area according to the position of the actor;

[0014] The human body safety module issues an alarm when the actor is in the dangerous area, and generates a human body early warning report;

[0015] The emergency module introduces dual power supply technology, sets a redundancy mode, and formulates an emergency response plan.

[0016] Preferably, wherein: real-time monitoring of objects and personnel on the stage, constructing a three-dimensional map of the stage, collecting stage environment data and image data, specifically including the following steps,

[0017] Deploying a laser radar and a camera on the stage, starting the laser radar to scan the stage, and obtaining point cloud data about the stage;

[0018] Taking pictures of the stage area through the camera to obtain stage image data;

[0019] Setting an initial point as a reference position, recording the laser radar point cloud data of the reference position and the image at the corresponding time;

[0020] Performing iterative closest point algorithm matching on the laser radar point cloud data between adjacent two frames to obtain the relative pose transformation matrix between the two frames, and simultaneously using a random sample consensus algorithm to estimate the relative motion between the two frames;

[0021] Converting the point cloud data of each frame to the global coordinate system according to the current pose transformation matrix, and accumulating it into the global point cloud map;

[0022] Using a segmentation algorithm to divide the point cloud data into different object categories;

[0023] Applying a multi-target tracking algorithm to track the position changes of dynamic objects to complete the construction of the three-dimensional map of the stage;

[0024] The three-dimensional map of the stage includes stage layout, obstacle position and actor position.

[0025] Preferably, wherein: the stage image data is pre-processed to obtain pre-processed stage image data, specifically including the following steps,

[0026] Using a Gaussian filter to remove noise in the stage image data;

[0027] Using histogram equalization to enhance the contrast of the stage image data;

[0028] Color correction is performed on the stage image data using color space conversion and white balance adjustment.

[0029] Preferably, wherein: generating the optimal path of stage equipment according to the three-dimensional map of the stage, specifically comprising the following steps,

[0030] According to the performance requirements, mark all the key positions that the stage equipment needs to reach in the three-dimensional map;

[0031] Convert the coordinates of the marked key positions from the image coordinate system to the global coordinate system;

[0032] Discretize the three-dimensional map into a grid map;

[0033] Project the discretized grid map to obtain a two-dimensional grid map, and the corresponding grid represents a passable and impassable area,

[0034] Define each grid as a path node, and set the movement cost between each node;

[0035] Set the starting point and target point in the two-dimensional grid map, and define the open list and closed list;

[0036] The open list records the nodes to be explored, and the closed list records the nodes that have been visited;

[0037] Find the node with the minimum total cost in the open list, mark it as the current node, and stop searching if the current node is the target point. Generate the path and move the current node to the closed list;

[0038] Based on the current node and the target node, use the Euclidean distance to define the heuristic function, which is expressed as:

[0039] ;

[0040] Wherein, represents the Euclidean distance value, represents the current node, represents the horizontal coordinate of the current node, represents the horizontal coordinate of the target node, represents the vertical coordinate of the current node, represents the vertical coordinate of the target node;

[0041] Based on the movement cost of the node, the actual cost of the node movement is obtained;

[0042] According to the heuristic function and the actual cost of the node movement, define the path cost function to measure the total cost of the path, which is expressed as:

[0043] ;

[0044] wherein, represents a path cost function value, represents an actual cost of node movement;

[0045] find all adjacent nodes of the current node, skip the impassable nodes and the nodes already in the closed list in the adjacent nodes;

[0046] calculate the actual cost of the current node to the adjacent nodes;

[0047] if the adjacent node is not in the open list, add it to the open list and record the parent node of the adjacent node as the current node;

[0048] if the adjacent node is in the open list and the new actual cost is smaller, update the actual cost and the path cost function value;

[0049] when the target point is selected as the current node, the search is completed;

[0050] starting from the target point, trace back along the parent node of each node until returning to the starting point;

[0051] arrange the nodes obtained by backtracking in reverse order to generate the optimal path from the starting point to the target point.

[0052] Preferably, wherein: the obstacles on the stage are detected in real time, and after detecting the obstacles, the optimal path is re-planned, specifically comprising the following steps,

[0053] using a laser radar and a camera to detect the obstacle information of the stage in real time, and mapping the obstacle information to a two-dimensional grid map;

[0054] setting the maximum linear velocity and the maximum angular velocity of the stage device, and defining a velocity sampling interval;

[0055] setting a prediction time step and a sampling period;

[0056] for each velocity combination, predicting the future trajectory of the stage device, which is expressed as:

[0057] ;

[0058] ;

[0059] ;

[0060] wherein, represents the horizontal coordinate of the stage device at time , represents the horizontal coordinate of the stage device at time , represents the linear velocity, denotes a cosine value, denotes an orientation angle of the stage device at time denotes an orientation angle of the stage device at time denotes a time step, denotes a longitudinal coordinate of the stage device at time denotes a longitudinal coordinate of the stage device at time denotes a longitudinal coordinate of the stage device at time denotes a longitudinal coordinate of the stage device at time denotes an orientation angle of the stage device at time denotes an orientation angle of the stage device at time denotes an angular velocity of the device;

[0061] optimizing the speed of the stage device within a dynamic window;

[0062] calculating an orientation score through a deviation angle between a current orientation of the stage device and a target direction;

[0063] calculating an obstacle avoidance score through a minimum distance between a trajectory of the stage device and an obstacle;

[0064] calculating a speed score through a ratio of a current speed to a maximum speed;

[0065] selecting a speed combination maximizing a total score function based on the orientation score, the obstacle avoidance score and the speed score, and an expression thereof is:

[0066]

[0067] wherein, denotes a total score value of the current speed combination, denotes the orientation score, denotes a weight of the orientation score, denotes the obstacle avoidance score, denotes a weight of the obstacle avoidance score, denotes the speed score, denotes a weight of the speed score;

[0068] updating a position and an orientation of the stage device based on the speed combination maximizing the total score, and generating an optimal path from a current position to a target position.

[0069] Preferably, wherein: a human body detection model is established according to the preprocessed stage image data, the positions of all actors are identified, and the actor positions are tracked in real time, and specifically includes the following steps,

[0070] extracting a shallow feature map and a deep feature map of the stage image using a YOLO model, and outputting a class score and a grid cell bounding box parameter of each feature map coordinate point using a detection head of the YOLO model;

[0071] converting the class score into a class probability using an activation function;​

[0072] The feature map is processed by a non-linear activation function, and the activated feature values ​​are weighted by the class probability.

[0073] The weighted features of the shallow feature map and the deep feature map are accumulated to obtain the multi-scale fused features, and then normalized.

[0074] Apply the Sigmoid function to the normalized result;

[0075] The center point of the bounding box parameter of the grid cell in the feature map is taken as the detection center point, and the Gaussian weight is calculated for the detection center point to obtain the calculation result of the Gaussian weight.

[0076] Set human body detection thresholds according to the needs of stage performances;

[0077] The probability of human recognition is obtained by calculating the output of the Sigmoid function and the result of the Gaussian weights, and its expression is as follows:

[0078] ;

[0079] in, Point In time The probability that the following belongs to the human body. This represents the Sigmoid function. Represents the normalization factor. The scale representing the feature. Represents a non-linear activation function. Representing the scale of the feature map At point The value, Point In scale and time The probability of belonging to the category of human targets. Point In time Spatial weights;

[0080] The probability of recognizing a human body is compared with a human detection threshold. If the probability is greater than the human detection threshold, the point is determined to be a human target, and the position of the human body is tracked and captured in real time.

[0081] Preferably, the process involves: setting safe and dangerous zones based on a 3D map of the stage, and assessing whether an actor is in a dangerous zone based on their position, specifically including the following steps:

[0082] The edges of the rotating stage, the edges of the lifting platform, and the range of motion of the robotic arm are defined as hazardous areas.

[0083] Other areas outside the dangerous area are defined as safe areas, and a safety threshold is set based on the distance from the dangerous area;

[0084] After projecting the stage onto a two-dimensional plane, it is divided into small cells, and the state of the corresponding cell is marked as a dangerous area and a safe area;

[0085] When the distance between the current position of the human body and the dangerous area is less than the safety threshold, it is determined that the actor is in the dangerous area, and a warning information is generated.

[0086] Preferably, wherein: when the actor is in the dangerous area, an alarm is issued, and a human body warning report is generated, specifically including the following steps,

[0087] When the actor enters the dangerous area, an early warning information is immediately sent to the on-site staff, and a protection mechanism is triggered;

[0088] A personnel warning report is generated, which includes the location of the dangerous area, the actor number and the time.

[0089] Preferably, wherein: dual power supply technology is introduced, specifically including the following steps,

[0090] A main power supply and a backup power supply are configured for the stage, both of which are independently powered and connected to a power monitoring sensor;

[0091] Redundant sensors are selected for temperature sensors and vibration sensors, and the redundant sensors are installed at different positions;

[0092] The data of the sensor and the redundant sensor are compared, and when the data of one sensor deviates significantly, it is considered that the sensor is failed, and the data of the other sensor is switched.

[0093] Preferably, wherein: a redundancy mode is set, and an emergency response plan is developed, specifically including the following steps,

[0094] When performing an operation, the user is required to send the same instruction twice, and the two instructions must be completely consistent before the operation is actually performed;

[0095] For major operations, the approval of the management personnel is also required;

[0096] When a fault is detected, an alarm will be triggered immediately, the fault location will be isolated, the maintenance personnel will be notified, and the fault information will be automatically recorded;

[0097] After the fault is repaired, a self-check is performed, and a success signal is sent after no error is found.

[0098] The present application has the beneficial effects that: by monitoring the obstacle information on the stage in real time, and mapping these information to the two-dimensional grid map. And predict the future trajectory, select the best speed combination, even in the case of sudden, can guarantee the safe operation of the stage equipment. The introduction of the orientation score, obstacle avoidance score and speed score makes the path adjustment not only consider the physical distance, but also takes into account the motion state of the equipment and environmental changes, realizes a more intelligent and efficient obstacle avoidance strategy. In addition, by establishing a human body recognition model, the position of the actor can be accurately recognized in the complex stage environment, and the Gaussian weight is introduced to calculate the human body recognition probability, and the human body detection threshold is set to judge, effectively reducing the probability of false detection and missed detection. BRIEF DESCRIPTION OF DRAWINGS

[0099] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0100] Figure 1 The schematic diagram of the stage equipment safety adjustment control system in Example 1.

[0101] Figure 2 The schematic diagram of the optimal path generation in Example 1. DETAILED DESCRIPTION

[0102] In order to make the above-mentioned purposes, features and advantages of the present application more apparent and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the drawings of the specification.

[0103] Example 1, refer to Figure 1 and Figure 2 , the first embodiment of the present application, the embodiment provides a stage equipment safety adjustment control system for theater performance, comprising the following steps:

[0104] The map construction module monitors objects and personnel on the stage in real time, constructs a three-dimensional map of the stage, and collects stage environment data and image data.

[0105] Laser radar and camera are deployed in the stage area.

[0106] The laser radar is used to obtain accurate three-dimensional point cloud data and capture the spatial position information of the surface of the stage object. The laser radar has high precision and all-weather working ability, and can provide millimeter-level distance measurement.

[0107] The camera is used to obtain two-dimensional image data of the stage, and provides rich texture, color and visual features, which is convenient for subsequent object segmentation, classification and tracking.

[0108] The laser radar is activated to scan the stage area, generating three-dimensional point cloud data about the stage. These point cloud data contain the position and geometric shape information of objects on the stage, such as obstacles, props, actors, etc.

[0109] The stage area is photographed by the camera, generating high-resolution two-dimensional image data. These image data contain the texture, color, and other visual features of the stage surface, providing supplementary information for the point cloud data.

[0110] A certain position on the stage is selected as the reference point, and the point cloud data of the reference point and the corresponding image at that moment are recorded.

[0111] The relative motion between adjacent two frames is efficiently calculated through the Iterative Closest Point (ICP) algorithm and the Random Sample Consensus (RANSAC) algorithm.

[0112] Each frame of point cloud data is converted to the global coordinate system according to the pose transformation matrix of the current frame, forming a unified three-dimensional stage map. The point cloud data is divided into different object categories using the point cloud segmentation algorithm.

[0113] The data association is used to associate the trajectories of dynamic objects through the point cloud segmentation results and camera images, ensuring the continuity of tracking.

[0114] The final generated three-dimensional stage map contains the following information: stage layout, obstacle position, and actor position.

[0115] The stage layout refers to the global geometric structure of the stage.

[0116] The obstacle position refers to the precise position and distribution of obstacles on the stage, such as tables, chairs, and props.

[0117] The preprocessing module pre-processes the stage image data to obtain pre-processed stage image data.

[0118] Gaussian filtering is a classic image smoothing tool. The stage images obtained from the camera may contain noise (such as sensor noise, environmental light interference). The Gaussian kernel is applied to the image convolution operation, smoothing the random noise in the image while preserving the edge and structure information of the objects. It can effectively remove high-frequency noise in the stage image, such as random bright spots, color abnormalities, etc.

[0119] Histogram equalization is a technique for adjusting the gray scale distribution of an image by redistributing the gray scale values of the image pixels to make the brightness and contrast of the image more uniform. Count the number of pixels for each gray scale value in the image to obtain the gray scale histogram of the image, generate a cumulative distribution function using the histogram, and redistribute the gray scale values for each pixel to output a contrast-enhanced image. This step can make the objects on the stage more clearly visible, especially the details in the dark and highlight areas.

[0120] Color space is a mathematical model for representing colors in an image, and white balance is an important step in image color correction, aiming to eliminate the influence of light source color on the image and make the white area appear true white.

[0121] Specifically, the RGB image is converted to Lab color space, the image is adjusted for white balance according to the stage lighting conditions, and the color distortion caused by the light source is eliminated. The corrected image is converted back to the RGB space.

[0122] Further explanation, correct the color distortion caused by the stage lighting, make the colors in the image closer to the real scene

[0123] Path generation module, according to the three-dimensional map of the stage, generates the optimal path of the stage equipment.

[0124] Key position is a specific position that stage equipment (such as lighting devices, sound equipment, props, etc.) needs to reach. The marked key position is converted from the image coordinate system (two-dimensional pixel coordinates) to the global coordinate system of the three-dimensional map (real physical coordinates).

[0125] Further explanation, global positioning of the key position of the equipment is provided, ensuring that the equipment can accurately reach the specified position.

[0126] Discretize the three-dimensional map, divide the continuous three-dimensional point cloud into fixed-size voxels (3D grid cells). Each voxel represents a small cubic space.

[0127] Project the three-dimensional grid map to a two-dimensional plane to generate a two-dimensional grid map.

[0128] Each grid in the two-dimensional grid map represents a fixed-size planar region, which is divided into passable and impassable regions. The passable region is marked as 0 and the impassable region is marked as 1.

[0129] Further explanation, the complex three-dimensional map is simplified to a two-dimensional grid map, reducing the computational complexity.

[0130] Define each grid in the two-dimensional grid map as a path node, and define the movement cost from one grid to an adjacent grid, for example, the horizontal or vertical movement cost is 1. The diagonal movement cost is .

[0131] The Euclidean distance is used as the heuristic function to estimate the cost from the current node to the goal node, and its expression is:

[0132] ;

[0133] wherein, denotes the Euclidean distance value, denotes the current node, denotes the horizontal coordinate of the current node, denotes the horizontal coordinate of the goal node, denotes the vertical coordinate of the current node, denotes the vertical coordinate of the goal node;

[0134] The actual cost of node movement is obtained by accumulating the cost of node movement based on the cost of node movement

[0135] According to the heuristic function and the actual cost of node movement, a path cost function is defined to measure the total cost of the path, and its expression is:

[0136] ;

[0137] wherein, denotes the path cost function value, denotes the actual cost of node movement;

[0138] Specifically, the start point and the goal point are set on the grid map, and the open list and the closed list are defined.

[0139] The open list is used to record the nodes to be explored, and the closed list is used to record the nodes that have been visited.

[0140] The node with the smallest total cost is found from the open list, and it is set as the current node.

[0141] It is checked whether the current node is the goal point, if it is the goal point, the search is stopped, and the path is generated. If not, the current node is moved into the closed list.

[0142] All adjacent nodes of the current node are found, and the nodes that are not passable and the nodes that are already in the closed list are skipped.

[0143] For each adjacent node, the actual cost and the total path cost are calculated.

[0144] If the adjacent node is not in the open list, it is added to the open list, and its parent node is recorded as the current node. If the adjacent node is already in the open list, and the new actual cost is smaller, the cost is updated and the parent node is re-recorded. When the goal point is selected as the current node, the search is completed.

[0145] Further illustrate, based on the discretization of the grid map and heuristic search algorithm, the optimal path of the device from the starting point to the target point is quickly generated, and the path cost is guaranteed to be the lowest, while avoiding obstacles and impassable areas.

[0146] The avoidance module detects obstacles on the stage in real time and replans the optimal path after detecting obstacles.

[0147] According to the physical performance of the stage device, the maximum linear velocity and maximum angular velocity of the device are set. Ensure that the device moves within a safe range to avoid out of control caused by excessive speed. Within the dynamic window, the linear velocity and angular velocity are discretely sampled.

[0148] The total time range of the prediction time step is usually 1 to 2 seconds.

[0149] The sampling period is the time step of each prediction, which determines the time resolution of the trajectory.

[0150] For each velocity combination, the future trajectory of the stage device is predicted, which is expressed as:

[0151] ;

[0152] ;

[0153] ;

[0154] where, represents the horizontal coordinate of the stage device at time , represents the horizontal coordinate of the stage device at time , represents the linear velocity, represents the cosine value, represents the orientation angle of the stage device at time , represents the time step, represents the vertical coordinate of the stage device at time , represents the vertical coordinate of the stage device at time , represents the orientation angle of the stage device at time , represents the angular velocity of the device;

[0155] By predicting the trajectory of each velocity combination, the motion path of the stage device within a certain period of time in the future is obtained.

[0156] Optimize the speed of the stage device within the dynamic window, the specific steps are,

[0157] According to the deviation angle calculation of the current orientation of the device and the target direction, the smaller the deviation, the higher the score, which is expressed as:

[0158] ;

[0159] wherein, represents the orientation score, represents the current orientation angle of the device, represents the orientation angle of the target direction of the device;

[0160] The larger the minimum distance between the device trajectory and the obstacle, the higher the score. Its expression is:

[0161] ;

[0162] wherein, represents the obstacle avoidance score, represents the minimum distance between the trajectory and the nearest obstacle.

[0163] The ratio of the current speed to the maximum speed is used to encourage the device to move at a higher speed, and its expression is:

[0164] ;

[0165] wherein, represents the speed score, represents the maximum speed.

[0166] Based on the orientation score, the obstacle avoidance score and the speed score, the speed combination that maximizes the total score function is selected, and its expression is:

[0167] ;

[0168] wherein, represents the total score value of the current speed combination, represents the weight of the orientation score, represents the weight of the obstacle avoidance score, represents the weight of the speed score;

[0169] Based on the selected optimal speed combination, the current position and orientation of the stage device are updated. With the updated device position as the starting point, the optimal path to the target point is re-planned. If the target point has not changed, only the path needs to be adjusted locally to avoid newly appearing obstacles.

[0170] The human body recognition module establishes a human body detection model according to the preprocessed stage image data, identifies the positions of all actors, and tracks the actor positions in real time.

[0171] The shallow feature map is extracted by the first few layers of convolution in the YOLO model, mainly containing low-level features of the image such as texture, edge, color, etc. It is suitable for detecting small objects or targets with rich details (such as gestures on the stage or small props). The deep feature map is extracted by the last few layers of convolution, mainly containing high-level semantic features of the image such as overall shape, structure, and complex semantic information. It is suitable for detecting large objects or targets with clear semantics (such as human bodies or large props).

[0172] The detection head of the YOLO model analyzes each grid cell of different feature maps and outputs class scores and bounding box parameters.

[0173] The class parameter is the predicted score of different classes in each grid cell, indicating the possibility of the cell belonging to a certain class (such as human body, prop, etc.).

[0174] The bounding box parameter is the predicted bounding box parameter of each grid cell, which contains the coordinates of the center point of the bounding box.

[0175] Use the activation function to convert the class score output by the detection head into class probability. The class probability represents the normalized prediction value of the grid cell belonging to a certain class (such as human body), ranging from [0, 1].

[0176] Apply a nonlinear activation function to the feature map output by YOLO to activate the feature values and highlight key features.

[0177] Add the weighted features of shallow and deep feature maps to get the fused multi-scale features.

[0178] Normalize the accumulated multi-scale features to ensure that the feature values are within a reasonable range, avoiding instability caused by excessively large or small values. Apply the Sigmoid function to the normalized result to map the feature values to the [0, 1] range, further enhancing numerical stability.

[0179] Further explanation, multi-scale fusion improves the detection ability of different scale targets. The use of normalization and Sigmoid function ensures the uniformity of feature value range, avoiding the influence of numerical fluctuation on detection performance.

[0180] Take the center point coordinates of the bounding box parameter of the feature map grid cell as the detection center point. According to the detection center point, calculate the Gaussian weight. The Gaussian weight reflects the spatial influence range of the target point, and the weight value decreases with the increase of the distance from the center point.

[0181] Combine the output of the Sigmoid function and the Gaussian weight to calculate the point At time belongs to the human body, its expression is:

[0182] ;

[0183] where, represents the point at time belongs to the human body, represents the Sigmoid function, represents the normalization factor, represents the scale of the feature, represents the nonlinear activation function, represents the feature map scale at point , represents the point at feature map scale and time belongs to the class probability of the human target, represents the point at time .

[0184] According to the requirements of stage performance, set the human body detection threshold. For example, set the human body detection threshold to 0.7,

[0185] If is less than 0.7, it is ignored, and greater than 0.7 is determined as a human body. For the detected human target, track its position change in real time.

[0186] The safety evaluation module sets the safety area and the dangerous area based on the three-dimensional map of the stage, and evaluates whether the actor is in the dangerous area according to the position of the actor.

[0187] The dangerous area refers to an area that may pose a potential threat to the actor or other personnel during stage performance. The edge of the rotating stage, the edge of the lifting platform, and the movement range of the mechanical arm are defined as dangerous areas. The area outside a certain distance from the dangerous area is defined as a safe area.

[0188] The three-dimensional stage environment is projected onto a two-dimensional plane for calculation and analysis of the stage area.

[0189] The stage is divided into small cells with a fixed resolution.

[0190] The cells in the dangerous areas such as the edge of the rotating stage, the edge of the lifting platform, and the movement range of the mechanical arm are marked as dangerous areas.

[0191] The cells that do not belong to the dangerous area are marked as safe areas, but further calculation of the safety level is required according to their distance from the dangerous area.

[0192] The safety threshold refers to the minimum safe distance between the actor and the dangerous area.

[0193] Real-time detection of the current position of the actor and calculation of the distance of the actor from the nearest dangerous area cell. When the distance is less than the safety threshold, it is determined that the actor is in a dangerous area, and a warning message is generated.

[0194] Human safety module, when the actor is in a dangerous area, an alarm is sounded, and a human warning report is generated.

[0195] When the actor enters a dangerous area, an early warning message is sent to the on-site staff. The early warning message is a sound warning, such as "Actor approaching dangerous area, please be careful!" There is also a visual warning, such as a red warning light (e.g. LED indicator at the edge of the dangerous area) lit up in the stage area.

[0196] The protection mechanism is to immediately reduce the speed of the rotating stage or pause the rotation to prevent the actor from sliding or being pulled into the edge of the stage due to inertia. Stop the height change of the lifting platform to prevent the actor from being injured due to falling. Pause the movement of the mechanical arm to prevent collision or injury to the actor.

[0197] After each warning event, a personnel warning report is automatically generated, detailing information related to the event. The contents of the report include the location of the dangerous area, the actor number, the event time, the event description, the protection mechanism status and the handling results.

[0198] Emergency module, introduce dual power supply technology, set up redundancy mode, and develop emergency response plan.

[0199] The main power supply provides regular power supply for the stage equipment, and is connected to the main power consumption equipment of the stage (such as lighting, sound, mechanical arm, lifting platform, etc.). Usually connected to the city power or stable industrial power grid to ensure the normal operation of the equipment.

[0200] The backup power supply is independently powered and serves as an emergency power supply when the main power supply fails. The backup power supply can be a UPS (uninterruptible power supply), a diesel generator or a stage-specific energy storage device, which can immediately switch to power supply when the main power supply is interrupted to prevent the stage equipment from stopping running.

[0201] Install power monitoring sensors between the main power supply and the backup power supply to monitor voltage, current, frequency and other parameters in real time.

[0202] When the main power supply has an abnormality (such as low voltage, excessive current or power failure), the monitoring sensor will immediately send a signal to trigger the backup power supply to switch. The switching time should be in milliseconds to ensure seamless operation of the stage equipment.

[0203] Each type of sensor is equipped with a set of redundant sensors to ensure that reliable data can still be provided when a single sensor fails.

[0204] Redundant sensors are installed in different locations to avoid failure caused by single point failure. For example: temperature sensors are installed on different sides of the power supply equipment. Vibration sensors are installed on different support points of the mechanical equipment.

[0205] The data of the sensors and redundant sensors are collected in real time and compared. If the difference between the data of the two sensors is within the allowed error range (such as 5%), the sensor is considered to be working normally. If the data of a certain sensor deviates significantly (such as more than 10%), the sensor is considered to be failed.

[0206] When the main sensor is detected to be failed, the redundant sensor is automatically switched to.

[0207] When the user performs an operation, he needs to send two identical instructions. The two instructions are compared, and only when the two instructions are identical, the operation will be executed.

[0208] For example: when starting major equipment such as rotating stage and lifting platform, double instruction confirmation is required.

[0209] For operations involving stage safety or major equipment (such as equipment reset, emergency stop, etc.), the approval of the management personnel is required.

[0210] When a fault is detected, alarms are sent through sound, light, screen and other means to remind the on-site staff. And the faulty equipment or area is automatically isolated to prevent the fault from spreading.

[0211] Automatic fault notification is sent to maintenance personnel, including fault type, location, time, etc.

[0212] The detailed information of the fault is recorded, including fault type, fault location, fault time and fault state.

[0213] After the fault is repaired, self-checking is performed to confirm whether the equipment is running normally. After the self-checking is passed, a success signal is sent to confirm that the fault has been repaired and the equipment has returned to normal operation.

[0214] In summary, the present application realizes the safe operation of the stage equipment by: real-time monitoring of the obstacle information on the stage, and mapping these information onto a two-dimensional grid map. And predicting the future trajectory, selecting the best speed combination, even in the case of sudden situation, also can guarantee the safe operation of the stage equipment. The introduction of orientation score, obstacle avoidance score and speed score makes the path adjustment not only consider the physical distance, but also take into account the motion state of the equipment and environmental changes, realizes more intelligent and efficient obstacle avoidance strategy. In addition, by establishing a human body recognition model, the position of the actor in the complex stage environment can be accurately recognized, and the Gaussian weight is introduced to calculate the human body recognition probability, and the human body detection threshold is set to judge, effectively reducing the probability of false detection and missed detection.

[0215] Embodiment 2, referring to Table 1, is a second embodiment of the present application, which gives the experimental simulation data of the safety adjustment and control system of the theater performance stage equipment to further verify the technical solutions of the present application.

[0216] In order to verify the effectiveness of the safety adjustment and control system of the theater performance stage equipment, an experiment is designed, which is carried out in a simulated theater environment. The experimental site is set as a stage area of 20 meters x 15 meters, including a rotating stage, a lifting platform, and multiple fixed obstacles (such as light stands, prop boxes), and dynamic obstacles (such as actors and moving equipment). The specific experimental steps are as follows:

[0217] Three-dimensional map construction and real-time monitoring.

[0218] Two laser radars (point cloud resolution of 0.2 meters) and four high-definition cameras (resolution of 1920x1080) are deployed in the stage area. The stage point cloud data is obtained by laser radar scanning. Combined with the camera image data, the stage environment is monitored in real time. The Iterative Closest Point (ICP) algorithm is used to match adjacent point cloud frames to generate a global three-dimensional map, and the point cloud update frequency is 10 frames per second. The Random Sample Consensus (RANSAC) algorithm is used to filter out noise and abnormal points in the point cloud data.

[0219] Image data preprocessing and human body detection model

[0220] Gaussian filter is used to remove image noise, and histogram equalization is used to enhance image contrast. The processed image is input into the YOLO model to detect dynamic human targets on the stage, identify the actor's position in real time, and the detection result is output in the form of center point coordinates. The confidence threshold of the model is set to 0.8, and all detection targets are tracked and updated to the three-dimensional map in real time.

[0221] Path planning and obstacle avoidance.

[0222] According to the performance requirements, mark the key equipment positions in the three-dimensional map, including the lifting platform (target point 1) and the center area of the stage (target point 2). Discretize the three-dimensional map into a two-dimensional grid map with a grid resolution of 0.5 meters x 0.5 meters, mark the passable area and obstacle area. The A* algorithm is used to calculate the optimal path from the current equipment position to the target point, and the dynamic obstacle detection module is set. When obstacles are detected, the grid map is updated in real time and the path is re-planned. The maximum linear velocity and angular velocity of the stage equipment are set, the velocity sampling interval is defined, and the prediction time step and sampling period are set, as shown in Table 1.

[0223] Table 1 Stage equipment constraint table

[0224] Parameter Name Value Maximum linear velocity (v_max) 1.5 m / s Maximum angular velocity (ω_max) 1.0 rad / s Linear velocity sampling interval 0.1 m / s Angular velocity sampling interval 0.05 rad / s Prediction time step 2.0 seconds Sampling period 0.1 seconds Goal distance weight (w_goal) 0.6 Obstacle distance weight (w_obstacle) 0.3 Velocity weight (w_speed) 0.1

[0225] Set the initial parameters of the stage device based on the future trajectory of the stage device for each speed combination, as shown in Table 2.

[0226] Table 2 Stage device initial parameter table

[0227] Parameter Name Value Unit Initial x-coordinate (x) 0 m Initial y-coordinate (y) 0 m Initial orientation angle (θ) 0 rad Linear velocity (v) 1 m / s Angular velocity (ω) 0.5 rad / s Time step (Δt) 0.1 s Total time (T) 2 s Total steps (N) 20 Steps

[0228] The expression for predicting the future trajectory of the stage device based on each speed combination is:

[0229] ;

[0230] ;

[0231] ;

[0232] Take 0 to 0.2 seconds as an example, which specifically includes the following steps,

[0233] At time step 0, the stage device has a horizontal coordinate of 0, a vertical coordinate of 0, and an orientation angle of 0;

[0234] The calculation process of the future trajectory of the vertical coordinate from 0 to 0.1 seconds is:

[0235] ;

[0236] ; ;

[0237] ;

[0238] The calculation process of the future trajectory of the horizontal coordinate from 0 to 0.1 seconds is:

[0239] ;

[0240] ;

[0241] ;

[0242] ;

[0243] The calculation process of the future trajectory of the orientation angle from 0 to 0.1 seconds is:

[0244] ;

[0245] ;

[0246] ;

[0247] At this time, the stage device horizontal coordinate is , the vertical coordinate is , and the orientation angle is ;

[0248] The calculation process of the vertical coordinate future trajectory from 0.1 second to 0.2 second is as follows:

[0249] ;

[0250] ;

[0251] ;

[0252] ;

[0253] ;

[0254] The calculation process of the horizontal coordinate future trajectory from 0.1 second to 0.2 second is as follows:

[0255] ;

[0256] ;

[0257] ;

[0258] ;

[0259] ;

[0260] The calculation process of the orientation angle future trajectory from 0.1 second to 0.2 second is as follows:

[0261] ;

[0262] ;

[0263] ;

[0264] ;

[0265] The following Table 3 shows the calculation results from 0.0 second to 0.5 second, and the calculation from 0.2 to 2.0 second repeats the above steps.

[0266] Table 3 Stage device prediction table

[0267] Time t (s) X-coordinate x (m) Y-coordinate y (m) Orientation angle θ (rad) 0 0 0 0 0.1 0.1 0 0.05 0.2 0.1999 0.005 0.1 0.3 0.2997 0.015 0.15 0.4 0.3994 0.03 0.2 0.5 0.4988 0.05 0.25

[0268] Through the above steps and the data table, based on the maximum linear velocity, the maximum angular velocity, the velocity sampling interval, the prediction time step and the sampling period of the stage device, the future trajectory of the stage device can be predicted.

[0269] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application but not limit the present application. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or equivalently replaced without departing from the spirit and scope of the present application, and they should be covered in the scope of the claims of the present application.

Claims

1. A safety adjustment and control system for theater stage equipment, characterized in that: include, The map building module monitors objects and people on the stage in real time, builds a 3D map of the stage, and collects stage environment data and image data. The preprocessing module preprocesses the stage image data to obtain preprocessed stage image data. The path generation module generates the optimal path for stage equipment based on the 3D map of the stage. The obstacle avoidance module detects obstacles on the stage in real time and replans the optimal path after detecting an obstacle. Specifically, it includes the following steps: The system uses LiDAR and cameras to detect obstacles on the stage in real time and maps the obstacle information onto a two-dimensional grid map; it sets the maximum linear velocity and maximum angular velocity of the stage equipment and defines the velocity sampling interval. Set the prediction time step and sampling period; For each speed combination, the expression for predicting the future trajectory of the stage equipment is: ； ； ； in, Indicates the stage equipment in time x-axis of time Indicates the stage equipment in time x-axis of time Indicates linear velocity. Represents the cosine value. Indicates the stage equipment in time The angle of orientation at that time Indicates the time step. Indicates the stage equipment in time The vertical axis of time, Indicates the stage equipment in time The vertical axis of time, Indicates the stage equipment in time The angle of orientation at that time Indicates the angular velocity of the device; Optimize the speed of stage equipment within a dynamic window; An orientation score is calculated by measuring the deviation angle between the current orientation of the stage equipment and the target orientation. Obstacle avoidance score is obtained by measuring the minimum distance between the stage equipment trajectory and obstacles; The speed score is obtained by calculating the ratio of the current speed to the maximum speed. Based on orientation score, obstacle avoidance score, and speed score, the speed combination that maximizes the overall score function is selected, and its expression is: ； in, This represents the total score for the current speed combination. Indicates the direction of the rating. This indicates the weight of the score. Indicates obstacle avoidance score. This indicates the weight of the obstacle avoidance score. Indicates speed score, Indicates the weight of the speed score; Based on the speed combination with the highest total score, update the position and orientation of the stage equipment, and generate the optimal path from the current position to the target position; The human body recognition module establishes a human body detection model based on the preprocessed stage image data, identifies the position of all actors, and tracks the actors' positions in real time. Specifically, it includes the following steps: The YOLO model is used to extract shallow and deep feature maps of the stage image, and the detection head of the YOLO model is used to output the class score and grid cell bounding box parameters of each feature map coordinate point. Use an activation function to convert class scores into class probabilities; The feature map is processed by a non-linear activation function, and the activated feature values ​​are weighted by the class probability. The weighted features of the shallow feature map and the deep feature map are accumulated to obtain the multi-scale fused features, and then normalized. Apply the Sigmoid function to the normalized result; The center point of the bounding box parameter of the grid cell in the feature map is taken as the detection center point, and the Gaussian weight is calculated for the detection center point to obtain the calculation result of the Gaussian weight. Set human body detection thresholds according to the needs of stage performances; The probability of human recognition is obtained by calculating the output of the Sigmoid function and the result of the Gaussian weights, and its expression is as follows: ； in, Point In time The probability that the following belongs to the human body. This represents the Sigmoid function. Represents the normalization factor. The scale representing the feature. Represents a non-linear activation function. Representing the scale of the feature map At point The value, Point In scale and time The probability of belonging to the category of human targets. Point In time Spatial weights; The probability of recognizing a human body is compared with the human body detection threshold. If the probability is greater than the human body detection threshold, the point is determined to be a human target, and the position of the human body is tracked and captured in real time. The safety assessment module sets up safe and dangerous areas based on a 3D map of the stage, and assesses whether an actor is in a dangerous area based on their position. The human safety module issues an alarm and generates a human safety warning report when an actor is in a dangerous area. The emergency module incorporates dual power supply technology, sets up a redundancy mode, and develops an emergency response plan.

2. The safety adjustment and control system for theater stage equipment as described in claim 1, characterized in that: Real-time monitoring of objects and people on the stage, construction of a 3D map of the stage, and collection of stage environmental and image data include the following steps: Deploy LiDAR and cameras on the stage, activate the LiDAR to scan the stage, and obtain point cloud data about the stage; The stage area is captured by a camera to obtain stage image data; Set an initial point as a reference position, and record the lidar point cloud data and the corresponding image at the reference position; The lidar point cloud data between two adjacent frames is matched by an iterative nearest point algorithm to obtain the relative pose transformation matrix between the two frames. At the same time, the random sampling consensus algorithm is used to estimate the relative motion between the two frames. The point cloud data of each frame is transformed into the global coordinate system according to the current pose transformation matrix and accumulated into the global point cloud map; Segmentation algorithms are used to divide point cloud data into different object categories; Multi-object tracking algorithms are used to track the positional changes of dynamic objects and complete the construction of a 3D map of the stage; The three-dimensional map of the stage includes the stage layout, obstacle locations, and actor positions.

3. The safety adjustment and control system for theater performance stage equipment as described in claim 2, characterized in that: The stage image data is preprocessed to obtain preprocessed stage image data, which specifically includes the following steps. Use a Gaussian filter to remove noise from the stage image data; Histogram equalization is used to enhance the contrast of stage image data; Color correction was performed on the stage image data using color space conversion and white balance adjustment.

4. The safety adjustment and control system for theater performance stage equipment as described in claim 3, characterized in that: The process of generating the optimal path for stage equipment based on a 3D map of the stage includes the following steps: Based on the performance requirements, mark the key locations where all stage equipment needs to be located on the 3D map; Transform the coordinates of the key locations marked from the image coordinate system to the global coordinate system; Discretize the 3D map into a raster image; Projecting the discretized raster image yields a two-dimensional raster map, where each raster represents a passable or impassable area. Define each grid as a path node and set the movement cost between each node; Set the start and target points on the 2D raster map, and define the open and closed lists; The open list records nodes to be explored, and the closed list records nodes that have been visited. Find the node with the minimum total cost in the open list and mark it as the current node. If the current node is the target point, stop the search, generate a path, and move the current node to the closed list. Based on the current node and the target point, a heuristic function is defined using Euclidean distance, and its expression is: ； in, Represents the Euclidean distance value. Indicates the current node, Represents the x-coordinate of the current node. Represents the x-coordinate of the target node. This represents the y-coordinate of the current node. Represents the y-coordinate of the target node; The actual cost of moving a node is obtained by accumulating the movement costs of the nodes. Based on the heuristic function and the actual cost of node movement, a path cost function is defined to measure the total cost of a path, and its expression is: ； in, This represents the path cost function value. This represents the actual cost of moving a node; Find all neighboring nodes of the current node, and skip the impassable nodes and nodes that are already in the closed list among the neighboring nodes; Calculate the actual cost from the current node to its neighboring nodes; If an adjacent node is not in the open list, add it to the open list and record the parent node of the adjacent node as the current node; If the adjacent node is in the open list and the new actual cost is smaller, then update the actual cost and path cost function values; The search is complete when the target point is selected as the current node; Starting from the target point, backtrack along the parent node of each node until you return to the starting point; The nodes obtained from backtracking are arranged in reverse order to generate the optimal path from the starting point to the target point.

5. The safety adjustment and control system for theater stage equipment as described in claim 1, characterized in that: Based on a 3D map of the stage, safe and dangerous zones are defined. An assessment is made, based on the actor's position, to determine if they are in a dangerous zone. This process includes the following steps: The edges of the rotating stage, the edges of the lifting platform, and the range of motion of the robotic arm are defined as hazardous areas. Define areas other than the danger zone as safe zones, and set safety thresholds based on the distance from the danger zone; After projecting the stage onto a two-dimensional plane, it is divided into small cells, and the status of the corresponding cells is marked as dangerous area and safe area; If the distance between the current position of the human body and the danger zone is less than the safety threshold, the actor is determined to be in the danger zone, and an early warning message is generated.

6. The safety adjustment and control system for theater stage equipment as described in claim 5, characterized in that: When an actor is in a danger zone, an alarm is triggered, generating a human body warning report. This process includes the following steps: When an actor enters a dangerous area, an early warning message is immediately sent to the on-site staff, and the protection mechanism is triggered at the same time. Generate a personnel early warning report, which includes the location of the danger zone, actor number, and time.

7. The safety adjustment and control system for theater stage equipment as described in claim 6, characterized in that: The introduction of dual power supply technology includes the following steps. The stage is equipped with a main power supply and a backup power supply, both of which are powered independently and connected to power monitoring sensors; Select redundant sensors for temperature and vibration sensors, and install the redundant sensors in different locations; The data from the sensor and the redundant sensor are compared. If the data from one sensor shows a significant deviation, the sensor is considered to be faulty, and the data from another sensor is switched.

8. The safety adjustment and control system for theater stage equipment as described in claim 7, characterized in that: Setting up a redundancy mode and developing an emergency response plan includes the following steps: When performing an operation, the user is required to send the same command twice. The operation will only be executed if the two commands are completely identical. For major operations, approval from management is required; Upon detection of a fault, an alarm will be triggered immediately, the fault location will be isolated, maintenance personnel will be notified, and fault information will be automatically recorded. After the fault is repaired, a self-test is performed, and a success signal is sent if no errors are found.

Images