Multi-sensor fusion stage machinery intelligent control method and system

Abstract

The present application provides a multi-sensor fusion stage machinery intelligent control method and system, which relates to the field of mechanical control technology, including: determining the target sensor category and deploying a fusion sensor network in the target stage space; obtaining real-time stage tasks and making sensor acquisition decisions; activating the fusion sensor network for multi-dimensional data acquisition; combining the basic characteristics of the stage, performing correlation analysis of real-time multi-dimensional data sets, and performing data fusion; collecting the fused data into an intelligent decision maker, and mapping it to the corresponding decision function component for multi-channel control decision; and controlling the stage machinery of the target stage according to the multi-channel control decision results. The present application can solve the technical problem of inaccurate stage machinery control in the prior art due to the failure to accurately match the actual needs of the stage tasks. By obtaining real-time stage tasks, adapting to changes in the stage environment in real time, accurately matching the needs of the stage tasks, and improving the accuracy of stage machinery control.

Description

Technical Field

[0001] The present application relates to the field of mechanical control technology, and in particular to a multi-sensor fusion stage mechanical intelligent control method and system. Background Art

[0002] During stage performances, mechanical control is used to accurately operate and schedule the equipment in the stage environment to ensure the smooth progress of the performance. By coordinating the actions of various mechanical equipment, including lifts, stage lights, audio, stage backgrounds, props, etc., the required visual, auditory and other effects are provided for the performance. At present, stage machinery control methods are generally based on a predetermined performance plan, and stage equipment is controlled through fixed programs. The various tasks of stage performances are usually arranged in chronological order, and tasks are performed according to preset schedules and procedures. However, the tasks in stage performances are dynamically changing, and the requirements for equipment in different scenes and different time periods are different. Existing methods are usually unable to accurately adjust the equipment control strategy according to the actual needs of each specific task, resulting in lagging or inaccurate equipment control.

[0003] In summary, the prior art has a technical problem of inaccurate stage machinery control due to control based on conventional or preset task modes without real-time and dynamic adjustment of control strategies to adapt to the ever-changing actual needs on site. Summary of the invention

[0004] The purpose of this application is to provide a multi-sensor fusion stage machinery intelligent control method and system to solve the technical problem in the prior art that the stage machinery control is inaccurate due to the control according to conventional or preset task modes, without real-time and dynamic adjustment of the control strategy to adapt to the ever-changing actual needs on site.

[0005] In view of the above problems, the present application provides a multi-sensor fusion stage machinery intelligent control method and system.

[0006] In the first aspect, the present application provides a multi-sensor fusion stage machinery intelligent control method, which is implemented by a multi-sensor fusion stage machinery intelligent control system, wherein the multi-sensor fusion stage machinery intelligent control method includes: interacting with the target stage space, determining the target sensor category based on the basic stage characteristics of the target stage, and deploying a fusion sensor network in the target stage space; obtaining the task schedule of the target stage, extracting real-time stage tasks, and making sensor acquisition decisions in combination with the real-time stage tasks; based on the sensor acquisition decision results, activating the fusion sensor network to perform multi-dimensional data acquisition and obtain a real-time multi-dimensional data set; combining the basic characteristics of the stage, performing correlation analysis of the real-time multi-dimensional data set, and performing data fusion based on the correlation analysis results to generate a fusion data set; collecting the fusion data set to an intelligent decision maker, and mapping it to a corresponding decision function component for multi-channel control decision; and controlling the stage machinery of the target stage based on the multi-channel control decision results.

[0007] In the second aspect, the present application also provides a multi-sensor fusion stage machinery intelligent control system for executing the multi-sensor fusion stage machinery intelligent control method as described in the first aspect, wherein the multi-sensor fusion stage machinery intelligent control system includes: a sensor deployment module, the sensor deployment module is used to interact with the target stage space, determine the target sensor category based on the basic stage characteristics of the target stage, and deploy a fusion sensor network in the target stage space; a task extraction module, the task extraction module is used to obtain the task schedule of the target stage, extract real-time stage tasks, and make sensor acquisition decisions in combination with the real-time stage tasks; a data acquisition module The data acquisition module is used to activate the fusion sensor network to perform multi-dimensional data acquisition based on the sensor acquisition decision result to obtain a real-time multi-dimensional data set; the association analysis module is used to combine the basic characteristics of the stage, perform association analysis of the real-time multi-dimensional data set, and perform data fusion according to the association analysis result to generate a fusion data set; the control decision module is used to collect the fusion data set to the intelligent decision maker, and map it to the corresponding decision function component to perform multi-channel control decision; the mechanical control module is used to control the stage machinery of the target stage according to the multi-channel control decision result.

[0008] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0009] Through the interactive target stage space, based on the basic stage characteristics of the target stage, the target sensor category is determined, and a fusion sensor network is deployed in the target stage space; the task schedule of the target stage is obtained, the real-time stage tasks are extracted, and the sensor acquisition decision is made in combination with the real-time stage tasks; based on the sensor acquisition decision result, the fusion sensor network is activated to perform multi-dimensional data acquisition to obtain a real-time multi-dimensional data set; combined with the basic characteristics of the stage, the real-time multi-dimensional data set is subjected to correlation analysis, and data fusion is performed according to the correlation analysis result to generate a fusion data set; the fusion data set is sent to the intelligent decision maker, and mapped to the corresponding decision function component for multi-channel control decision; according to the multi-channel control decision result, the stage machinery of the target stage is controlled. In other words, sensors are deployed through the basic characteristics of the stage, data is collected according to the real-time stage tasks, and combined with the basic characteristics of the stage, the real-time multi-dimensional data set is subjected to correlation analysis and data fusion, and the stage machinery of the target stage is controlled, thereby improving the accuracy of the stage machinery control.

[0010] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented according to the contents of the specification, and in order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are specifically cited below. It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easy to understand through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the present application or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0012] Figure 1 This is a flow chart of the multi-sensor fusion stage machinery intelligent control method of this application;

[0013] Figure 2 This is a schematic diagram of the structure of the multi-sensor fusion stage machinery intelligent control system of this application.

[0014] Explanation of the reference numerals: sensor deployment module 11 , task extraction module 12 , data acquisition module 13 , correlation analysis module 14 , control decision module 15 , mechanical control module 16 . DETAILED DESCRIPTION

[0015] This application solves the technical problem of inaccurate stage machinery control in the prior art by providing a multi-sensor fusion stage machinery intelligent control method and system, which is caused by the control according to conventional or preset task modes without real-time and dynamic adjustment of control strategies to adapt to the changing actual needs of the scene. Sensors are arranged according to the basic characteristics of the stage, data is collected according to real-time stage tasks, and correlation analysis and data fusion are performed on real-time multidimensional data sets in combination with the basic characteristics of the stage, and stage machinery control is performed on the target stage, thereby improving the accuracy of stage machinery control.

[0016] Below, the technical solutions in the present application will be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments of the present application. It should be understood that the present application is not limited to the example embodiments described herein. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application. It should also be noted that, for the convenience of description, only the parts related to the present application are shown in the accompanying drawings, rather than all of them.

[0017] For example, please refer to the attached Figure 1 The present application provides a multi-sensor fusion stage machinery intelligent control method, wherein the multi-sensor fusion stage machinery intelligent control method is applied to a multi-sensor fusion stage machinery intelligent control system, and the multi-sensor fusion stage machinery intelligent control method specifically includes the following steps:

[0018] Step 1: Interact with the target stage space, determine the target sensor category based on the basic stage characteristics of the target stage, and deploy a fusion sensor network in the target stage space.

[0019] Specifically, the target stage space refers to the entire performance environment of a specific stage, including all physical space areas of the stage, not only the stage itself, but also the space areas used during the performance, such as backstage, actor passages, auditorium, technical areas, etc. The definition and use of the target stage space are different in different types of stages. Determine the type of target stage and define the specific layout and structure of the stage space. By identifying the type of stage (such as open stage, closed stage, unmanned stage, etc.), determine its possible monitoring needs. For example, an open stage needs to prevent the audience from entering restricted areas, while a closed stage needs to monitor the safe interaction between machinery and performers. As a stage for purely mechanical performances, the unmanned stage pays more attention to the prediction and control of mechanical abnormalities.

[0020] According to the type and layout of the stage, combined with the target control requirements, select the corresponding sensor type, including visual sensors, lidar, inertial measurement units (monitoring the posture and motion status of stage machinery), pressure sensors, ultrasonic sensors (detecting close-range obstacles and boundaries), audio sensors, posture sensors (real-time posture of stage machinery), etc. For example, an open stage needs to install boundary sensors or cameras to monitor audience behavior to ensure that there is no situation that exceeds safety limits; a closed stage needs to focus on monitoring the interaction safety between mechanical equipment and personnel inside the stage, and select position sensors, infrared sensors, etc.; unmanned stage: it is necessary to install a variety of sensors to monitor the status of mechanical equipment, such as load sensors, temperature sensors, vibration sensors, etc. Select the type and layout of sensors according to the basic characteristics of the stage to ensure that the sensors can fully monitor the key areas and equipment in the stage space.

[0021] After determining the category of the target sensor, a fusion sensor network will be deployed in the stage space. Sensors will be installed in key areas of the stage, such as near mechanical equipment, stage boundaries, and actor activity areas. The fusion sensor network is composed of multiple different types of sensors. Through data fusion technology, the data collected by different sensors are integrated to form a unified monitoring system, covering multiple sensor types, such as temperature sensors, humidity sensors, acceleration sensors, pressure sensors, etc., to achieve multi-dimensional data collection. Through sensor data fusion, the status of mechanical equipment, personnel, and other environmental changes can be monitored in real time, so as to achieve precise control and scheduling. According to the basic characteristics of the stage (such as open stage, closed stage, unmanned stage, etc.), the sensor category is accurately selected, and the fusion sensor network is deployed in the stage space to ensure a comprehensive monitoring range and real-time acquisition of key information.

[0022] Step 2: Obtain the task schedule of the target stage, extract the real-time stage tasks, and make sensor acquisition decisions based on the real-time stage tasks.

[0023] Specifically, the task schedule of the target stage is obtained through the performance arrangers, technical teams, etc., that is, the arrangement and time allocation of all activities on the stage (such as mechanical movements, actor performances, stage props movement, etc.), which stipulates the start and stop time of each step, task, and equipment of the performance, the specific content of execution, and the relevant priority. Real-time stage tasks are extracted from the task schedule, including the transformation of stage scenery, the control of lighting and sound, the movement of stage machinery, etc., reflecting the current and next performance tasks. According to the specific needs of each task, when, how, and which sensors are used for sensor collection are determined to ensure that relevant data is collected at the right time. Sensor collection decision refers to determining the sensors that need to be activated, when to activate the sensors, and the collection frequency according to the needs of real-time stage tasks and task schedules. By obtaining the task schedule of the target stage and extracting real-time stage tasks, and combining real-time tasks to make sensor collection decisions, key data can be collected at the right time to avoid data omissions.

[0024] Step 3: Based on the sensor acquisition decision result, activate the fusion sensor network to perform multi-dimensional data acquisition to obtain a real-time multi-dimensional data set.

[0025] Specifically, the sensor acquisition decision results include deciding when, from which locations, and which types of sensors to use. According to the sensor acquisition decision results, an activation command is sent to the sensor network to activate the corresponding sensors in the fusion sensor network for multi-dimensional data acquisition. After receiving the activation command, the sensor collects data from multiple dimensions or multiple sensors at a preset sampling rate at the same time, including temperature, humidity, mechanical displacement, acceleration and other information, to ensure that all sensors start collecting data at the same time or in a predetermined time sequence. A real-time multi-dimensional data set refers to a collection of various types of sensor data collected, including spatial position information, mechanical data, visual images, environmental parameters and other information. By activating the fusion sensor network for multi-dimensional data acquisition, comprehensive real-time stage status information can be obtained.

[0026] Step 4: Combine the basic characteristics of the stage, perform correlation analysis on the real-time multi-dimensional data set, and perform data fusion according to the correlation analysis results to generate a fused data set.

[0027] Specifically, the basic characteristics of the stage refer to basic information such as the structure, function and working environment of the stage, including the type of stage (e.g. open, closed, unmanned), the partitions of the stage (e.g. actor area, mechanical area, audience area), the types and functions of stage mechanical equipment, etc. The associated sensor set is determined based on the basic characteristics of the stage, and each sensor group corresponds to a control function in the stage control target. For example, for an open stage, the focus is on monitoring the status of the lifting platform, the movement trajectory of the mechanical equipment, and the monitoring data of the audience safety area.

[0028] According to the basic characteristics of the stage, the real-time multidimensional data set is analyzed for association, that is, the real-time multidimensional data set is matched with the basic characteristics of the stage to identify which sensor data are related to certain control targets or tasks of the stage. For example, temperature sensors are related to mechanical equipment temperature control tasks, and load sensors are related to mechanical load monitoring tasks. According to the different areas of the stage, the real-time multidimensional data set is streamlined, and the data that is irrelevant to the various areas of the stage is eliminated. The sensor data determined to be useful in the association analysis process is screened out to ensure that these data are closely related to the target task or target control function. After screening, there may be problems such as missing data and outliers, so data cleaning and preprocessing are required to ensure data quality.

[0029] Use adaptive hierarchical data fusion methods to integrate data, including information layer fusion, feature layer fusion, and decision layer fusion. Data fusion refers to combining data from different sources (such as data from different sensors or different time points) to generate a more complete, accurate, and meaningful data set. Data fusion often uses algorithms (such as weighted average, Kalman filtering, etc.) to process data to eliminate redundancy, improve accuracy, fill missing values, etc. Directly merge the original sensor data; extract key features from the sensor data and perform feature fusion of multi-sensor data; and provide decision support based on the fusion results of the information layer and feature layer.

[0030] The final fused data set is generated through data fusion, which contains multiple categories of data, such as human data, clearance neighborhood data, and mechanical status data. The human data includes personnel position monitoring data, which tracks the position of personnel on the stage in real time; the clearance neighborhood data includes possible obstacles, dangerous areas or other factors that affect mechanical movement on the stage; the mechanical status data includes the movement status, load information, position changes, etc. of mechanical equipment. Through association analysis and fusion algorithms, the deviations between different sensors are eliminated, and the data from multiple sensors are integrated to dynamically respond to changes in the stage environment.

[0031] Step 5: Collect the fused data into an intelligent decision maker and map it to corresponding decision function components to make multi-channel control decisions.

[0032] Specifically, the fused data set is input into the intelligent decision maker, which distributes the input data to different decision-making functional components for processing. The intelligent decision maker is a component based on artificial intelligence algorithms that integrates artificial intelligence algorithms, decision models, and data processing capabilities. It is responsible for receiving data input and making decisions based on preset rules or learned patterns. In stage machinery control, the intelligent decision maker makes decisions based on real-time data to guide the equipment to perform appropriate actions. The decision-making functional component is a specific module of the intelligent decision maker, which is responsible for specific control decision tasks and executes specific types of control decisions, such as mechanical motion control, obstacle avoidance control, etc., based on the different contents of the fused data set. In a stage machinery control system, there may be multiple decision-making functional components, such as intrusion detection components, avoidance decision components, etc. Each component makes different decisions based on different sensor data.

[0033] The intelligent decision maker maps the received fusion data to different decision function components for processing according to predefined rules. For example, if the data involves the mechanical state, it will be mapped to the mechanical control decision component. Each decision function component executes the corresponding control decision model according to its task. For example, the intrusion discrimination component may use location-based data to make a decision on whether to issue a safety warning. The intelligent decision maker will start multiple decision function components, and each component will independently process its corresponding data type. Multi-channel control decisions achieve synchronization and collaboration of multiple devices by coordinating various control channels to ensure that various control decisions during stage performances can be effectively executed. In the multi-channel control decision process, the intelligent decision maker will integrate the outputs of various decision function components for global optimization.

[0034] Multi-channel control decisions include intrusion decision results, avoidance decision results and state management decision results. The intrusion decision results are obtained by analyzing the human data class according to the target detection method to determine whether non-stage personnel have entered the operating area of ​​the stage machinery to ensure the safety of the stage personnel and the audience. The avoidance decision results are to ensure the safe distance between the stage machinery when there is a risk of interference or collision between multiple stage machinery (such as robotic arms, lifting platforms, etc.) to avoid conflicts and unnecessary damage. The state management decision results are based on the real-time monitoring of the state of the stage machinery (such as temperature, vibration, strain, etc.), combined with the fault prediction model and safety standards, to ensure the safe operation of the stage machinery, and through real-time monitoring and fault prediction, to ensure that the mechanical equipment will not fail or overload. By transmitting the fused data set to the intelligent decision maker and mapping it to the corresponding decision function component, a multi-channel control decision is finally generated. Each decision function component makes a decision based on its specialized field, thereby improving the accuracy of control.

[0035] Step 6: Based on the multi-channel control decision results, the stage machinery of the target stage is controlled.

[0036] Specifically, the decision results from different control channels are integrated to form a unified control strategy, and the multi-channel control decision results are obtained to guide the various mechanical equipment on the target stage to perform corresponding tasks. In other words, the control decision results are obtained from various decision-making functional components (such as mechanical control, avoidance decision, fault prediction, etc.). These decision results usually include the motion trajectory, posture adjustment, and execution of the machine. All control decision results are integrated to ensure that there is no conflict between the various control channels and form a global optimal control solution. According to the multi-channel control decision results, the control tasks of each mechanical equipment on the target stage are specifically executed. According to the specific task requirements of each device and the multi-channel control decision, the position, motion trajectory, speed, posture, etc. of the stage machinery are accurately adjusted. The control objectives and parameters of each machine will be dynamically adjusted according to real-time environmental changes and performance requirements. In addition, for complex mechanical collaboration, coordination and synchronization are also required to ensure that the actions of multiple mechanical equipment do not conflict.

[0037] Finally, the control commands of all mechanical equipment are executed to realize the actual control operation on the stage. According to the decision results, each mechanical equipment on the stage is directly controlled to ensure that each mechanical equipment works according to the predetermined action, time and position. Each mechanical equipment (such as lifting platform, robotic arm, lighting equipment, etc.) is precisely adjusted according to its specific control target. For complex tasks, it is necessary to consider the interaction and collaboration between various devices, such as ensuring that there is no collision or occlusion between different robotic arms.

[0038] In the process of executing control commands, the status of mechanical equipment is monitored in real time and adjusted dynamically to ensure that all tasks are completed smoothly as expected. The feedback data of the equipment (such as position, speed, posture, etc.) is continuously obtained from the sensor, and the status of the equipment is analyzed in real time. If an abnormality is detected (such as collision, equipment failure, error, etc.), the control strategy is dynamically adjusted, and the control command is recalculated and executed. Through multi-channel control decision results, the actions and tasks of multiple stage machinery can be accurately controlled, and a high degree of coordination and synchronization between machinery can be achieved, avoiding interference and conflict between equipment.

[0039] Furthermore, step one of this application includes:

[0040] Interact with the target stage space, obtain the stage type information, stage partition information, and stage machinery information of the target stage, and extract them as the basic features of the stage; combine the basic features of the stage with the control objectives of the target stage, perform spatial division in the target stage space, and obtain a multi-level monitoring area, wherein the multi-level monitoring area at least includes an access monitoring area, a buffer monitoring area, a clearance monitoring area, and a machinery monitoring area; traverse the multi-level monitoring area, determine the corresponding target sensor category in combination with the control objective, and deploy sensors according to the target sensor category.

[0041] Specifically, the target stage space is interacted with to collect the stage type information, stage partition information and stage machinery information of the target stage, which constitutes the basic characteristics of the stage. Stage type information refers to the characteristics of the stage type, structural form, function and use environment, such as fixed, adjustable, rotating, etc. Different types of stages require different mechanical facilities and control methods. Stage partition information refers to dividing the stage space into different areas so that different areas can be controlled and managed differently, including performance areas, audience areas, technical areas, backstage, etc., and the control requirements and priorities of each area are different. Stage machinery information includes all information related to the mechanical equipment on the stage (such as lifting platforms, lighting hangers, curtain lifting devices, etc.), such as location, status, operation requirements, etc. For example, the mechanical equipment control requirements of a fixed stage are different from those of a rotating stage, so the stage type information will help determine the control strategy. At the same time, the stage partition information determines the control requirements of mechanical equipment in different areas, and the mechanical information helps identify which equipment on the stage needs to be controlled. The basic characteristics of the stage refer to the comprehensive feature data extracted by obtaining the stage type, area information, mechanical information, etc., which can accurately describe the structure and working mode of the target stage.

[0042] The control target of the target stage refers to the control effect that the stage machinery needs to achieve, such as precise position control, speed control, safety monitoring, etc. According to the basic characteristics and control objectives of the stage, the target stage space is divided into multiple monitoring areas, including at least the access monitoring area, buffer monitoring area, clearance monitoring area and mechanical monitoring area. Among them, the access monitoring area is responsible for monitoring and controlling the personnel and equipment entering the stage or operation area; the buffer monitoring area is used to monitor the transition part between the stage area and the external area to ensure that the stage equipment does not interfere with other areas during operation; the clearance monitoring area is used to monitor the clearance range of the stage machinery during operation to ensure safety and accessibility; the mechanical monitoring area is used to monitor the status of the stage machinery and equipment in real time, such as position, movement, operation efficiency, etc., to ensure that the equipment operates as expected. For example, an indoor stage may need to be divided into an access monitoring area for the performance area, a buffer monitoring area for the backstage, a clearance monitoring area for the boom above the stage, and a mechanical monitoring area for the boom itself.

[0043] Traverse the multi-level monitoring areas and determine the different types of sensors required for each area according to the control objectives. Different monitoring areas and control objectives require different types of sensors, such as position sensors, temperature sensors, pressure sensors, etc. According to the target sensor category of each area, the sensors are deployed in the area and the sensors are reasonably installed at specific locations in the target stage space. For example, the clearance monitoring area may be equipped with laser sensors to detect the range of motion of stage mechanical equipment, and the mechanical monitoring area may be equipped with position sensors to provide real-time feedback on the equipment status. By accurately dividing the stage space, it is ensured that the tasks and equipment in each monitoring area can be properly monitored, thereby improving the accuracy and safety of stage mechanical control.

[0044] Further, step 4 of this application includes:

[0045] Collect mechanical control records, and take the control target as the extraction target, statistically analyze the mechanical control records, and obtain the corresponding associated sensor set, wherein each associated sensor group in the associated sensor set corresponds to a control function in the control target; perform associated partitioning of the real-time multidimensional data set based on the associated sensor set as the partition constraint to obtain an initial associated data set; perform data simplification of the initial associated data set based on the multi-level monitoring area as the retention constraint, and output a standard associated data set; traverse the standard associated data set to perform adaptive hierarchical data fusion on each standard associated group, and output the fusion result as the fused data set, wherein the fused data set includes a human data class, a clearance neighborhood data class, and a mechanical state data class.

[0046] Specifically, mechanical control records are collected from the control systems of stage mechanical equipment, including the start time, stop time, execution status, etc. of each mechanical equipment. Mechanical control records refer to the control logs of mechanical equipment on the stage over a certain period of time, which record the start, stop, adjustment, state change, etc. of the equipment, including timestamps, control instructions, execution status, mechanical response, etc. The control target is taken as the extraction target, that is, the focus is on the mechanical control records related to the control target.

[0047] Perform statistical analysis on mechanical control records to identify sensor data patterns related to the control target and form an associated sensor set. For example, for a task to control the temperature of a mechanical device, the associated sensor set may include a temperature sensor; for a task to control the load of the device, the associated sensor set may include a pressure sensor and a load sensor. An associated sensor set is a set of sensors related to a control target and is used to collect data related to the control target to support the execution and monitoring of the task. Each sensor group corresponds to a control function in the control target.

[0048] According to the associated sensor set, the real-time multidimensional data set is associated and divided, and the data is classified according to the control target and the related sensors to ensure that each data point is associated with the corresponding control target. The real-time multidimensional data set is divided into several parts, each of which contains data related to a specific control function. Association partitioning refers to classifying and organizing the real-time multidimensional data set according to the relationship of the sensor set to ensure that the data related to the specific control target can be effectively identified. It is usually completed using data grouping and clustering algorithms, such as K-means clustering algorithm or decision tree algorithm, which automatically divides the data according to the relationship between sensor type and control target.

[0049] Taking the multi-level monitoring area as a retention constraint, the initial associated data set is simplified, that is, the data irrelevant to the multi-level monitoring area is eliminated, and the standard associated data set is output, which helps to reduce the complexity of data processing and ensure that only the most meaningful data for the control target is retained. The standard associated data set is a streamlined and optimized data set that meets the constraint requirements of the multi-level monitoring area, removes redundant data and improves data quality.

[0050] Traverse the standard association data set, perform adaptive hierarchical data fusion on each standard association group, perform layer-by-layer fusion of the information layer, feature layer and decision layer, and finally obtain the fused data set. Information layer fusion is to synchronize, filter and reduce noise of data from multiple sensors to ensure data quality; feature layer fusion is to extract key features (such as position change rate, load change, etc.) from each data source for comprehensive analysis; decision layer fusion is to make decisions based on the results of feature layer fusion, such as adjusting mechanical operation instructions. The fused data set refers to the data set processed by adaptive hierarchical data fusion, which contains integrated multi-dimensional information. Adaptive hierarchical data fusion is a multi-level data fusion method that can effectively integrate heterogeneous data from different sensors to improve data accuracy and availability.

[0051] The fused data set is output, which contains multiple data categories (such as human behavior, clearance area, and mechanical status). The human data category refers to the monitored personnel location information; the clearance neighborhood data category is the obstacle location and clearance status in the stage performance area; the mechanical status data category is the real-time location, load status, and motion trajectory of the mechanical equipment. Through adaptive hierarchical data fusion, the noise and error in the sensor data can be eliminated, the accuracy and reliability of the data can be improved, the amount of data processing can be reduced, and the processing efficiency can be improved.

[0052] Further, step five of this application includes:

[0053] Extract the person in the data class from the fused data set, and transmit the person in the data class to the limit intrusion decision component for limit intrusion discrimination, and generate a limit intrusion decision result based on the limit intrusion discrimination result; extract the clearance neighborhood data class and the mechanical state data class from the fused data set, and transmit them to the mechanical avoidance decision component for avoidance decision, and generate an avoidance decision result; extract the mechanical state data class from the fused data set, use the mechanical state data class as the safety state prediction input, perform fault risk prediction through a pre-built mechanical fault analysis model component, and perform a state management decision based on the fault risk prediction result to obtain a state management decision result; integrate the limit intrusion decision result, the avoidance decision result and the state management decision result, and output them as the multi-channel control decision result.

[0054] Specifically, the human data class is extracted from the fusion data set, including infrared sensor data, image sensor data, millimeter wave radar data, etc. The human data class refers to the data related to personnel obtained by sensors, usually including infrared sensors, image sensors, millimeter wave radars, etc., to identify the location, dynamics and behavior of personnel and monitor personnel. The human data class is transmitted to the limit intrusion decision component for limit intrusion judgment. If there is a limit intrusion (that is, the person enters the dangerous area), the corresponding safety measures (such as alarm, avoidance, emergency stop, etc.) will be triggered. The limit intrusion decision component is a decision-making component used to determine whether the audience or staff enters the dangerous area of ​​the stage. According to the location and identity of the person, it is compared with the preset safety boundary. If it is detected that the person has entered the restricted area or dangerous area, the limit intrusion judgment is triggered. According to the specific limit intrusion judgment result, the corresponding decision result is generated, such as issuing an alarm, starting an avoidance program or performing an emergency stop operation.

[0055] The clearance neighborhood data class and the mechanical state data class are extracted from the fused data set. The clearance neighborhood data class includes information about obstacles around the machine, while the mechanical state data class provides the machine's operating state data (such as posture, temperature, vibration, etc.). The clearance neighborhood data class and the mechanical state data class are transmitted to the mechanical avoidance decision component to determine whether the machine needs to avoid obstacles or adjust its motion path. The spatial data around the machine is obtained using millimeter-wave radar, laser radar, image sensor, proximity sensor, etc., including the dynamic position of obstacles, personnel and machinery. Real-time data about the machine's motion state, posture, load, temperature, etc. are extracted from the mechanical state sensor. Based on the clearance neighborhood data class and the mechanical state data class, avoidance decisions are made to determine whether the mechanical equipment's motion path needs to be adjusted to avoid obstacles or personnel. According to the avoidance decision results, corresponding control instructions are generated to enable the machine to avoid obstacles and ensure safe operation. For example, if an obstacle is detected in front of the machine (such as an actor mistakenly entering the stage area) through the millimeter-wave radar, combined with the current state data of the machine (such as rotation angle and speed), the mechanical avoidance decision component will make an avoidance decision to avoid collision between the machine and the obstacle.

[0056] Extract the mechanical state data class from the fused data set, including the mechanical posture, position, speed, temperature, vibration, pressure and other data. Use the mechanical state data class as the input for safety state prediction to predict fault risks. The mechanical fault analysis model component evaluates whether the equipment is currently in normal operation based on the mechanical state data (such as temperature, vibration, pressure, etc.) and predicts potential fault risks. Collect and prepare historical data of mechanical equipment, and standardize or normalize the data so that data of different dimensions can be processed in the same model. Extract features that are helpful for fault diagnosis and prediction from the collected data, and convert the original data into feature vectors that can effectively reflect the operating status of the equipment. Select a suitable algorithm based on the feature data, such as a classification model (support vector machine, decision tree, random forest) or a regression model (linear regression, neural network).

[0057] Use historical data to train the fault analysis model and evaluate the performance of the model through methods such as cross-validation. Use the validation data set or test data set to evaluate the model and calculate evaluation indicators such as prediction accuracy, recall rate, and F1 value. If the model's prediction effect is not good, it can be optimized by adjusting parameters, selecting features, or replacing the model. Use the trained mechanical fault analysis model to predict the real-time data of the equipment and generate fault risk assessment and management decisions.

[0058] Input the mechanical state data class into the mechanical fault analysis model component to predict the fault risk and obtain the fault risk prediction result. Generate corresponding state management decisions based on the fault risk prediction result, such as whether to shut down for maintenance, reduce equipment load, adjust equipment operation status, etc. If the fault risk is high, shut down immediately for troubleshooting and maintenance; if the risk is low, continue to operate normally. Determine the severity of the fault based on the status of the equipment and output the corresponding treatment plan (such as repair, replacement of parts, etc.).

[0059] Multi-channel control decision refers to the generation of decision plans for different control objectives based on the comprehensive results of multiple decisions, taking into account various data inputs, and generating a comprehensive control plan after fusion. There are multiple control objectives involved in stage control, such as personnel intrusion judgment, mechanical avoidance, equipment failure risk prediction, etc. By integrating different decision results and generating a complete control decision, it is possible to avoid conflicts between personnel and machinery and prevent mechanical failures. Different decision results (intrusion decision results, avoidance decision results and state management decision results) are integrated, and the final control decision is generated through priority scheduling or weighted decision. According to the fusion results of various decisions, multi-channel control instructions are output and control measures are implemented. Decision fusion is performed according to the priority or weight of each decision, and the decision results are output to the actuator to ensure safe control. Through multi-sensor fusion and intelligent decision-making systems, the safety of the stage environment is ensured to avoid potential dangers to personnel and machinery.

[0060] Furthermore, the present application also includes the following steps:

[0061] Based on the target detection method, the person is identified in the data class to determine the target identity information, target location information and target dynamic information; the target location information is used as the first discrimination factor to determine the limit intrusion status, if the target location information is in the access monitoring area, the buffer monitoring area or the clearance monitoring area, the limit intrusion status is yes; the target identity information is used as the second discrimination factor to determine the limit intrusion category, and generate the limit intrusion category, wherein the limit intrusion category includes the limit intrusion of non-access objects and the limit intrusion of access objects; according to the limit intrusion status and the limit intrusion category, the corresponding limit intrusion control plan is called as the limit intrusion decision result.

[0062] Specifically, the target detection algorithm is used to identify the target of people in the data class, identify the target objects (such as actors, staff, audience, etc.), and extract identity information, location information, and dynamic information. The target detection method is a computer vision technology that aims to identify specific objects in images or videos and assign identification information (such as location, category, etc.) to these objects. Common target detection methods include deep learning-based convolutional neural networks, regional convolutional neural networks, etc. For example, assuming that the image stream of the stage is obtained through a camera, the target detection algorithm can identify the person in the image and extract the person's identity (such as an actor), location (such as the coordinates on the left side of the stage), and dynamics (such as walking towards mechanical equipment).

[0063] Determine whether the target has entered the preset monitoring area based on the target location information, and compare the target location information with the preset monitoring area to determine whether the target has entered the access monitoring area, buffer monitoring area or clearance monitoring area. If the target location information is in the access monitoring area, buffer monitoring area or clearance monitoring area, the intrusion status is determined to be yes, which means there is an intrusion risk. Determine whether the target is in the safe area through a simple coordinate comparison method.

[0064] The access monitoring area refers to the area where the target is allowed to enter, usually referring to the safe area outside the working area of ​​the machinery or equipment on the stage. For example, actors can freely enter the performance area, but cannot enter the mechanical operation area to avoid safety accidents. The buffer monitoring area refers to the area between the access monitoring area and the clearance monitoring area, which is usually set up to provide an additional safety buffer zone. For example, people can be allowed to enter this area, but they must be careful to avoid getting too close to the dangerous area. The clearance monitoring area refers to any potential dangerous area on the stage, such as the mechanical movement area, equipment lifting area, etc., which is prohibited for non-staff or actors to enter. For example, the working area of ​​the lifting platform or the movement range of the robotic arm, any person approaching these areas needs to be warned or trigger risk avoidance measures.

[0065] According to the target identity information, as the second discrimination factor, the intrusion behavior is classified, mainly into (such as actors, staff) or non-access objects (such as audiences, outsiders). Some rules are preset to determine which targets are access objects and which are non-access objects, and based on the target identity information, the intrusion category is generated (such as non-access object intrusion and access object intrusion). According to the intrusion status and category, the corresponding control measures are selected, such as alarm, avoidance, emergency stop, etc. For example, for access object intrusion, a warning reminder is sent, or an avoidance operation is triggered; for non-access object intrusion, the emergency stop of the mechanical equipment may be directly triggered. Through accurate target recognition and intrusion discrimination, the target's safety intrusion problem is judged and handled in real time, and the handling method of different targets is intelligently determined to avoid false alarms and unnecessary downtime.

[0066] Furthermore, the present application also includes the following steps:

[0067] If there are multiple stage machines in the target stage space, and the clearance monitoring areas of the multiple stage machines overlap, then the real-time stage task sets of the multiple stage machines are obtained; the real-time stage task sets are parsed, the control instructions of the multiple stage machines are extracted, and the timing trajectory sets of the multiple stage machines are constructed according to the control instructions; interference analysis is performed on the timing trajectory set, and dynamic avoidance decisions of the multiple stage machines are made according to the interference analysis results.

[0068] Specifically, if there are multiple stage machinery in the target stage space, that is, there are multiple mechanical equipment for different performance purposes, such as lifting platforms, revolving stages, robotic arms, etc., these equipment work together on the stage and collaborate to perform tasks. Get the current task set of all stage machinery to understand the tasks and schedules of each machine. When the clearance monitoring areas of two or more stage machinery overlap in space, there is a potential risk of collision. The overlap of clearance monitoring areas refers to the overlapping working areas of multiple stage machinery (that is, the areas where barrier-free operation needs to be ensured), which means that at certain moments, the activity areas of multiple machines may intersect, resulting in potential risks of collision or interference.

[0069] When there are multiple stage machines and the clearance monitoring areas of multiple stage machines overlap, obtain the real-time stage task set of these stage machines, which refers to the task and operation instruction set of all mechanical equipment on the current stage at a specific time point, including the task requirements, task start and end time of each mechanical equipment, etc. Parse the real-time stage task set and extract the specific control instructions of each mechanical equipment, including the task target position, speed, execution time, operation direction (such as raising, lowering, moving) and other information.

[0070] According to the control instructions, the motion trajectory of each stage machine in the stage space is constructed, including the time sequence and spatial position of the trajectory. In other words, for each stage machine, the changes in its position, speed, time, etc. during the task execution are calculated according to the control instructions, and the timing trajectory of the machine is obtained, which reflects the motion state of the mechanical equipment that changes over time during the task execution. The timing trajectory set refers to a set of information such as the position, posture, speed, etc. of each machine at different time points recorded according to the control instructions and task arrangements of the stage machine, and analyzes the dynamic behavior of the machine in time and space.

[0071] Analyze the trajectory sets of multiple stage machines to determine whether there is interference or collision. Interference analysis refers to analyzing the activity trajectories of multiple stage machines, checking whether there are intersections or overlaps in their working areas, and determining whether these intersections may cause collisions or other safety hazards. Based on the time-series trajectories of multiple machines, determine whether there are machines occupying the same space at the same time, and use geometric calculation methods (such as line segment intersection calculation, rectangular bounding box, etc.) to detect trajectory overlap; use geometric calculation methods (such as line segment intersection calculation, rectangular bounding box, etc.) to detect trajectory overlap.

[0072] Based on the results of the interference analysis, avoidance decisions are generated to ensure that there will be no collisions between stage machinery. Based on the results of the interference analysis, the trajectory, speed or pause operation of the machinery is dynamically adjusted to avoid collisions, including changing the trajectory, adjusting the order of task execution or adjusting the speed of the machinery. Dynamic avoidance decisions refer to the automatic calculation and decision of how to adjust the operating sequence or trajectory of these machines to avoid interference or conflict when potential interference or collision is found in the working trajectories of multiple stage machinery. For example, if the trajectory of the lifting platform coincides with that of the revolving stage, the dynamic avoidance decision may require the revolving stage to pause or change its path until the platform is lifted and lowered. Through interference analysis and dynamic avoidance decisions, collisions and interferences between multiple stage machinery can be avoided in real time to ensure the safe operation of stage machinery.

[0073] Furthermore, the present application also includes the following steps:

[0074] The neighboring stage of the target stage is obtained, and a virtual stage environment is constructed according to the relative positions of the target stage and the neighboring stage; the stage machinery of the target stage and the neighboring stage are mapped to the virtual stage environment; and a virtual occlusion analysis is performed according to the real-time stage tasks of the target stage and the neighboring stage; and a perspective avoidance decision is made according to the virtual occlusion analysis results.

[0075] Specifically, identify and acquire the stage space around the target stage. These neighboring stages may have physical or mission connections with the target stage. Neighboring stages are other stages adjacent to the target stage, usually used in stage performance environments where multiple stage systems collaborate or are connected. The target stage may be adjacent to the stage next to it (such as an auxiliary stage or background stage). These stages work together and may have multiple mechanical systems operating at the same time. Use stage layout design drawings or sensors to identify other stages around the target stage and calculate the relative position and spatial relationship between the two. Obtain information such as coordinates, angles, dimensions, etc. of each stage, and determine the distance and angle between adjacent stages.

[0076] According to the relative position and layout of the neighboring stages, a virtual stage environment is constructed through 3D modeling tools to accurately reflect the spatial relationship between the stages. Using virtual reality modeling technology, a digital stage environment is created based on the physical layout of the stage and the control information of the mechanical equipment. Identify all mechanical equipment on the target stage and the neighboring stage, such as lifting platforms, revolving stages, etc. According to the specific position and function of each machine, the position, movement trajectory and related control data of the mechanical equipment of the target stage and the neighboring stage are accurately mapped in the virtual environment to ensure that the position, movement range, etc. of each machine in the virtual environment are consistent with the actual stage. For example, suppose there is a lifting platform and a revolving stage, which are located on the target stage and the neighboring stage respectively. In the virtual stage environment, through device mapping, the position and movement trajectory of the lifting platform are drawn together with the revolving stage to ensure that the movement of the mechanical equipment is correctly presented in the virtual environment.

[0077] Obtain the real-time stage tasks of the target stage and neighboring stages, including the motion plan and performance movements of the machines. Simulate the relative positions of different machines, motion occlusion and other factors, and perform occlusion analysis based on the virtual environment to determine whether occlusion will occur, including factors such as simulated mechanical motion occlusion and lighting effects. Optimize the position and movement trajectory of the stage machinery to ensure that the machinery behind is not blocked.

[0078] Virtual occlusion analysis refers to simulating the relative positions and movements of different stage mechanical equipment in a virtual stage environment to analyze whether these mechanical equipment will block each other's view or movement, which helps to optimize the position and trajectory of stage machinery and ensure that different machines do not interfere with each other when performing tasks. For example, if the lifting platform on the target stage is rising and the revolving stage on the neighboring stage is rotating, the virtual occlusion analysis will simulate the lifting process of the lifting platform to check whether it will block the movement of the revolving stage. According to the results of the virtual occlusion analysis, determine which mechanical movements may be blocked from the audience's view. Develop corresponding avoidance strategies, including adjusting the position of the machinery, changing the trajectory or delaying the movement, to ensure that the rear machinery is not blocked by the front machinery, thereby optimizing the audience's viewing experience. Through the construction of a virtual stage environment, the movement trajectory of mechanical equipment is accurately simulated, the stage layout is optimized, and it is ensured that mechanical equipment does not block each other when performing tasks, improving the smoothness and safety of the performance.

[0079] In summary, the multi-sensor fusion stage machinery intelligent control method provided by this application has the following technical effects:

[0080] Through the interactive target stage space, based on the basic stage characteristics of the target stage, the target sensor category is determined, and a fusion sensor network is deployed in the target stage space; the task schedule of the target stage is obtained, the real-time stage tasks are extracted, and the sensor acquisition decision is made in combination with the real-time stage tasks; based on the sensor acquisition decision result, the fusion sensor network is activated to perform multi-dimensional data acquisition to obtain a real-time multi-dimensional data set; combined with the basic characteristics of the stage, the real-time multi-dimensional data set is subjected to correlation analysis, and data fusion is performed according to the correlation analysis result to generate a fusion data set; the fusion data set is sent to the intelligent decision maker, and mapped to the corresponding decision function component for multi-channel control decision; according to the multi-channel control decision result, the stage machinery of the target stage is controlled. In other words, sensors are deployed through the basic characteristics of the stage, data is collected according to the real-time stage tasks, and combined with the basic characteristics of the stage, the real-time multi-dimensional data set is subjected to correlation analysis and data fusion, and the stage machinery of the target stage is controlled, thereby improving the accuracy of the stage machinery control.

[0081] Embodiment 2, based on the same inventive concept as the multi-sensor fusion stage machinery intelligent control method in the aforementioned embodiment 1, the present application also provides a multi-sensor fusion stage machinery intelligent control system, please refer to the attached Figure 2 , the multi-sensor fusion stage machinery intelligent control system includes:

[0082] A sensor deployment module 11, the sensor deployment module 11 is used to interact with the target stage space, determine the target sensor category based on the basic stage characteristics of the target stage, and deploy a fusion sensor network in the target stage space; a task extraction module 12, the task extraction module 12 is used to obtain the task schedule of the target stage, extract the real-time stage tasks, and make sensor acquisition decisions in combination with the real-time stage tasks; a data acquisition module 13, the data acquisition module 13 is used to activate the fusion sensor network for multi-dimensional data acquisition based on the sensor acquisition decision results, and obtain a real-time multi-dimensional data set; an association analysis module 14, the association analysis module 14 is used to combine the basic characteristics of the stage, perform association analysis of the real-time multi-dimensional data set, and perform data fusion according to the association analysis results to generate a fusion data set; a control decision module 15, the control decision module 15 is used to collect the fusion data into an intelligent decision maker, and map it to the corresponding decision function component for multi-channel control decision; a mechanical control module 16, the mechanical control module 16 is used to control the stage machinery of the target stage according to the multi-channel control decision results.

[0083] Furthermore, the sensor layout module 11 in the multi-sensor fusion stage machinery intelligent control system is also used for:

[0084] Interact with the target stage space, obtain the stage type information, stage partition information, and stage machinery information of the target stage, and extract them as the basic features of the stage; combine the basic features of the stage with the control objectives of the target stage, perform spatial division in the target stage space, and obtain a multi-level monitoring area, wherein the multi-level monitoring area at least includes an access monitoring area, a buffer monitoring area, a clearance monitoring area, and a machinery monitoring area; traverse the multi-level monitoring area, determine the corresponding target sensor category in combination with the control objective, and deploy sensors according to the target sensor category.

[0085] Furthermore, the correlation analysis module 14 in the multi-sensor fusion stage machinery intelligent control system is also used for:

[0086] Collect mechanical control records, and take the control target as the extraction target, statistically analyze the mechanical control records, and obtain the corresponding associated sensor set, wherein each associated sensor group in the associated sensor set corresponds to a control function in the control target; perform associated partitioning of the real-time multidimensional data set based on the associated sensor set as the partition constraint to obtain an initial associated data set; perform data simplification of the initial associated data set based on the multi-level monitoring area as the retention constraint, and output a standard associated data set; traverse the standard associated data set to perform adaptive hierarchical data fusion on each standard associated group, and output the fusion result as the fused data set, wherein the fused data set includes a human data class, a clearance neighborhood data class, and a mechanical state data class.

[0087] Furthermore, the control decision module 15 in the multi-sensor fusion stage machinery intelligent control system is also used for:

[0088] Extract the person in the data class from the fused data set, and transmit the person in the data class to the limit intrusion decision component for limit intrusion discrimination, and generate a limit intrusion decision result based on the limit intrusion discrimination result; extract the clearance neighborhood data class and the mechanical state data class from the fused data set, and transmit them to the mechanical avoidance decision component for avoidance decision, and generate an avoidance decision result; extract the mechanical state data class from the fused data set, use the mechanical state data class as the safety state prediction input, perform fault risk prediction through a pre-built mechanical fault analysis model component, and perform a state management decision based on the fault risk prediction result to obtain a state management decision result; integrate the limit intrusion decision result, the avoidance decision result and the state management decision result, and output them as the multi-channel control decision result.

[0089] Furthermore, the control decision module 15 in the multi-sensor fusion stage machinery intelligent control system is also used for:

[0090] Based on the target detection method, the person is identified in the data class to determine the target identity information, target location information and target dynamic information; the target location information is used as the first discrimination factor to determine the limit intrusion status, if the target location information is in the access monitoring area, the buffer monitoring area or the clearance monitoring area, the limit intrusion status is yes; the target identity information is used as the second discrimination factor to determine the limit intrusion category, and generate the limit intrusion category, wherein the limit intrusion category includes the limit intrusion of non-access objects and the limit intrusion of access objects; according to the limit intrusion status and the limit intrusion category, the corresponding limit intrusion control plan is called as the limit intrusion decision result.

[0091] Furthermore, the control decision module 15 in the multi-sensor fusion stage machinery intelligent control system is also used for:

[0092] If there are multiple stage machines in the target stage space, and the clearance monitoring areas of the multiple stage machines overlap, then the real-time stage task sets of the multiple stage machines are obtained; the real-time stage task sets are parsed, the control instructions of the multiple stage machines are extracted, and the timing trajectory sets of the multiple stage machines are constructed according to the control instructions; interference analysis is performed on the timing trajectory set, and dynamic avoidance decisions of the multiple stage machines are made according to the interference analysis results.

[0093] Furthermore, the control decision module 15 in the multi-sensor fusion stage machinery intelligent control system is also used for:

[0094] The neighboring stage of the target stage is obtained, and a virtual stage environment is constructed according to the relative positions of the target stage and the neighboring stage; the stage machinery of the target stage and the neighboring stage are mapped to the virtual stage environment; and a virtual occlusion analysis is performed according to the real-time stage tasks of the target stage and the neighboring stage; and a perspective avoidance decision is made according to the virtual occlusion analysis results.

[0095] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. Figure 1 The multi-sensor fusion stage machinery intelligent control method and specific examples in the first embodiment are also applicable to the multi-sensor fusion stage machinery intelligent control system of this embodiment. Through the detailed description of the multi-sensor fusion stage machinery intelligent control method, those skilled in the art can clearly understand the multi-sensor fusion stage machinery intelligent control system of this embodiment, so for the sake of brevity of the specification, it will not be described in detail here. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0096] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

[0097] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application belong to the scope of the present application and its equivalent technology, the present application is also intended to include these modifications and variations.

Claims

1. A multi-sensor fusion stage machinery intelligent control method, characterized in that: include: Interactive target stage space, determining the target sensor category based on the basic stage characteristics of the target stage, and deploying a fusion sensor network in the target stage space; Obtaining a task schedule for a target stage, extracting real-time stage tasks, and making sensor acquisition decisions based on the real-time stage tasks; Based on the sensor collection decision result, activating the fusion sensor network to collect multi-dimensional data and obtain a real-time multi-dimensional data set; Combining the basic characteristics of the stage, performing correlation analysis on the real-time multi-dimensional data set, and performing data fusion according to the correlation analysis results to generate a fused data set; The fused data is collected into an intelligent decision maker and mapped to corresponding decision function components for multi-channel control decision making; According to the multi-channel control decision results, the stage machinery control of the target stage is carried out.

2. The multi-sensor fusion stage machinery intelligent control method according to claim 1, characterized in that: Interactive target stage space, based on the basic stage characteristics of the target stage, determine the target sensor category, and deploy a fusion sensor network in the target stage space, including: Interacting with the target stage space, obtaining stage type information, stage partition information, and stage machinery information of the target stage, and extracting them as the basic features of the stage; Combining the basic characteristics of the stage with the control objectives of the target stage, performing spatial division in the target stage space to obtain a multi-level monitoring area, wherein the multi-level monitoring area at least includes an access monitoring area, a buffer monitoring area, a clearance monitoring area, and a mechanical monitoring area; The multi-level monitoring area is traversed, the corresponding target sensor category is determined in combination with the control target, and sensors are deployed according to the target sensor category.

3. The multi-sensor fusion stage machinery intelligent control method according to claim 2, characterized in that: Combined with the basic characteristics of the stage, the real-time multi-dimensional data set is subjected to correlation analysis, and data fusion is performed according to the correlation analysis result to generate a fused data set, including: Collecting machine control records, and taking the control target as the extraction target, statistically analyzing the machine control records, and acquiring a corresponding associated sensor set, wherein each associated sensor group in the associated sensor set corresponds to a control function in the control target; According to the associated sensor set as a partition constraint, performing associated partitioning of the real-time multidimensional data set to obtain an initial associated data set; Taking the multi-level monitoring areas as retention constraints, simplifying the initial associated data set, and outputting a standard associated data set; The standard association data set is traversed to perform adaptive hierarchical data fusion on each standard association group, and the fusion result is output as the fused data set, wherein the fused data set includes a human data class, a clearance neighborhood data class, and a mechanical state data class.

4. The multi-sensor fusion stage machinery intelligent control method according to claim 3, characterized in that: The fused data is collected into an intelligent decision maker and mapped to corresponding decision function components for multi-channel control decision making, including: Extracting the person in the data class from the fused data set, transmitting the person in the data class to the limit intrusion decision component for limit intrusion discrimination, and generating a limit intrusion decision result according to the limit intrusion discrimination result; Extracting the clearance neighborhood data class and the machine state data class from the fused data set, transmitting them to the machine avoidance decision component for making avoidance decisions, and generating avoidance decision results; Extracting the mechanical state data class from the fused data set, using the mechanical state data class as a safety state prediction input, performing fault risk prediction through a pre-built mechanical fault analysis model component, and performing a state management decision according to the fault risk prediction result to obtain a state management decision result; The limit intrusion decision result, the avoidance decision result and the state management decision result are integrated and output as the multi-channel control decision result.

5. The multi-sensor fusion stage machinery intelligent control method according to claim 4, characterized in that: Extracting the person in the data class from the fused data set, and transmitting the person in the data class to the limit violation decision component for limit violation discrimination, and generating a limit violation decision result according to the limit violation discrimination result, including: Based on the target detection method, target identification is performed on the person in the data class to determine the target identity information, target location information and target dynamic information; Taking the target location information as the first discrimination factor, the intrusion state is judged. If the target location information is in the access monitoring area, the buffer monitoring area or the clearance monitoring area, the intrusion state is yes; Using the target identity information as the second discrimination factor, performing intrusion classification discrimination to generate intrusion classifications, wherein the intrusion classifications include intrusions by non-admitted objects and intrusions by permitted objects; According to the limit intrusion status and limit intrusion category, the corresponding limit intrusion control plan is called as the limit intrusion decision result.

6. The multi-sensor fusion stage machinery intelligent control method according to claim 3, characterized in that: The fusion data is collected into an intelligent decision maker and mapped to corresponding decision function components for multi-channel control decision making, and further includes: If there are a plurality of stage machines in the target stage space, and the clearance monitoring areas of the plurality of stage machines overlap, then obtaining the real-time stage task sets of the plurality of stage machines; Parsing the real-time stage task set, extracting control instructions of a plurality of stage machines, and constructing a timing trajectory set of a plurality of stage machines according to the control instructions; An interference analysis is performed on the time sequence trajectory set, and dynamic avoidance decisions of a plurality of stage machinery are made according to the interference analysis results.

7. The multi-sensor fusion stage machinery intelligent control method according to claim 3, characterized in that: The fusion data is collected into an intelligent decision maker and mapped to corresponding decision function components for multi-channel control decision making, and further includes: Obtaining the neighboring stage of the target stage, and constructing a virtual stage environment according to the relative positions of the target stage and the neighboring stage; Mapping the stage machinery of the target stage and the neighboring stage to the virtual stage environment; And conduct virtual occlusion analysis based on the real-time stage tasks of the target stage and the neighboring stages; Make perspective avoidance decisions based on the results of virtual occlusion analysis.

8. Multi-sensor fusion stage machinery intelligent control system, characterized by: The steps for implementing the multi-sensor fusion stage machinery intelligent control method according to any one of claims 1 to 7, wherein the multi-sensor fusion stage machinery intelligent control system comprises: A sensor deployment module, the sensor deployment module is used to interact with the target stage space, determine the target sensor category based on the basic stage characteristics of the target stage, and deploy a fusion sensor network in the target stage space; A task extraction module, the task extraction module is used to obtain the task schedule of the target stage, extract the real-time stage tasks, and make sensor acquisition decisions based on the real-time stage tasks; A data acquisition module, the data acquisition module is used to activate the fusion sensor network to perform multi-dimensional data acquisition based on the sensor acquisition decision result, and obtain a real-time multi-dimensional data set; A correlation analysis module, the correlation analysis module is used to combine the basic characteristics of the stage, perform correlation analysis on the real-time multi-dimensional data set, and perform data fusion according to the correlation analysis result to generate a fused data set; A control decision module, which is used to collect the fusion data into an intelligent decision maker and map it to corresponding decision function components to make multi-channel control decisions; A mechanical control module is used to control the stage machinery of the target stage according to the multi-channel control decision result.

Images