Theater seat state real-time monitoring method based on multi-source perception fusion

Abstract

The present application relates to the technical field of seat monitoring, and discloses a theater seat state real-time monitoring method based on multi-source perception fusion, comprising: constructing a multi-modal seat state perception network and generating a standard data set for each seat; determining the state category of each data in the standard data set and determining the state category of each seat; collecting real-time environmental data, analyzing the influence coefficient of real-time environmental data on standard perception data of seats with suspected abnormal states, setting corresponding weights, weighting the standard perception data according to the weights, obtaining a fusion feature vector and performing secondary state determination on the seats with suspected abnormal states, performing ticket information verification on the seats with abnormal states, judging whether to generate a warning instruction according to the verification result, and realizing high-precision and real-time monitoring and abnormal early warning of the theater seat state to overcome the problems of insufficient real-time performance, lack of accuracy, weak abnormal identification capability and poor linkage with the ticket system in the prior art.

Description

Technical Field

[0001] This application relates to the field of seat monitoring technology, and in particular to a method for real-time monitoring of theater seat status based on multi-source sensing fusion. Background Technology

[0002] As the core venue for cultural performances and conferences, the real-time and accurate monitoring of seating status in theaters is a key link in ensuring performance order, improving service quality, and strengthening safety management.

[0003] In existing technologies, theater seat status monitoring mainly relies on manual inspections and single-camera visual recognition. Both suffer from insufficient real-time performance, failing to update the entire seating status in milliseconds during performances, thus making it difficult to meet dynamic management needs. Secondly, accuracy is lacking, prone to misjudgments and omissions, and unable to accurately reflect the actual occupancy of seats. Thirdly, anomaly detection capabilities are weak, lacking effective identification and early warning mechanisms for various abnormal occupancy behaviors, making it difficult to address safety hazards caused by children sitting alone, resource waste caused by items occupying seats, and emergency response needs for urgent departures. Fourthly, the lack of integration with the ticketing system makes it impossible to verify the rationality of seat occupancy through ticket information, easily leading to problems such as unlicensed occupancy and misplaced seats, affecting the theater's operational order. Summary of the Invention

[0004] To address the aforementioned technical issues, this application provides a real-time monitoring method for theater seat status based on multi-source sensing fusion. This method collects multi-source data, including seat pressure, thermal sensing data, and images, through a multimodal sensing network to generate a standard dataset. A pre-defined state analysis model is used to initially classify the seat data, identifying normal, abnormal, and suspected abnormal states. For seats suspected of being abnormal, the impact of real-time environmental data on each sensing data point is further analyzed, weights are dynamically set, and weighted fusion is performed to form a fused feature vector for secondary state determination. For seats determined to be in an abnormal state, verification is performed in conjunction with ticketing information. Finally, based on the verification results and the type of abnormality, corresponding warning instructions are generated. This achieves high-precision, real-time monitoring and anomaly warning of theater seat status, overcoming the problems of insufficient real-time performance, lack of accuracy, weak anomaly identification capabilities, and poor integration with the ticketing system in existing technologies.

[0005] In some embodiments of this application, a method for real-time monitoring of theater seat status based on multi-source sensing fusion is provided, including:

[0006] A multimodal seat state perception network is constructed, and a standard dataset for each seat is generated, which includes several standard perception data. Based on a preset state analysis model, the state category of each data point in the standard dataset is determined, and the state category of each seat is determined. The state categories include normal state, abnormal state, and suspected abnormal state. Collect real-time environmental data, analyze the influence coefficient of real-time environmental data on each standard sensing data of seats suspected of being in abnormal states, set corresponding weights, and perform weighted processing on the standard sensing data according to the weights to obtain a fusion feature vector; The system performs secondary state determination on seats suspected of being abnormal based on fused feature vectors, verifies ticketing information for seats in abnormal states, and determines whether to generate an early warning instruction based on the verification results.

[0007] In some embodiments of this application, a standard dataset for each seat is generated, including: The multimodal seat status perception network includes a pressure sensor array deployed in each seat, an infrared thermal sensor array deployed in the theater, and an AI camera; The pressure data collected by the pressure sensor array, the thermal data collected by the infrared thermal array, and the image data collected by the AI ​​camera are uniformly mapped to a standard time reference frame. The mapped data is spatiotemporally aligned and preprocessed to obtain a standard dataset for each seat, which includes several standard perception data. Each standard perception data point includes the corresponding data type, data value, and corresponding seat number information.

[0008] In some embodiments of this application, the preset state analysis model includes: Define several preset state categories; Generate several expected datasets for each seat under each preset state category, and each historical dataset is mapped to a corresponding data type; Generate a preset state analysis sub-model for the corresponding seat based on the data type mapped to each historical dataset and the corresponding preset state type; Several preset state analysis sub-models for each seat are generated sequentially; Generate a preset state analysis model based on all preset state analysis sub-models.

[0009] In some embodiments of this application, the state category of each data point in the standard dataset is determined based on a preset state analysis model, and the state category of each seat is determined, including: The standard dataset and corresponding number information of each seat are input into the preset state analysis model to obtain the state category of each data in the standard dataset. The state category includes normal category and abnormal category. If all data in the same standard dataset have the same state category and are classified as normal, then the corresponding seat's state category is normal. If all data in the same standard dataset have the same state category and are all in the abnormal category, then the corresponding seat's state category is abnormal. If the state categories of all data in the same standard dataset are not all of the same type, then the state category of the corresponding seat is a suspected abnormal state.

[0010] In some embodiments of this application, the influence coefficient of real-time environmental data on each standard sensing data of a seat suspected of being in an abnormal state is analyzed, and corresponding weights are set, including: Generate a historical dataset of seats in suspected abnormal states under each environmental data. The historical dataset contains different historical values ​​of the corresponding environmental data, the historical collected value of each sensing data under each historical value, and the historical actual value. Based on the historical collection values ​​of the same sensing data under different historical values, the correlation coefficient between the corresponding environmental data and each sensing data is generated; Based on the correlation coefficient, determine the strongly correlated environmental data for each sensing data, construct a strongly correlated environmental data sequence for each sensing data, and set the weight coefficient for each data in the sequence; Calculate the historical difference between the historical collected value and the historical actual value of the sensing data under different historical values ​​of the same strongly correlated environmental data. Based on the historical difference, set several historical value intervals for the sensing data for each strongly correlated environmental data, and generate the historical influence sub-coefficient for each historical value interval. Determine the historical value range of the real-time environmental data for each strongly correlated environmental data, extract the historical influence sub-coefficients of the corresponding range, and perform weight processing to obtain the influence coefficient of the real-time environmental data on the corresponding standard sensing data. Generate initial weights for each standard sense data; The initial weights are adjusted based on the influence coefficients to obtain the weights for each standard sensed data.

[0011] In some embodiments of this application, the initial weights for each standard sense data are generated, including: A preset time window is set for each sensing data for each preset state category. The corresponding time window is selected according to the current state category of each standard sensing data of the seat suspected of being abnormal, and marked on a unified time reference axis. Overlapping time windows were selected, and standard perception data of seats suspected of being in abnormal states were extracted within the overlapping time windows. Data change curves for each standard perception data point were then constructed. Based on the data change curve and several preset data feature indicators of the corresponding data, the actual data features of each standard perceived data in the overlapping time window are generated. Generate standard data features for the current state category of each sensed data point within an overlapping time window; Calculate the feature differences between the actual data features and the corresponding standard data features, evaluate and assign values ​​to obtain the difference evaluation values ​​for each feature; A comprehensive difference assessment value is generated based on the difference assessment values ​​of all data features of the same standard perceived data. The initial weight of each standard perception data point for seats suspected of being in an abnormal state is set according to the comprehensive difference assessment value.

[0012] In some embodiments of this application, standard perceptual data is weighted according to weights to obtain a fused feature vector, including: Normalize the standard sensing data of seats suspected of being in abnormal condition at the current time point; The processed standard sensing data is multiplied by the corresponding weights to obtain weighted sensing data; By stitching together all weighted sensing data of the same seat that is suspected of being in an abnormal state, a time slice feature vector at that time point is generated. Generate feature vectors for several time slices within an overlapping time window, and stack them in chronological order to obtain a time series feature matrix; The time series feature matrix is ​​subjected to statistical feature compression to generate a fused feature vector.

[0013] In some embodiments of this application, a secondary state determination is performed on seats suspected of being in an abnormal state based on fused feature vectors, including: The fused feature vector is input into a pre-trained state determination model to obtain the secondary state determination result of the seat suspected of being in an abnormal state; The model employs a deep learning network architecture and is trained on a large number of historical fusion feature vectors labeled with actual state categories. The process continues until the secondary state determination result is no longer considered a suspected abnormal state, at which point the secondary state determination result is taken as the final state determination result for the corresponding seat.

[0014] In some embodiments of this application, ticket information is verified for seats in abnormal states, and a warning instruction is generated based on the verification result, including: Generate a list of sold seats based on ticketing information; The seat number information that is determined to be in an abnormal state is compared with the seat sales list. Based on the comparison result and the actual state category corresponding to the abnormal state, and in combination with the preset ticketing-state joint determination rules, it is determined whether to generate an early warning instruction.

[0015] In some embodiments of this application, an electronic device is also included, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the program to implement the above-described method for real-time monitoring of theater seat status based on multi-source perception fusion.

[0016] The real-time monitoring method for theater seat status based on multi-source sensing fusion in this application has the following advantages compared with the prior art: Multi-source data, including pressure, thermal, and image data of seats, are collected through a multimodal sensing network to generate a standard dataset. A pre-defined state analysis model is used to perform preliminary state classification of each seat's data, identifying normal, abnormal, and suspected abnormal states. For seats suspected of being abnormal, the impact of real-time environmental data on each sensing data is further analyzed, weights are dynamically set, and weighted fusion is performed to form a fused feature vector for secondary state determination. For seats determined to be in an abnormal state, verification is performed in conjunction with ticketing information. Finally, based on the verification results and the type of abnormality, corresponding early warning instructions are generated, achieving high-precision, real-time monitoring and early warning of abnormalities in theater seating. This overcomes the problems of insufficient real-time performance, lack of accuracy, weak anomaly identification capabilities, and poor linkage with the ticketing system in existing technologies. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the real-time monitoring method for theater seat status based on multi-source sensing fusion in an embodiment of this application. Figure 2 This is a schematic diagram of the structure of the electronic device in the embodiments of this application. Detailed Implementation

[0018] The specific embodiments of this application will be described in further detail below with reference to the accompanying drawings and examples. The following examples are used to illustrate this application, but are not intended to limit the scope of this application.

[0019] In the description of this application, it should be understood that the terms "center", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application.

[0020] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0021] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection between two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0022] like Figure 1 As shown in the figure, the real-time monitoring method for theater seat status based on multi-source sensing fusion in this application includes: S101: Construct a multimodal seat state perception network and generate a standard dataset for each seat, wherein the standard dataset includes several standard perception data; S102: Determine the state category of each data in the standard dataset based on the preset state analysis model, and determine the state category of each seat. The state categories include normal state, abnormal state, and suspected abnormal state. S103: Collect real-time environmental data, analyze the influence coefficient of real-time environmental data on each standard sensing data of seats suspected of being in abnormal states, set corresponding weights, and perform weighted processing on the standard sensing data according to the weights to obtain a fusion feature vector; S104: Based on the fused feature vector, perform secondary state determination on seats suspected of being in abnormal state, verify ticketing information for seats in abnormal state, and determine whether to generate a warning instruction based on the verification result.

[0023] In some embodiments of this application, a standard dataset for each seat is generated, including: The multimodal seat status perception network includes a pressure sensor array deployed in each seat, an infrared thermal sensor array deployed in the theater, and an AI camera; The pressure data collected by the pressure sensor array, the thermal data collected by the infrared thermal array, and the image data collected by the AI ​​camera are uniformly mapped to a standard time reference frame. The mapped data is spatiotemporally aligned and preprocessed to obtain a standard dataset for each seat, which includes several standard perception data. Each standard perception data point includes the corresponding data type, data value, and corresponding seat number information.

[0024] In this embodiment, the construction process of the standard time reference framework includes three key steps: time source selection, time synchronization mechanism design, and timestamp calibration. In this application, high-precision UTC time provided by the BeiDou Navigation Satellite System (BDS) is used as the time reference source. Addressing the sampling frequency differences among different devices in the multimodal seat status perception network, such as pressure sensor arrays, infrared thermal arrays, and AI cameras (in this application, the pressure sensor sampling frequency is 100Hz, the infrared thermal array sampling frequency is 20Hz, and the AI ​​camera frame rate is 25fps), a time synchronization mechanism based on the IEEE 1588 Precise Time Protocol (PTP) is designed. This involves deploying a PTP slave clock module at each sensing device node to periodically synchronize with the system clock. During the timestamp calibration stage, the system compensates for deviations in the original data timestamps uploaded by each device. Specifically, by pre-calibrating a fixed delay between each device and the system clock, the original timestamps are dynamically corrected to ensure that all sensing data can be accurately mapped to a unified standard time axis, providing precise time reference coordinates for subsequent spatiotemporal alignment processing and multi-source data fusion.

[0025] In this embodiment, time alignment is performed to ensure consistency in timestamps and spatial locations of data from different sources. The pressure data, thermal data, and image data that have undergone time-space alignment are preprocessed, including outlier removal, noise filtering, and data standardization, and finally integrated into a standard dataset for each seat.

[0026] In this embodiment, the standard sensing data includes pressure value sequences of several key points output by the pressure sensor array after standardization, temperature distribution data generated by the infrared thermal array for a specific seat area, and image feature parameters extracted after the AI ​​camera performs image recognition on the seat area. Specifically, the image feature parameters include the coordinates of the face recognition box, the coordinates of key points in the human posture, and the feature vector of clothing color. This standard sensing data is encapsulated in a unified data format, with each data entry carrying a unique seat number, data acquisition timestamp, and data type identifier. This facilitates accurate matching and processing in the subsequent state analysis model, collectively forming multi-dimensional information describing the seat's state.

[0027] In this embodiment, by generating a standard dataset for each seat, not only is multi-source heterogeneous data from pressure sensor arrays, infrared thermal arrays, and AI cameras integrated, but also time synchronization and spatiotemporal alignment processes are used to ensure the correlation and consistency of different types of data in time and space dimensions. This enables the system to comprehensively characterize the state features of the seats from multiple dimensions, further improving data quality and providing reliable data support for subsequent state category determination based on a preset state analysis model.

[0028] In some embodiments of this application, the preset state analysis model includes: Define several preset state categories; Generate several expected datasets for each seat under each preset state category, and each historical dataset is mapped to a corresponding data type; Generate a preset state analysis sub-model for the corresponding seat based on the data type mapped to each historical dataset and the corresponding preset state type; Several preset state analysis sub-models for each seat are generated sequentially; Generate a preset state analysis model based on all preset state analysis sub-models.

[0029] In this embodiment, the preset status categories include empty seats, normal seating, children sitting alone, seats occupied by items, emergency departure, and abnormal stay, which are divided according to common seat usage scenarios and management needs during theater operation.

[0030] In this embodiment, the expected dataset for each seat in each preset state category refers to the multi-source sensing data samples constructed based on the corresponding state of the seat during historical operation. The expected dataset includes the pressure distribution characteristics of the human buttocks and legs collected by the pressure sensor array in the corresponding state (e.g., the pressure value range of 300N-800N in the normal sitting state, and the pressure center coordinates are located within ±5cm of the geometric center of the seat), the temperature distribution data of the human torso and limbs captured by the infrared thermal array (e.g., the core area temperature in the normal sitting state is between 36℃-37.5℃, and the edge area temperature changes dynamically with the ambient temperature), and the sitting posture feature parameters extracted by the AI ​​camera through human posture recognition (e.g., the shoulder contour integrity is greater than 90% in the normal sitting state).

[0031] In this embodiment, the expected dataset is obtained through statistical analysis and feature extraction of a large amount of historical sample data. Each data sample contains a corresponding data type label (such as "stress data", "thermal data", "image feature data") and a state category label (such as "normal seating"), which are used to train a preset state analysis sub-model.

[0032] In this embodiment, when training the preset state analysis sub-model, for each preset state category of each seat, the corresponding expected dataset is divided into a training set, a validation set, and a test set in a ratio of 7:2:1. A hybrid model of an improved convolutional neural network (CNN) and a long short-term memory network (LSTM) is selected as the model architecture. The CNN extracts spatial features from pressure and thermal data, while the LSTM captures the dynamic changes of image feature parameters over time. During training, the cross-entropy loss function is used as the optimization objective, and the model parameters are iteratively updated using the Adam optimizer.

[0033] In this embodiment, after all the preset state analysis sub-models of all seats have been trained, the system integrates these sub-models to generate the final preset state analysis model, so as to achieve a comprehensive judgment on the state of all seats in the theater.

[0034] In some embodiments of this application, the state category of each data point in the standard dataset is determined based on a preset state analysis model, and the state category of each seat is determined, including: The standard dataset and corresponding number information of each seat are input into the preset state analysis model to obtain the state category of each data in the standard dataset. The state category includes normal category and abnormal category. If all data in the same standard dataset have the same state category and are classified as normal, then the corresponding seat's state category is normal. If all data in the same standard dataset have the same state category and are all in the abnormal category, then the corresponding seat's state category is abnormal. If the state categories of all data in the same standard dataset are not all of the same type, then the state category of the corresponding seat is a suspected abnormal state.

[0035] In this embodiment, the normal category includes states that conform to the normal use of theater seats, such as empty seats, normal seating, and children sitting alone; the abnormal category covers states that may affect the rational use of seat resources or pose safety hazards, such as items occupying seats, emergency departures, and abnormal lingering.

[0036] In this embodiment, when the pressure data, thermal data, and image feature data in the standard dataset all point to the same normal category, the system determines that the seat is in the normal category. If all the data consistently point to the abnormal category, it is determined to be in the abnormal category. When different types of data contradict each other, the system marks the seat as a suspected abnormal state, and further determination is required through multi-source data fusion analysis.

[0037] In this embodiment, by classifying and judging the status of each data category in the standard dataset and comprehensively judging the status of seats, the initial screening and classification of theater seat status is achieved. For seats suspected of being in abnormal status, the accuracy of status judgment is further improved by introducing real-time environmental data impact analysis and multi-source data fusion mechanism, effectively avoiding misjudgment caused by environmental interference of data sources, and laying the foundation for refined management of theater seat resources and timely early warning of safety risks.

[0038] In some embodiments of this application, the influence coefficient of real-time environmental data on each standard sensing data of a seat suspected of being in an abnormal state is analyzed, and corresponding weights are set, including: Generate a historical dataset of seats in suspected abnormal states under each environmental data. The historical dataset contains different historical values ​​of the corresponding environmental data, the historical collected value of each sensing data under each historical value, and the historical actual value. Based on the historical collection values ​​of the same sensing data under different historical values, the correlation coefficient between the corresponding environmental data and each sensing data is generated; Based on the correlation coefficient, determine the strongly correlated environmental data for each sensing data, construct a strongly correlated environmental data sequence for each sensing data, and set the weight coefficient for each data in the sequence; Calculate the historical difference between the historical collected value and the historical actual value of the sensing data under different historical values ​​of the same strongly correlated environmental data. Based on the historical difference, set several historical value intervals for the sensing data for each strongly correlated environmental data, and generate the historical influence sub-coefficient for each historical value interval. Determine the historical value range of the real-time environmental data for each strongly correlated environmental data, extract the historical influence sub-coefficients of the corresponding range, and perform weight processing to obtain the influence coefficient of the real-time environmental data on the corresponding standard sensing data. Generate initial weights for each standard sense data; The initial weights are adjusted based on the influence coefficients to obtain the weights for each standard sensed data.

[0039] In this embodiment, environmental data includes parameters such as temperature, humidity, light intensity, ground vibration intensity, seat surface temperature, environmental noise, and audience density in the theater. Real-time environmental data is aggregated to the system data processing center through an IoT gateway, providing basic data support for subsequent analysis of its impact coefficient on the standard perception data of seats suspected of being in abnormal states.

[0040] In this embodiment, for pressure sensor data, strongly correlated environmental data may include seat surface temperature and theater floor vibration; for infrared thermal sensing data, strongly correlated environmental data may be ambient temperature and infrared obstructions; for image feature data from AI cameras, strongly correlated environmental data may be light intensity and lens cleanliness.

[0041] In this embodiment, the correlation coefficient is calculated using both Pearson correlation coefficient and Spearman rank correlation coefficient. For the linear correlation between continuous environmental data (such as ambient temperature and humidity) and perceived data (such as pressure sensor values ​​and infrared thermal temperature), Pearson correlation coefficient is used. The formula is existing technology and will not be elaborated here. For the correlation between discontinuous or nonlinearly correlated environmental data (such as light intensity level and audience entry density level) and perceived data (such as clothing color feature vector in image feature parameters), Spearman rank correlation coefficient is used. The degree of correlation is measured by calculating the rank difference after sorting the data. The correlation coefficient ranges from -1 to 1.

[0042] In this embodiment, by setting a correlation coefficient threshold (0.6 in this application), when the absolute value of the correlation coefficient is greater than 0.6, it is set as strongly correlated environmental data that has a significant impact on the sensing data, and it is included in the strongly correlated environmental data sequence of the pressure sensor data and sorted according to the size of the correlation coefficient.

[0043] In this embodiment, the historical values ​​of each strongly correlated environmental data are compared with the historical difference between the historical collected value of the sensing data and the historical actual set. If the difference between the historical values ​​is less than 5%, the range of values ​​is divided into the same historical value interval, and the corresponding historical influence sub-coefficient is set according to the mean of the historical difference within the interval.

[0044] In this embodiment, setting the corresponding historical impact sub-coefficient based on the mean of historical differences specifically includes: firstly, calculating the mean of all historical differences within the historical value interval, denoted as Δ_avg, and setting the historical impact sub-coefficient as 1 + |Δ_avg| / the mean of historical actual values. If the absolute value of Δ_avg is less than 1% of the mean of historical actual values ​​(set to 0.01 in this application), then the impact of the environmental data value interval on the perceived data is considered negligible, and the historical impact sub-coefficient is set to 1.0. This transforms the statistical characteristics of historical differences into quantifiable impact sub-coefficients, providing accurate parameter basis for subsequent real-time environmental data impact analysis.

[0045] In this embodiment, the influence coefficient is calculated based on the extracted historical influence sub-coefficients and the weight coefficient of each strongly correlated environmental data in the strongly correlated environmental data sequence. The higher the ranking of the strongly correlated environmental data in the sequence, the larger the weight coefficient, and vice versa. This influence coefficient reflects the degree of comprehensive influence of each strongly correlated environmental factor on the measurement results of the sensing data under the current environmental conditions. When the influence coefficient is larger, it indicates that the current environmental factors interfere with the sensing data more strongly. At this time, the acquisition accuracy of the standard sensing data is worse, and the weight of the sensing data needs to be adjusted more significantly, that is, its weight in the state determination should be reduced to reduce the risk of misjudgment caused by environmental interference. Conversely, the smaller the influence coefficient, the weaker the environmental interference, the higher the reliability of the sensing data, and its weight can be appropriately increased.

[0046] In this embodiment, correcting the initial weights based on the influence coefficient means converting the influence coefficient into a first correction coefficient for the initial weights. The larger the influence coefficient, the smaller the first correction coefficient, and vice versa. The first correction coefficient ranges from 0.8 to 1.2. The conversion relationship between the influence coefficient and the first correction coefficient is achieved through a preset nonlinear mapping function. This function takes the influence coefficient as input and adjusts the output first correction coefficient through a piecewise function. For example, when the influence coefficient is in the range [1.0, 1.1), the first correction coefficient is 0.95; when the influence coefficient is in the range [1.1, 1.2), the first correction coefficient is 0.9; when the influence coefficient is greater than or equal to 1.2, the first correction coefficient is 0.8; and as the influence coefficient gets closer to 1.0, the first correction coefficient increases linearly with the decrease of the influence coefficient, reaching a maximum of 1.2.

[0047] In this embodiment, the weight calculation formula is: weight = initial weight × first correction coefficient. Through this dynamic adjustment mechanism, the weight of each sensing data under different environmental conditions can adapt to its reliability, ensuring that in state analysis, the sensing data that is less affected by the environment and has higher data quality is given priority, thereby improving the final judgment accuracy of seats in suspected abnormal states.

[0048] In some embodiments of this application, the initial weights for each standard sense data are generated, including: A preset time window is set for each sensing data for each preset state category. The corresponding time window is selected according to the current state category of each standard sensing data of the seat suspected of being abnormal, and marked on a unified time reference axis. Overlapping time windows were selected, and standard perception data of seats suspected of being in abnormal states were extracted within the overlapping time windows. Data change curves for each standard perception data point were then constructed. Based on the data change curve and several preset data feature indicators of the corresponding data, the actual data features of each standard perceived data in the overlapping time window are generated. Generate standard data features for the current state category of each sensed data point within an overlapping time window; Calculate the feature differences between the actual data features and the corresponding standard data features, evaluate and assign values ​​to obtain the difference evaluation values ​​for each feature; A comprehensive difference assessment value is generated based on the difference assessment values ​​of all data features of the same standard perceived data. The initial weight of each standard perception data point for seats suspected of being in an abnormal state is set according to the comprehensive difference assessment value.

[0049] In this embodiment, the preset time window refers to a specific time interval predefined for different types of perceived data and preset state categories, used to analyze changes in data characteristics. The length and start time of the preset time window are determined based on the typical change duration of data characteristics under various states in historical data, ensuring that the key change stages of the data under that state can be fully included.

[0050] In this embodiment, the overlapping time window refers to the intersection of the preset time windows corresponding to each sensing data on a unified time reference axis. Feature extraction and analysis are performed only on the standard sensing data within the overlapping time period to ensure the temporal correlation of data features and the consistency of analysis.

[0051] In this embodiment, the standard data features of the overlapping time window refer to the set of typical features that the standard perceived data should possess within the overlapping time window under a preset state category. This set is obtained by constructing a standard data change curve and extracting the corresponding features.

[0052] In this embodiment, the data change curve is plotted with time on the horizontal axis and the sensed data collection value on the vertical axis.

[0053] In this embodiment, when the standard sensing data is pressure data, the data change curve is a series of continuous curves showing the pressure values ​​of the seat at various key points in the overlapping time window as a function of time. Its data characteristic indicators include pressure peak value, pressure average value, pressure fluctuation amplitude, and pressure duration.

[0054] In this embodiment, when the standard sensing data is thermal data, the data change curve is a thermal curve of the temperature distribution data of a specific seat area changing over time within an overlapping time window. Its data characteristic indicators include the average temperature of the core area, the uniformity of temperature distribution, the rate of temperature change, and the duration of the high-temperature area.

[0055] In this embodiment, when the standard perception data is image feature data, its data change curve is represented as the dynamic change trajectory of image feature parameters (such as face recognition box coordinates, human pose key point coordinates, and clothing color feature vector) within the overlapping time window. The corresponding feature indicators include face box stability (coordinate fluctuation range), human pose integrity (key point missing ratio), clothing color feature matching degree, and feature parameter change frequency.

[0056] In this embodiment, the evaluation value of feature difference refers to the degree of deviation between the actual data feature and the standard data feature. For each data feature indicator, the absolute difference between the actual data feature value and the standard data feature value is calculated, and the ratio of the difference to the standard data feature value is normalized and used as the difference evaluation value of the feature. That is, the difference evaluation value ranges from 0 to 1. When the actual data feature value is completely consistent with the standard data feature value, the difference evaluation value is 0; when the actual data feature value deviates more from the standard data feature value, the difference evaluation value is closer to 1.

[0057] In this embodiment, the comprehensive difference assessment value is obtained by weighted averaging of the difference assessment values ​​of all data feature indicators of the same standard perceived data. The weight of each feature indicator is preset according to its importance to the state determination.

[0058] In this embodiment, the comprehensive difference evaluation value is converted into a second correction coefficient with a value range of 0.75-1, and the preset weight of each standard sensing data is corrected to obtain the initial weight. The preset weight is set in advance. In this application, the preset weight of pressure sensor data is 0.4, the preset weight of thermal sensing data is 0.3, and the preset weight of image feature data is 0.3.

[0059] In this embodiment, the conversion method is as follows: the smaller the comprehensive difference evaluation value, the closer the actual data characteristics are to the standard data characteristics, the higher the data reliability, the larger the second correction coefficient, and the larger the initial weight; conversely, the larger the comprehensive difference evaluation value, the further the data deviates from the standard characteristics, the smaller the second correction coefficient, and the smaller the initial weight. The conversion relationship is also realized through a preset nonlinear mapping function. In this application, when the comprehensive difference evaluation value is in the range [0, 0.1), the second correction coefficient is 1.0; when the comprehensive difference evaluation value is in the range [0.1, 0.2), the second correction coefficient is 0.95; when the comprehensive difference evaluation value is in the range [0.2, 0.3), the second correction coefficient is 0.9; when the comprehensive difference evaluation value is in the range [0.3, 0.4), the second correction coefficient is 0.85; when the comprehensive difference evaluation value is in the range [0.4, 0.5], the second correction coefficient is 0.8; and when the comprehensive difference evaluation value is greater than 0.5, the second correction coefficient is 0.75.

[0060] In this embodiment, the initial weight is calculated as follows: Initial weight = Preset weight × Second correction coefficient.

[0061] In this embodiment, by calculating the initial weight of each standard sensing data, the sensing data with smaller differences from the standard features and more stable data performance are given a higher initial weight, which lays the foundation for subsequent dynamic adjustment of weights in combination with the environmental impact coefficient and further optimizes the reliability of multi-source data fusion.

[0062] In some embodiments of this application, standard perceptual data is weighted according to weights to obtain a fused feature vector, including: Normalize the standard sensing data of seats suspected of being in abnormal condition at the current time point; The processed standard sensing data is multiplied by the corresponding weights to obtain weighted sensing data; By stitching together all weighted sensing data of the same seat that is suspected of being in an abnormal state, a time slice feature vector at that time point is generated. Generate feature vectors for several time slices within an overlapping time window, and stack them in chronological order to obtain a time series feature matrix; The time series feature matrix is ​​subjected to statistical feature compression to generate a fused feature vector.

[0063] In this embodiment, the normalization process involves linearly scaling each standard sensing data according to a preset value range corresponding to its data type, so that the processed data values ​​are all mapped to the [0,1] interval, thereby eliminating the influence of the difference in the dimensions of different sensing data on the weighted fusion. Among them, pressure data is normalized according to its historical maximum and minimum pressure values, thermal data is adjusted according to the normal human body temperature range and the ambient temperature range, numerical parameters in image feature data (such as face bounding box coordinates and pose key point coordinates) are normalized by dividing by the image resolution or a preset coordinate range, while features such as clothing color feature vectors that are already in the standardized space directly retain their original values.

[0064] In this embodiment, the dimension of the time-slice feature vector is determined by the quantity of standard sensing data and their respective feature dimensions. Specifically, if pressure data includes four features: peak pressure, mean pressure, fluctuation amplitude, and duration; thermal data includes four features: mean temperature in the core area, distribution uniformity, rate of change, and duration of high-temperature areas; and image feature data includes four features: face bounding box stability, pose integrity, color matching, and parameter change frequency, and each feature has been normalized to a single numerical value, then the length of each time-slice feature vector is 4 + 4 + 4 = 12 dimensions, with each dimension corresponding to a weighted sensing data feature value.

[0065] In this embodiment, when generating several time-slice feature vectors within the overlapping time window, the sampling interval of the time slice is determined according to the acquisition frequency of the sensing data. These time-slice feature vectors are stacked sequentially in chronological order to form a time series feature matrix with a dimension of (number of time slices × feature vector dimension). This matrix completely preserves the dynamic characteristics of the sensing data changing over time within the overlapping time window.

[0066] In this embodiment, statistical feature compression aims to condense dynamic time-series information into a static fusion feature vector of fixed dimensions for input into the state determination model. Specifically, for each column of the time-series feature matrix (i.e., the value of each perceived data feature at different time slices), its statistical characteristics within the entire overlapping time window are calculated, such as mean, standard deviation, maximum value, and minimum value. If the above four statistical features are selected, the fusion feature vector obtained after compression for a 12-dimensional time-slice feature vector will have a dimension of 12 × 4 = 48 dimensions, where statistical feature compression includes at least one of mean, standard deviation, maximum value, and minimum value.

[0067] In this embodiment, by constructing a fusion feature vector, the multi-source sensing data is transformed from the original time series form into a comprehensive feature representation that includes time dynamics and statistical regularities. This not only preserves the changing trends of different sensing data in the time dimension, but also achieves feature dimensionality reduction and information aggregation through statistical compression. This provides high-dimensional and highly representative input data for subsequent seat state determination, effectively improving the model's recognition accuracy for complex state patterns.

[0068] In some embodiments of this application, a secondary state determination is performed on seats suspected of being in an abnormal state based on fused feature vectors, including: The fused feature vector is input into a pre-trained state determination model to obtain the secondary state determination result of the seat suspected of being in an abnormal state; The model employs a deep learning network architecture and is trained on a large number of historical fusion feature vectors labeled with actual state categories. The process continues until the secondary state determination result is no longer considered a suspected abnormal state, at which point the secondary state determination result is taken as the final state determination result for the corresponding seat.

[0069] In this embodiment, the deep learning network architecture of the state determination model adopts a combination of bidirectional long short-term memory network and attention mechanism. The training dataset contains historical fusion feature vectors formed by historical perception data of different performances, time periods and seating areas in the theater, covering various normal and abnormal state scenarios, ensuring that the model has good generalization ability and robustness.

[0070] In this embodiment, after the fused feature vector is input into the model, the model outputs the probability distribution of each state category, and selects the category with the highest probability as the secondary state determination result. If the result is a normal category, it is directly determined as the final state; if it is still a suspected abnormal state, it is further combined with a manual review mechanism, and the final state determination result of each seat is either a normal category or an abnormal category.

[0071] In this embodiment, the normal category includes subcategories such as "no one is seated", "normally seated", and "briefly absent", while the abnormal category includes subcategories such as "absent for a long time", "abnormal occupancy (such as placing items)" and "equipment failure (such as abnormal sensor triggering)".

[0072] In some embodiments of this application, ticket information is verified for seats in abnormal states, and a warning instruction is generated based on the verification result, including: Generate a list of sold seats based on ticketing information; The seat number information that is determined to be in an abnormal state is compared with the seat sales list. Based on the comparison result and the actual state category corresponding to the abnormal state, and in combination with the preset ticketing-state joint determination rules, it is determined whether to generate an early warning instruction.

[0073] In this embodiment, the ticketing-status joint determination rule includes: if the abnormal status is "seat occupied by an item" and the ticketing information shows that the seat has been sold, a level one warning instruction is generated; if the abnormal status is "seat occupied by an item" and the ticketing information shows that the seat has not been sold, a prompt instruction is generated, but no on-site warning is triggered; if the abnormal status is "child sitting alone" and the ticketing information shows that the seat has been sold, a level two warning instruction is generated; if the abnormal status is "child sitting alone" and the ticketing information shows that the seat has not been sold, a level one warning instruction is generated and an abnormal occupancy notification is sent to the ticketing system at the same time; if the abnormal status is "emergency departure" or "abnormal posture", a level three warning instruction is generated regardless of the ticketing information status; if the abnormal status is "abnormal lingering" and the ticketing information shows that the seat has not been sold, a level two warning instruction is generated and the ticketing audit process is triggered.

[0074] In this embodiment, the warning instruction includes the type of abnormal state, the specific location of the seat (such as theater area, row number, seat number), the duration of the abnormality, a summary of key sensing data (such as pressure data peak, temperature change trend of thermal sensing area, key frame screenshot of image features), and the ticket information verification results, so that managers can quickly locate the problem and take targeted measures.

[0075] In this embodiment, the Level 1 warning command will send an audio-visual alert to the handheld terminal of the on-site patrol personnel through the system management platform, and the abnormal seat will be highlighted on the large screen of the monitoring center; the Level 3 warning command will simultaneously trigger the emergency response process of the theater security department to ensure rapid intervention in emergency situations.

[0076] In some embodiments of this application, such as Figure 2 As shown, it also includes an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, when the processor executes the program, it implements the above-described method for real-time monitoring of theater seat status based on multi-source perception fusion.

[0077] like Figure 2 As shown, the electronic device may include a processor 201, a communications interface 202, a memory 203, and a communication bus 204. The processor 201, communications interface 202, and memory 203 communicate with each other via the communication bus 204. The processor 201 can call logical instructions from the memory 203 to execute a real-time theater seating status monitoring method based on multi-source sensor fusion.

[0078] Furthermore, the logical instructions in the aforementioned memory 203 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0079] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and substitutions can be made without departing from the technical principles of this application, and these improvements and substitutions should also be considered within the scope of protection of this application.

Claims

1. A method for real-time monitoring of theater seating status based on multi-source sensing fusion, characterized in that, include: A multimodal seat state perception network is constructed, and a standard dataset for each seat is generated, which includes several standard perception data. Based on a preset state analysis model, the state category of each data point in the standard dataset is determined, and the state category of each seat is determined. The state categories include normal state, abnormal state, and suspected abnormal state. Collect real-time environmental data, analyze the influence coefficient of real-time environmental data on each standard sensing data of seats suspected of being in abnormal states, set corresponding weights, and perform weighted processing on the standard sensing data according to the weights to obtain a fusion feature vector; The system performs secondary state determination on seats suspected of being abnormal based on fused feature vectors, verifies ticketing information for seats in abnormal states, and determines whether to generate an early warning instruction based on the verification results.

2. The method for real-time monitoring of theater seating status based on multi-source sensing fusion as described in claim 1, characterized in that, Generate a standard dataset for each seat, including: The multimodal seat status perception network includes a pressure sensor array deployed in each seat, an infrared thermal sensor array deployed in the theater, and an AI camera; The pressure data collected by the pressure sensor array, the thermal data collected by the infrared thermal array, and the image data collected by the AI ​​camera are uniformly mapped to a standard time reference frame. The mapped data is spatiotemporally aligned and preprocessed to obtain a standard dataset for each seat, which includes several standard perception data. Each standard perception data point includes the corresponding data type, data value, and corresponding seat number information.

3. The method for real-time monitoring of theater seating status based on multi-source sensing fusion as described in claim 1, characterized in that, The preset state analysis model includes: Define several preset state categories; Generate several expected datasets for each seat under each preset state category, and each historical dataset is mapped to a corresponding data type; Generate a preset state analysis sub-model for the corresponding seat based on the data type mapped to each historical dataset and the corresponding preset state type; Several preset state analysis sub-models for each seat are generated sequentially; Generate a preset state analysis model based on all preset state analysis sub-models.

4. The method for real-time monitoring of theater seating status based on multi-source sensing fusion as described in claim 3, characterized in that, Based on a pre-defined state analysis model, the state category of each data point in the standard dataset is determined, and the state category of each seat is also determined, including: The standard dataset and corresponding number information of each seat are input into the preset state analysis model to obtain the state category of each data in the standard dataset. The state category includes normal category and abnormal category. If all data in the same standard dataset have the same state category and are classified as normal, then the corresponding seat's state category is normal. If all data in the same standard dataset have the same state category and are all in the abnormal category, then the corresponding seat's state category is abnormal. If the state categories of all data in the same standard dataset are not all of the same type, then the state category of the corresponding seat is a suspected abnormal state.

5. The method for real-time monitoring of theater seating status based on multi-source sensing fusion as described in claim 1, characterized in that, Analyze the impact coefficient of real-time environmental data on each standard sensing data point of seats suspected of being in abnormal states, and assign corresponding weights, including: Generate a historical dataset of seats in suspected abnormal states under each environmental data. The historical dataset contains different historical values ​​of the corresponding environmental data, the historical collected value of each sensing data under each historical value, and the historical actual value. Based on the historical collection values ​​of the same sensing data under different historical values, the correlation coefficient between the corresponding environmental data and each sensing data is generated; Based on the correlation coefficient, determine the strongly correlated environmental data for each sensing data, construct a strongly correlated environmental data sequence for each sensing data, and set the weight coefficient for each data in the sequence; Calculate the historical difference between the historical collected value and the historical actual value of the sensing data under different historical values ​​of the same strongly correlated environmental data. Based on the historical difference, set several historical value intervals for the sensing data for each strongly correlated environmental data, and generate the historical influence sub-coefficient for each historical value interval. Determine the historical value range of the real-time environmental data for each strongly correlated environmental data, extract the historical influence sub-coefficients of the corresponding range, and perform weight processing to obtain the influence coefficient of the real-time environmental data on the corresponding standard sensing data. Generate initial weights for each standard sense data; The initial weights are adjusted based on the influence coefficients to obtain the weights for each standard sensed data.

6. The method for real-time monitoring of theater seating status based on multi-source sensing fusion as described in claim 5, characterized in that, Generate initial weights for each standard sensed data point, including: A preset time window is set for each sensing data for each preset state category. The corresponding time window is selected according to the current state category of each standard sensing data of the seat suspected of being abnormal, and marked on a unified time reference axis. Overlapping time windows were selected, and standard perception data of seats suspected of being in abnormal states were extracted within the overlapping time windows. Data change curves for each standard perception data point were then constructed. Based on the data change curve and several preset data feature indicators of the corresponding data, the actual data features of each standard perceived data in the overlapping time window are generated. Generate standard data features for the current state category of each sensed data point within an overlapping time window; Calculate the feature differences between the actual data features and the corresponding standard data features, evaluate and assign values ​​to obtain the difference evaluation values ​​for each feature; A comprehensive difference assessment value is generated based on the difference assessment values ​​of all data features of the same standard perceived data. The initial weight of each standard perception data point for seats suspected of being in an abnormal state is set according to the comprehensive difference assessment value.

7. The method for real-time monitoring of theater seating status based on multi-source sensing fusion as described in claim 5, characterized in that, The standard perceived data is weighted according to weights to obtain a fused feature vector, including: Normalize the standard sensing data of seats suspected of being in abnormal condition at the current time point; The processed standard sensing data is multiplied by the corresponding weights to obtain weighted sensing data; By stitching together all weighted sensing data of the same seat that is suspected of being in an abnormal state, a time slice feature vector at that time point is generated. Generate feature vectors for several time slices within an overlapping time window, and stack them in chronological order to obtain a time series feature matrix; The time series feature matrix is ​​subjected to statistical feature compression to generate a fused feature vector.

8. The method for real-time monitoring of theater seating status based on multi-source sensing fusion as described in claim 7, characterized in that, A secondary state determination is performed on seats suspected of being in an abnormal state based on the fused feature vector, including: The fused feature vector is input into a pre-trained state determination model to obtain the secondary state determination result of the seat suspected of being in an abnormal state; The model employs a deep learning network architecture and is trained on a large number of historical fusion feature vectors labeled with actual state categories. The process continues until the secondary state determination result is no longer considered a suspected abnormal state, at which point the secondary state determination result is taken as the final state determination result for the corresponding seat.

9. The method for real-time monitoring of theater seating status based on multi-source sensing fusion as described in claim 8, characterized in that, For seats in abnormal states, verify the ticketing information and determine whether to generate an alert based on the verification results, including: Generate a list of sold seats based on ticketing information; The seat number information that is determined to be in an abnormal state is compared with the seat sales list. Based on the comparison result and the actual state category corresponding to the abnormal state, and in combination with the preset ticketing-state joint determination rules, it is determined whether to generate an early warning instruction.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the real-time monitoring method for theater seat status based on multi-source perception fusion as described in any one of claims 1 to 9.

Images