Cultural relic protection method based on multiple modes and related equipment

Abstract

The invention relates to the technical field of cultural relic protection, and provides a multi-modal-based cultural relic protection method and related equipment, and the method comprises the steps: obtaining data of a plurality of sensors corresponding to a target cultural relic, personnel recognition data, and positioning data of the target cultural relic; calculating a fire risk value and a collision risk value of the target cultural relic based on all sensor data; calculating an illegal intrusion risk value based on the personnel identification data, and calculating a transportation abnormal value based on the positioning data; fusing the fire risk value, the collision risk value, the illegal intrusion risk value and the transportation abnormal value to obtain an overall risk index of the target cultural relic; and when the overall risk index reaches a protection early warning condition, outputting early warning information. According to the method, the protection capability of the cultural relics can be improved, and the safety of the cultural relics is improved.

Description

Technical Field

[0001] This application relates to the field of cultural relic protection technology, and in particular to a multimodal cultural relic protection method and related equipment. Background Technology

[0002] Currently, existing solutions for cultural relic protection mainly fall into three categories:

[0003] Traditional security monitoring systems, such as video surveillance equipment from brands like Hikvision and Dahua, use cameras to capture real-time images of cultural relic areas, relying on manual inspections or post-event review to achieve basic security functions.

[0004] Single environmental monitoring equipment: Temperature and humidity monitoring stations and smoke detectors commonly used in museums can only collect environmental parameters individually. Some devices support alarms when the threshold is exceeded, but they lack cross-device linkage or intelligent control capabilities.

[0005] Single-function artifact display case management system: High-end display cases integrate functions such as temperature and humidity control and lighting control, but only optimize the local environment of the display scene and lack overall protection for personnel access, artifact status and location tracking.

[0006] The existing solutions have obvious limitations and cannot meet the comprehensive and intelligent needs of modern cultural relic protection:

[0007] Functional isolation: Each system operates independently, with no integration of data from environmental monitoring, security, and positioning, forming "information silos" that cannot achieve comprehensive risk assessment and coordinated response (e.g., when environmental standards are exceeded, it is impossible to automatically link security footage to trace the cause).

[0008] Low level of intelligence: Relies on manual monitoring or passive alarms, lacking proactive early warning capabilities. For example, traditional monitoring requires manual identification of anomalies, with a missed detection rate exceeding 30%; environmental monitoring equipment only alarms without automatic control, and cannot prevent cultural relics from being damaged by drastic changes in temperature and humidity.

[0009] Lack of positioning and status monitoring: The existing solution does not cover the "black box" problem of cultural relic transportation. The positioning error often exceeds 10 meters, and there is no real-time monitoring of physical risks such as vibration and collision, resulting in a transportation damage rate of 68% of accidents.

[0010] This shows that current methods of cultural relic protection have the problem of poor protection capabilities. Summary of the Invention

[0011] This application provides a multimodal method and related equipment for the protection of cultural relics, which can solve the problem of poor protection capabilities for cultural relics.

[0012] In a first aspect, embodiments of this application provide a multimodal method for the protection of cultural relics, which includes:

[0013] Acquire multiple sensor data, personnel identification data, and location data of the target cultural relic; the multiple sensor data are used to describe the state of the environment in which the target cultural relic is located, and the personnel identification data are used to describe the identification results of relevant personnel in the environment in which the target cultural relic is located;

[0014] The fire risk value and collision risk value of the target cultural relic are calculated based on all sensor data;

[0015] The risk value of unauthorized intrusion is calculated based on personnel identification data, and the value of transportation anomalies is calculated based on location data.

[0016] By integrating fire risk value, collision risk value, illegal intrusion risk value, and transportation anomaly value, the overall risk index of the target cultural relic is obtained;

[0017] When the overall risk index reaches the protection warning condition, a warning message is output.

[0018] Optionally, multiple sensor data include temperature sensor data, humidity sensor data, smoke sensor data, vibration sensor data, and distance sensor data;

[0019] The steps for calculating the fire risk and collision risk values ​​of the target cultural relic based on all sensor data include:

[0020] The fire risk value of the target cultural relic is calculated based on temperature sensor data, humidity sensor data, and smoke sensor data.

[0021] The collision risk value of the target cultural relic is calculated based on vibration sensor data and distance sensor data.

[0022] Optionally, the fire risk value of the target cultural relic can be calculated based on temperature sensor data, humidity sensor data, and smoke sensor data, including:

[0023] The fire risk value of the target cultural relic is obtained by fusing temperature sensor data, humidity sensor data, and smoke sensor data using the Kalman filter algorithm.

[0024] Optionally, the collision risk value of the target cultural relic can be calculated based on vibration sensor data and distance sensor data, including:

[0025] By using an attention mechanism to fuse vibration sensor data and distance sensor data, the collision risk value of the target cultural relic can be obtained.

[0026] Optionally, the personnel identification data includes face matching results, UID matching results, and human signal strength;

[0027] Calculate the risk value of unauthorized intrusion based on personnel identification data, including:

[0028] Determine if the human body signal strength is less than or equal to 0;

[0029] If so, then the illegal intrusion risk value will be set to 0;

[0030] Otherwise, calculate the illegal intrusion risk value based on the face matching result and the UID matching result.

[0031] Optionally, transportation anomalies can be calculated based on location data, including:

[0032] The Kalman filter algorithm is used to calculate the location data to obtain the transportation anomaly value.

[0033] Optionally, fire risk values, collision risk values, illegal intrusion risk values, and transportation anomaly values ​​can be integrated to obtain the overall risk index of the target cultural relic, including:

[0034] Through the formula:

[0035]

[0036] Calculate the overall risk index ;

[0037] in, Indicates the fire risk value. Indicates the collision risk value. Indicates the risk value of unauthorized intrusion. This indicates an abnormal value in the transportation process.

[0038] Secondly, embodiments of this application provide a multimodal cultural relic protection device, comprising:

[0039] The acquisition module is used to acquire multiple sensor data, personnel identification data, and location data of the target cultural relic; the multiple sensor data are used to describe the state of the environment in which the target cultural relic is located, and the personnel identification data are used to describe the identification results of relevant personnel in the environment in which the target cultural relic is located;

[0040] The first calculation module is used to calculate the fire risk value and collision risk value of the target cultural relic based on all sensor data;

[0041] The second calculation module is used to calculate the risk value of illegal intrusion based on personnel identification data and to calculate the transportation anomaly value based on location data.

[0042] The fusion module is used to merge fire risk values, collision risk values, illegal intrusion risk values, and transportation anomaly values ​​to obtain the overall risk index of the target cultural relic;

[0043] The early warning module is used to output early warning information when the overall risk index reaches the protection warning conditions.

[0044] Thirdly, embodiments of this application provide a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the aforementioned multimodal cultural relic protection method.

[0045] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned multimodal-based cultural relic protection method.

[0046] The above-mentioned solution in this application has the following beneficial effects:

[0047] In the embodiments of this application, multiple sensor data, personnel identification data, and location data of the target cultural relic are acquired. Then, the fire risk value and collision risk value of the target cultural relic are calculated based on all sensor data. Next, the illegal intrusion risk value is calculated based on the personnel identification data, and the transportation anomaly value is calculated based on the location data. Finally, the fire risk value, collision risk value, illegal intrusion risk value, and transportation anomaly value are fused to obtain the overall risk index of the target cultural relic. Finally, when the overall risk index reaches the protection warning condition, a warning message is output. This approach of protecting cultural relics based on sensor data, personnel identification data, and location data integrates environmental information, personnel information, and location information. This avoids a one-sided assessment of the cultural relic's condition, reduces misjudgments caused by single data points, and achieves a comprehensive assessment of the cultural relic's environment, condition, and personnel behavior, thereby improving the cultural relic's protection capabilities and ultimately enhancing its safety. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 A flowchart illustrating a multimodal cultural relic protection method provided in an embodiment of this application;

[0050] Figure 2 A schematic diagram of the hardware structure of a multimodal cultural relic protection system provided in an embodiment of this application;

[0051] Figure 3 A schematic diagram of the system architecture of a multimodal cultural relic protection system provided in an embodiment of this application;

[0052] Figure 4A schematic diagram of the structure of a multimodal cultural relic protection device provided in an embodiment of this application;

[0053] Figure 5 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application. Detailed Implementation

[0054] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0055] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0056] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0057] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0058] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0059] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0060] To address the problem of poor protection capabilities in existing cultural relics protection methods, this application provides a multimodal cultural relic protection method. This method acquires multiple sensor data, personnel identification data, and location data corresponding to the target cultural relic. Then, it calculates the fire risk value and collision risk value of the target cultural relic based on all sensor data, the illegal intrusion risk value based on personnel identification data, and the transportation anomaly value based on location data. These values ​​are then fused to obtain an overall risk index for the target cultural relic. Finally, when the overall risk index reaches the protection warning condition, a warning message is output. This method, based on sensor data, personnel identification data, and location data, integrates environmental, personnel, and location information, avoiding one-sided assessments of the cultural relic's condition and reducing misjudgments caused by single data points. It achieves a comprehensive assessment of the cultural relic's environment, condition, and personnel behavior, improving the protection capability and ultimately enhancing the safety of the cultural relic.

[0061] The following is an illustrative example of the multimodal cultural relic protection method provided in this application.

[0062] like Figure 1 As shown, the multimodal-based cultural relic protection method provided in this application includes the following steps:

[0063] Step 11: Obtain multiple sensor data, personnel identification data, and location data of the target cultural relic.

[0064] The aforementioned target cultural relics are those that require protection, such as museum exhibits. Multiple sensor data are used to describe the state of the environment in which the target cultural relics are located, and personnel identification data is used to describe the identification results of relevant personnel in the environment in which the target cultural relics are located.

[0065] The aforementioned sensor data includes temperature sensor data (describing the temperature of the environment surrounding the target artifact), humidity sensor data (describing the humidity of the environment surrounding the target artifact), smoke sensor data (describing the smoke concentration of the environment surrounding the target artifact), vibration sensor data (describing the vibration amplitude of the target artifact), and distance sensor data (describing the distance between the target artifact and other objects, such as walls). The environment surrounding the target artifact is a museum, warehouse, etc. Personnel identification data includes face matching results (describing whether the face of a person entering the environment surrounding the target artifact matches a face database, with a value of 0 (no match) or 1 (match)), user identification (UID) matching results (describing whether the identification of a person entering the environment surrounding the target artifact matches an identification database, with a value of 0 (no match) or 1 (match)), and human signal strength (describing the probability of a human being present within a certain range of the target artifact).

[0066] In some embodiments of this application, multiple sensor data can be acquired using different sensors, such as temperature sensor data, humidity sensor data, smoke sensor data, vibration sensor data, distance sensor data, etc., set in the environment where the target cultural relic is located; personnel identification data can be acquired using face sensors, radio frequency identification (RFID) devices, etc., set in the environment where the target cultural relic is located; and positioning data can be acquired using a Beidou positioning system set around the target cultural relic (such as a safe or transport vehicle carrying the target cultural relic).

[0067] For example, after obtaining the above data, each data point needs to be preprocessed, including steps such as time synchronization, outlier removal, and data standardization. Time synchronization: Using a preset clock (such as the system clock of the Raspberry Pi 4B) as a reference, timestamps are aligned for data from different acquisition frequencies (e.g., downsampling the high-frequency data (10Hz) of the MPU6050 to 1Hz and synchronizing it with the temperature and humidity data (0.2Hz); Outlier removal: The "3σ criterion" is used to remove abnormal sensor data (such as occasional -20℃ outliers, instantaneous 10000ppm spikes, etc.); Data standardization: All data are mapped to the [0,1] interval to eliminate dimensional differences.

[0068] Step 12: Calculate the fire risk value and collision risk value of the target cultural relic based on all sensor data.

[0069] The fire risk values ​​described above are used to describe the risk of fire in the environment where the target cultural relic is located. The higher the value, the greater the risk of fire. The collision risk values ​​described above are used to describe the risk of collision with the target cultural relic. The higher the value, the greater the risk of collision.

[0070] In some embodiments of this application, the steps of calculating the fire risk value and collision risk value of the target cultural relic based on all sensor data include:

[0071] The first step is to calculate the fire risk value of the target cultural relic based on temperature sensor data, humidity sensor data, and smoke sensor data.

[0072] Specifically, the Kalman filter algorithm is used to fuse data from temperature sensor, humidity sensor, and smoke sensor to obtain the fire risk value of the target cultural relic.

[0073] For example, the core formula of the Kalman filter algorithm includes:

[0074] State prediction equation:

[0075]

[0076]

[0077] in, Let k be the prior state estimate at time k. Let be the state transition matrix. For the posterior state estimate at time k-1, Let K be the prior estimate of the covariance matrix at time k. Let be the posterior estimated covariance matrix at time k-1. Let be the process noise covariance matrix.

[0078] Observational update equation:

[0079]

[0080]

[0081]

[0082] in, For Kalman gain, For the observation matrix, Let k be the observation value at time k. To observe the noise covariance matrix, Let k be the posterior state estimate at time k. Let k be the estimated covariance matrix at time k. It is an identity matrix.

[0083] In practical applications, data from different types of sensors (temperature sensor data, humidity sensor data, and smoke sensor data) are input as observations into the Kalman filter algorithm. The specific implementation steps are as follows:

[0084] First, initialize state estimation And estimate the covariance matrix .

[0085] Then, the state prediction equation is used to predict the prior state estimate at the current time step. and prior estimation of covariance matrix .

[0086] Then, data from different types of sensors are used as observations. .

[0087] Then, the Kalman gain is calculated using the observation update equation. .

[0088] Finally, calculate the fused state estimate. and posterior estimation of covariance matrix In this example, the fused state estimate This refers to the fire risk value of the target cultural relic.

[0089] The second step is to calculate the collision risk value of the target cultural relic based on vibration sensor data and distance sensor data.

[0090] Specifically, the collision risk value of the target cultural relic is obtained by fusing vibration sensor data and distance sensor data using an attention mechanism.

[0091] For example, vibration sensor data and distance sensor data are used as input data for the fusion attention mechanism. The fusion attention mechanism is used to calculate the collision risk value of the target cultural relic.

[0092] Step 13: Calculate the illegal intrusion risk value based on personnel identification data and calculate the transportation anomaly value based on location data.

[0093] The aforementioned illegal intrusion risk values ​​describe the risk of illegal intrusion into the environment surrounding the target cultural relic; the higher the value, the higher the risk. Transportation anomaly values ​​describe the risk to the target cultural relic during transportation; the higher the value, the higher the risk.

[0094] In some embodiments of this application, the steps of calculating the illegal intrusion risk value based on personnel identification data and calculating the transportation anomaly value based on location data include:

[0095] The first step is to determine whether the human signal strength is less than or equal to 0.

[0096] If so, the illegal intrusion risk value will be set to 0.

[0097] Otherwise, calculate the illegal intrusion risk value based on the face matching result and the UID matching result.

[0098] For example, a weighted summation method can be used to calculate the illegal intrusion risk value based on the face matching result and the UID matching result, such as: illegal intrusion risk value I = 1 - (0.6×f + 0.4×r), where f represents the face matching result and r represents the UID matching result.

[0099] The second step involves using the Kalman filter algorithm to calculate the location data and obtain the transportation anomaly value.

[0100] For example, the location data is used as the observation value in the Kalman filter algorithm, and the Kalman filter algorithm is used to calculate and output the transportation anomaly value.

[0101] Step 14: Combine the fire risk value, collision risk value, illegal intrusion risk value, and transportation anomaly value to obtain the overall risk index of the target cultural relic.

[0102] The overall risk index described above is used to describe the probability that the target cultural relic is at risk. The higher the value, the higher the probability of risk.

[0103] Specifically, through the formula:

[0104]

[0105] Calculate the overall risk index .

[0106] in, Indicates the fire risk value. Indicates the collision risk value. Indicates the risk value of unauthorized intrusion. This indicates an abnormal value in the transportation process.

[0107] Step 15: When the overall risk index reaches the protection warning condition, output the warning information.

[0108] The above warning information is used to describe that the target cultural relic is at risk of safety.

[0109] Specifically, the protection and early warning conditions can be: the overall risk value is greater than the preset risk value.

[0110] For example, the safety of a target cultural relic can be determined based on the overall risk index by setting a threshold. For example: S < 0.3: safe state, no alarm; 0.3≤S < 0.6: warning state, only cloud platform pushes reminder; S≥0.6: alarm state, triggering sound and light alarm + device linkage + mobile phone SMS push.

[0111] In addition, separate alarms can be triggered based on fire risk values, collision risk values, illegal intrusion risk values, and transportation anomaly values. For example:

[0112] Fire risk value F (range: 0-1):

[0113] Physical meaning: Reflects the comprehensive fire hazard of "whether the ambient temperature and humidity are suitable + whether there is flammable gas / smoke". For example: F=0.8 means "the temperature is too high (32℃) + the humidity is too low (35%) + the smoke concentration reaches 1800ppm". At this time, cultural relics (such as paper paintings and calligraphy, textiles) are easily ignited and an immediate warning is required.

[0114] Collision risk value C (range 0-1):

[0115] Physical meaning: Reflects the collision risk of "whether the cultural relic is too close to the obstacle + whether it has been subjected to vibration and impact". For example: C=0.7 means "the cultural relic is only 8cm away from the display case glass + a horizontal vibration of 2.5m / s² was detected", which may be caused by personnel accidentally touching the display case, causing the cultural relic to shift, and a distance warning needs to be triggered.

[0116] Intrusion risk value I (range: 0-1):

[0117] Physical meaning: Reflects the security risk of "whether unauthorized personnel have entered the cultural relic protection area". Example: I=0.9 means "human infrared signal detected (someone has broken in) + face matching score of only 0.2 (unauthorized personnel) + RFID not matched", which is determined to be an illegal intrusion. The access control should be locked and the alarm should be triggered immediately.

[0118] Transportation anomaly value T (range: 0-1):

[0119] Physical meaning: This reflects the risk of damage from "exceeding speed during the transportation of cultural relics" and "experiencing severe impact." For example, T=0.85 means that "the transportation speed reaches 75km / h (exceeding the 60km / h threshold) + an impact acceleration of 4.2m / s² is detected," which may cause fragile cultural relics such as porcelain and bronzes to crack. The driver should be reminded to slow down and check the condition of the cultural relics.

[0120] Overall safety index S (range: 0-1):

[0121] Physical meaning: It comprehensively reflects the total risks currently faced by cultural relics, including "environmental degradation + physical damage + human-caused destruction + loss of location control". For example, S=0.65 means "there is a moderate fire risk (F=0.5) + a relatively high intrusion risk (I=0.8) + a low collision / transportation risk", so the problem of human intrusion should be addressed first, while also paying attention to the environmental temperature and humidity.

[0122] It is worth mentioning that the protection of cultural relics based on sensor data, personnel identification data, and positioning data integrates environmental information, personnel information, and positioning information. This avoids one-sided assessments of the condition of cultural relics, reduces misjudgments caused by single data, and enables a comprehensive assessment of the environment, condition, and personnel behavior of cultural relics, thereby improving the protection capabilities of cultural relics and ultimately enhancing their safety.

[0123] In some embodiments of this application, the hardware structure of the multimodal-based cultural relic protection system running the method of this application is as follows: Figure 2 As shown, the system includes an STM32F103C8T6 control chip, an LCD display connected to the chip, a relay module, a DHT11 temperature and humidity sensor, an MQ-2 smoke sensor, a human body sensor module, an RC-522 module, a buzzer, a Raspberry Pi 4B, an HCSR004 ultrasonic sensor connected to the Raspberry Pi 4B, a DR1612Z1 GPS+BeiDou dual-mode positioning module, a DC motor + fan, an MPU6050 vibration and attitude sensor, a K210, a Thingsboard cloud platform, and a mobile APP. The Raspberry Pi 4B is connected to the STM32F103C8T6 and K210 via a UART serial port, and to the Thingsboard cloud platform and mobile APP via wireless WIFI.

[0124] The architecture design of the above system is as follows: Figure 3 As shown, it includes a perception layer, a network layer, and an application layer. The perception layer includes a DHT11 temperature and humidity sensor, an MQ-2 smoke sensor, an MPU6050 vibration and attitude sensor, an HCSR004 ultrasonic sensor, a human body sensing module, a K210 face recognition module, and a GPS+BeiDou dual-mode positioning module. The network layer includes a Raspberry Pi 4B and an STM32F103C8T6. The application layer includes the ONENET cloud platform and an APP client. Data transmission between the network layer and the application layer is carried out through protocols such as MQTT, ESP8266 WIFI upload, and HTTP.

[0125] The Raspberry Pi 4B plays a crucial role in data processing, system coordination, and communication hub. It was sourced from reputable online retailers or authorized resellers, ensuring quality and stable supply. The Raspberry Pi 4B's powerful data processing capabilities and abundant interface resources enable it to easily handle the multi-task concurrent processing needs of the cultural relic protection system, efficiently completing tasks such as data acquisition, analysis, and communication with the cloud.

[0126] The K210 chip is the core of the facial recognition function, and it is sourced from chip manufacturers or authorized distributors. With its superior image processing capabilities and low power consumption, the K210 chip provides high-precision and high-efficiency facial recognition for cultural relic protection systems. In cultural relic protection scenarios, facial recognition is a key technology ensuring that only authorized personnel can access cultural relics, and the application of the K210 chip significantly enhances the level of intelligence in cultural relic security.

[0127] The STM32F103C8T6 is primarily responsible for RFID identification and basic data acquisition and processing tasks in the system. Purchased from chip manufacturers or authorized distributors, it features high performance and low power consumption. The STM32F103C8T6's powerful processing capabilities enable it to quickly read RFID tag information and perform preliminary data processing and analysis, providing efficient technical support for the management of cultural relics entering and leaving the warehouse, and access control for display cases.

[0128] The ESP8266 module, sourced from an electronic component distributor, plays a crucial role in IoT communication. With its high cost-effectiveness and stable Wi-Fi communication capabilities, the ESP8266 uploads various data from the cultural relic protection system to the cloud platform, enabling remote data sharing and monitoring. Through the ESP8266, administrators can monitor the status of cultural relics anytime, anywhere via mobile phone or computer, truly realizing intelligent and information-based cultural relic protection.

[0129] The DHT11 temperature and humidity sensor monitors ambient temperature and humidity in real time, providing reliable data support for temperature and humidity control in the preservation environment of cultural relics; the MQ-2 smoke sensor accurately detects smoke concentration and can issue timely warnings in the early stages of a fire, buying time for rescue efforts; the MPU6050 vibration and attitude sensor acquires vibration and attitude information of cultural relics in real time, which is crucial for monitoring the stability of cultural relics during transportation and display, and can promptly detect accidental collisions or tilting issues; the HCSR004 ultrasonic sensor accurately obtains the distance between cultural relics and surrounding obstacles through non-contact measurement, effectively preventing damage to cultural relics due to collisions and providing data support for safe layout and display; the human body sensing module can quickly detect changes in human infrared radiation, promptly detect unauthorized personnel entering, and ensure the safety of cultural relics during non-open hours or when unattended.

[0130] The SR1612Z1 GPS+BeiDou dual-mode positioning module is a core component of the cultural relic location tracking function. It can simultaneously receive signals from both GPS and BeiDou satellite systems, using satellite signals to pinpoint the precise geographical location of the cultural relic or its transport carrier. The positioning module connects to a Raspberry Pi 4B via a UART serial port, sending positioning data, including longitude, latitude, altitude, and speed, to the Raspberry Pi according to the NMEA-0183 protocol. The Raspberry Pi's code reads and parses this data via the serial port, extracting the longitude, latitude, and other location information. The parsed data is then uploaded to the Onenet cloud platform via an ESP8266 module, allowing administrators to view the cultural relic's location trajectory anytime via mobile phone or computer.

[0131] The functions of each sensor in the above system are shown in Table 1:

[0132] Table 1

[0133] The aforementioned system enables the implementation of the cultural relic protection method described in this application, allowing for the fusion of data from various sensor types, including temperature, humidity, smoke, vibration, attitude, and distance. By establishing a mathematical model and weighting the fusion of data from different sensors, the accuracy and reliability of cultural relic condition assessment are effectively improved. For example, in fire early warning, the fusion of data from temperature, humidity, and smoke sensors allows for more accurate assessment of fire hazards; in cultural relic stability monitoring, the combination of data from vibration, attitude, and distance sensors enables timely detection of potential collision risks to cultural relics. This multimodal data fusion technology overcomes the limitations of single-sensor data, providing stronger technical support for the comprehensive protection of cultural relics.

[0134] The following is an exemplary description of the multimodal cultural relic protection device provided in this application.

[0135] like Figure 4 As shown, this application provides a multimodal cultural relic protection device, the multimodal cultural relic protection device 400 comprising:

[0136] The acquisition module 401 is used to acquire multiple sensor data, personnel identification data and location data of the target cultural relic; the multiple sensor data are used to describe the state of the environment in which the target cultural relic is located, and the personnel identification data are used to describe the identification results of relevant personnel in the environment in which the target cultural relic is located;

[0137] The first calculation module 402 is used to calculate the fire risk value and collision risk value of the target cultural relic based on all sensor data;

[0138] The second calculation module 403 is used to calculate the illegal intrusion risk value based on personnel identification data and to calculate the transportation anomaly value based on location data.

[0139] The fusion module 404 is used to fuse fire risk value, collision risk value, illegal intrusion risk value, and transportation anomaly value to obtain the overall risk index of the target cultural relic;

[0140] The early warning module 405 is used to output early warning information when the overall risk index reaches the protection early warning condition.

[0141] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0142] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0143] like Figure 5 As shown, an embodiment of this application provides a terminal device, wherein the terminal device D10 of this embodiment includes: at least one processor D100 ( Figure 5 The diagram shows only one processor, a memory D101, and a computer program D102 stored in the memory D101 and executable on the at least one processor D100, wherein the processor D100 executes the computer program D102 to implement the steps in any of the above method embodiments.

[0144] Specifically, when the processor D100 executes the computer program D102, it acquires multiple sensor data, personnel identification data, and location data corresponding to the target cultural relic. Then, based on all sensor data, it calculates the fire risk value and collision risk value of the target cultural relic; based on personnel identification data, it calculates the illegal intrusion risk value; and based on location data, it calculates the transportation anomaly value. Finally, it fuses the fire risk value, collision risk value, illegal intrusion risk value, and transportation anomaly value to obtain the overall risk index of the target cultural relic. Finally, when the overall risk index reaches the protection warning condition, it outputs a warning message. This approach to cultural relic protection based on sensor data, personnel identification data, and location data integrates environmental information, personnel information, and location information. This avoids one-sided assessments of the cultural relic's condition, reduces misjudgments caused by single data points, and achieves a comprehensive assessment of the cultural relic's environment, condition, and personnel behavior, thereby improving the cultural relic's protection capabilities and ultimately enhancing its safety.

[0145] The processor D100 can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0146] In some embodiments, the memory D101 may be an internal storage unit of the terminal device D10, such as a hard disk or memory of the terminal device D10. In other embodiments, the memory D101 may be an external storage device of the terminal device D10, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the terminal device D10. Furthermore, the memory D101 may include both internal and external storage units of the terminal device D10. The memory D101 is used to store the operating system, applications, bootloader, data, and other programs, such as the program code of the computer program. The memory D101 can also be used to temporarily store data that has been output or will be output.

[0147] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above.

[0148] This application provides a computer program product that, when run on a terminal device, enables the terminal device to implement the steps described in the various method embodiments above.

[0149] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a multimodal cultural relic protection method / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0150] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0151] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0152] The above description is the preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention.

Claims

1. A multimodal approach to cultural relic preservation, characterized in that, include: The system acquires multiple sensor data, personnel identification data, and location data of the target cultural relic; the multiple sensor data are used to describe the state of the environment in which the target cultural relic is located, and the personnel identification data are used to describe the identification results of relevant personnel in the environment in which the target cultural relic is located; The fire risk value and collision risk value of the target cultural relic are calculated based on all sensor data; The illegal intrusion risk value is calculated based on the personnel identification data, and the transportation anomaly value is calculated based on the location data; The fire risk value, collision risk value, illegal intrusion risk value, and transportation anomaly value are integrated to obtain the overall risk index of the target cultural relic; When the overall risk index reaches the protection warning condition, a warning message is output.

2. The method for protecting cultural relics according to claim 1, characterized in that, The data from the multiple sensors include temperature sensor data, humidity sensor data, smoke sensor data, vibration sensor data, and distance sensor data; The steps for calculating the fire risk value and collision risk value of the target cultural relic based on all sensor data include: The fire risk value of the target cultural relic is calculated based on the temperature sensor data, humidity sensor data, and smoke sensor data. The collision risk value of the target cultural relic is calculated based on the vibration sensor data and the distance sensor data.

3. The method for protecting cultural relics according to claim 2, characterized in that, The calculation of the fire risk value of the target cultural relic based on the temperature sensor data, humidity sensor data, and smoke sensor data includes: The fire risk value of the target cultural relic is obtained by fusing the temperature sensor data, humidity sensor data, and smoke sensor data using the Kalman filter algorithm.

4. The method for protecting cultural relics according to claim 2, characterized in that, The calculation of the collision risk value of the target cultural relic based on the vibration sensor data and the distance sensor data includes: The collision risk value of the target cultural relic is obtained by fusing the vibration sensor data and the distance sensor data using an attention mechanism.

5. The method for protecting cultural relics according to claim 1, characterized in that, The personnel identification data includes face matching results, UID matching results, and human signal strength; The calculation of the illegal intrusion risk value based on the personnel identification data includes: Determine whether the intensity of the human signal is less than or equal to 0; If so, then the illegal intrusion risk value will be set to 0; Otherwise, an illegal intrusion risk value is calculated based on the face matching result and the UID matching result.

6. The method for protecting cultural relics according to claim 1, characterized in that, The calculation of transportation anomaly values ​​based on the location data includes: The location data is calculated using the Kalman filter algorithm to obtain transportation anomaly values.

7. The method for protecting cultural relics according to claim 1, characterized in that, The process of integrating the fire risk value, collision risk value, illegal intrusion risk value, and transportation anomaly value to obtain the overall risk index of the target cultural relic includes: Through the formula: ； Calculate the overall risk index ; in, Indicates the fire risk value. Indicates the collision risk value. Indicates the risk value of unauthorized intrusion. This indicates an abnormal value in the transportation process.

8. A multimodal cultural relic protection device, characterized in that, include: The acquisition module is used to acquire multiple sensor data, personnel identification data, and location data of the target cultural relic. The first calculation module is used to calculate the fire risk value and collision risk value of the target cultural relic based on all sensor data; the multiple sensor data are used to describe the state of the environment in which the target cultural relic is located, and the personnel identification data are used to describe the identification results of relevant personnel in the environment in which the target cultural relic is located; The second calculation module is used to calculate the illegal intrusion risk value based on the personnel identification data and to calculate the transportation anomaly value based on the location data. The fusion module is used to fuse the fire risk value, collision risk value, illegal intrusion risk value, and transportation anomaly value to obtain the overall risk index of the target cultural relic; The early warning module is used to output early warning information when the overall risk index reaches the protection early warning condition.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the multimodal cultural relic protection method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the multimodal-based cultural relic protection method as described in any one of claims 1 to 7.

Images