360-degree video real-time streaming and environmental analysis system and method

Abstract

This invention provides a system and method for real-time streaming of 360-degree video and environmental analysis. [Solution] The system may include a 360-degree camera that obtains 360-degree video of the work site, an environmental sensor module that measures environmental data in real time, including at least one of gas concentration, temperature, humidity, and pressure data of the work environment, an embedded module that processes the video of the work site obtained from the 360-degree camera, and a data analysis unit that analyzes the video processed by the embedded module and the environmental data.

Description

Technical Field

[0001] The present invention relates to a 360-degree video real-time streaming and environment analysis system and method, which transmit and analyze video in real time, and monitor work sites and security environments remotely to improve safety.

Background Art

[0002] Digital Twin technology is a technology that can replicate corresponding assets in a virtual space based on real-time data of physical assets and systems, and perform various simulations and analyses. Thereby, the state of physical assets can be monitored in real time, future performance can be predicted, and maintenance and operation can be optimized. Digital Twin is applied to various industrial fields such as manufacturing, energy industry, construction, transportation, etc. In particular, it plays a role in greatly improving safety through real-time monitoring and rapid response in high-risk environments.

[0003] If Digital Twin technology is applied in industrial sites, it becomes possible to prevent unexpected accidents by continuously collecting and analyzing data of physical assets, or to plan maintenance work in advance. For example, it is possible to detect abnormal signs at an early stage while monitoring the operating state of equipment in real time and solve problems. In addition, various scenarios can be simulated in a virtual environment to derive an optimal operation plan, thereby maximizing operation efficiency and saving costs.

[0004] However, existing video streaming and environmental monitoring systems have many limitations that hinder the full realization of digital twin technology. Firstly, most existing systems rely on local networks, making it difficult to transmit and analyze real-time data remotely. Secondly, existing systems rely primarily on simple analysis patterns, limiting their ability to analyze and respond in real time to the diverse variables and hazards that may arise in complex work environments. For example, currently used AI-based analysis systems can only analyze specific objects such as facial recognition or whether a security helmet is being worn, and are insufficient for a comprehensive analysis of worker conditions and environmental changes.

[0005] Furthermore, the system lacks the capability to integrate and process data from multiple cameras and various sensors. In advanced industrial environments, it is necessary to collect and comprehensively analyze data from many cameras and sensors, and existing systems show limitations in integrating and processing such data in real time. As a result, real-time analysis and response capabilities are reduced.

[0006] Security is also a significant issue. Existing systems have security vulnerabilities during data transmission over networks, potentially exposing them to external attacks and hacking. Systems accessible remotely are particularly vulnerable to data breaches and hacking, requiring strong encryption and security technologies to prevent this.

[0007] Therefore, there is a growing need for systems that combine real-time 360-degree video streaming with AI-based video and environmental analysis. Such systems must be able to detect various hazards that may occur in industrial sites and security environments in real time, monitor them remotely, and provide a high level of security.

[0008] Prior art is Korean Patent Publication No. 10-2627538. This document concerns a system for generating safety education content using 360-degree panoramic video. The system constructs a virtual space based on video captured with a 360-degree panoramic camera for the purpose of safety education at construction sites, tags hazardous elements, and provides educational data.

[0009] However, this system operates on a local network and has limitations in monitoring or managing safety conditions in real time from remote locations. Furthermore, the system focuses on generating and managing educational content, lacking the ability to monitor real-time situations occurring in actual industrial settings. Because it relies on transmitting captured video to a server to compose educational materials, it is difficult to immediately confirm or respond to safety conditions at industrial sites from remote locations.

[0010] Furthermore, network connectivity is limited to local areas, resulting in insufficient infrastructure for remote administrators and supervisors to monitor or respond to on-site situations in real time. This makes it difficult to respond quickly to emergencies and can reduce the overall efficiency of the safety management system.

[0011] Therefore, a system is required that can monitor and respond to safety situations in real time, even in remote locations, thereby overcoming the limitations of on-site management. [Prior art documents] [Patent Documents]

[0012] [Patent Document 1] Korean Registered Patent Publication No. 10-2627538 [Overview of the Initiative] [Problems that the invention aims to solve]

[0013] The object of the present invention is to provide a 360-degree video real-time streaming and environmental analysis system and method that can stream 360-degree video in real time anywhere without relying on a local network, and improve security and safety management capabilities through AI-powered video and environmental analysis. [Means for solving the problem]

[0014] A first aspect of the present invention for solving the aforementioned problems relates to a 360-degree video real-time streaming and environmental analysis system. The system includes a 360-degree camera for obtaining 360-degree video of a work site, an environmental sensor module for measuring environmental data in real time, including at least one of gas concentration, temperature, humidity, and pressure data of the work environment, an embedded module for processing the video of the work site obtained from the 360-degree camera, and a data analysis unit for analyzing the video processed by the embedded module and the environmental data. The data analyzed by the data analysis unit can be provided in real time to on-site workers or remote managers to support safety management and response.

[0015] According to an embodiment of the present invention, the data analysis unit may include an AI computation module that tracks a worker in video processed from the embedded module and performs real-time AI analysis on worker data including at least one of face detection, human tracking, respiration and pulse analysis, and area intrusion detection, and a sensor analysis module that analyzes environmental data received from the environmental sensor module in real time.

[0016] According to an embodiment of the present invention, the present invention may include a web server that receives the data analyzed by the data analysis unit and provides it to a remote user in real time, along with the video processed by the embedded module.

[0017] According to an embodiment of the present invention, it includes an alarm module that provides a warning notification signal so that an operator or administrator can visually or auditorily recognize a warning, and the data analysis unit analyzes the operator data and the environmental data in real time to generate a warning notification signal and transmits it to the alarm module, and the alarm module may transmit the warning notification signal to the web server.

[0018] According to an embodiment of the present invention, the AI calculation module may detect the face of an operator in the video processed by the embedded module, track the state of the operator through human recognition, sense abnormal behavior or risk factors of the operator through image analysis, and generate the warning notification signal.

[0019] According to an embodiment of the present invention, the AI calculation module analyzes minute hue changes occurring in the face of an operator in the video processed by the embedded module to sense a change in blood volume (Blood Volume Pulse; BVP), and thereby may estimate the pulse state of the operator.

[0020] According to an embodiment of the present invention, the AI calculation module may sense the movement of an operator using a motion branch and remove the influence of the corresponding movement on the analysis of biometric information.

[0021] According to an embodiment of the present invention, the AI calculation module may track changes in the skin hue of an operator over a certain period using an appearance branch and extract changes in blood volume and a breathing pattern.

[0022] According to an embodiment of the present invention, the AI calculation module may select a main area to be analyzed using an attention mask and filter out irrelevant areas to improve the accuracy of biometric information estimation.

[0023] According to an embodiment of the present invention, the AI calculation module may measure a respiratory cycle based on zero crossing in a breath waveform and analyze the respiratory state of an operator in real time.

[0024] According to an embodiment of the present invention, the AI calculation module may calculate a stress level based on the biometric information of an operator and generate a warning notification signal when the stress exceeds a set value.

[0025] According to an embodiment of the present invention, when the environmental data value exceeds a predetermined critical value based on the environmental data, the sensor analysis module may generate the warning notification signal and transmit it to the alarm module.

[0026] According to an embodiment of the present invention, it may include an encryption module that encrypts the operator data and environmental data analyzed by the AI calculation module and the sensor analysis module and then transmits them to the web server.

[0027] A second aspect of the present invention is a method for real-time streaming of 360-degree video and environmental analysis performed via a system including a 360-degree camera, an environmental sensor module, an embedded module, and a data analysis unit, the method comprising: a) obtaining a 360-degree video of a work site by the 360-degree camera; b) measuring in real time environmental data including at least one of gas concentration, temperature, humidity, and pressure data of a work environment by the environmental sensor module; c) processing the video of the work site obtained from the 360-degree camera by the embedded module; d) analyzing the video processed by the embedded module and the environmental data by the data analysis unit; and e) providing the data analyzed by the data analysis unit to a field operator or a remote administrator in real time to assist in performing safety management and response.

[0028] According to embodiments of the present invention, step d) may involve an AI computing module tracking a worker in video processed from the embedded module and performing real-time AI analysis on worker data including at least one of face detection, human tracking, respiration and pulse analysis, and area intrusion detection, and a sensor analysis module analyzing environmental data received from the environmental sensor module in real time.

[0029] According to an embodiment of the present invention, step e) may include the step of a web server receiving the data analyzed by the data analysis unit and providing it to a remote user in real time together with the video processed by the embedded module.

[0030] According to an embodiment of the present invention, step d) includes the data analysis unit analyzing the worker data and the environmental data in real time, generating a warning notification signal and transmitting it to an alarm module, the alarm module transmitting the warning notification signal to the web server, and step e) may further include the alarm module providing the warning notification signal so that a worker or manager can visually or audibly recognize the warning.

[0031] According to embodiments of the present invention, step d) may include the AI ​​computing module detecting the worker's face in the video processed from the embedded module, tracking the worker's state through human recognition, sensing abnormal behavior or hazardous elements of the worker through image analysis, and generating the warning signal. [Effects of the Invention]

[0032] According to the present invention, it is possible to stream 360-degree video in real time to remote locations via a web server, overcoming the limitations of existing local network-based systems. This allows users to easily monitor work sites and security systems from anywhere, regardless of physical distance, significantly improving real-time response capabilities.

[0033] Furthermore, the embedded module enables real-time video analysis using AI, allowing for immediate detection and warning of abnormal behavior in industrial settings and security situations through features such as face detection, human recognition, respiration and pulse analysis, and intrusion detection in designated areas. This offers significant benefits in terms of accident prevention and safety management.

[0034] Furthermore, the system monitors gas concentrations in the work environment in real time via gas sensors and sends immediate warnings when hazardous conditions occur, ensuring worker safety. This system provides both physical and remote warnings simultaneously, enabling a quick and efficient response.

[0035] Furthermore, encryption and security modules can be applied during data transmission to securely protect the system from external attacks. This allows for real-time remote data monitoring while minimizing security risks, thus supporting safer system operation. [Brief explanation of the drawing]

[0036] [Figure 1] This figure shows the components of the 360-degree video real-time streaming and environmental analysis system according to the present invention. [Figure 2] This diagram shows the path by which an alarm is transmitted to an operator or remote user when an abnormal value occurs in the results analyzed by the AI ​​calculation module according to the present invention. [Figure 3] This figure illustrates each process of the real-time biometric information estimation technology applied to the AI ​​computing module according to the present invention. [Figure 4] This diagram shows the path by which an alarm is transmitted to an operator or remote user when an abnormal value occurs in the results analyzed by the sensor analysis module according to the present invention. [Modes for carrying out the invention]

[0037] The specific details for implementing the present invention will be described below with reference to the attached drawings. In describing the present invention, if it is determined that the prior art functions involved are obvious to the articulate, and that describing them would unnecessarily obscure the gist of the present invention, then a detailed explanation will be omitted.

[0038] Figure 1 shows the configurations of the 360-degree video real-time streaming and environmental analysis system according to the present invention.

[0039] Referring to Figure 1, the 360-degree video real-time streaming and environmental analysis system according to the present invention consists of a 360-degree streaming device 1000 and a network 2000. The 360-degree streaming device 1000 is composed of various modules that collect and process 360-degree video and environmental data from the work site. The data is transmitted to a web server via a security module through the network 2000, allowing it to be viewed in real time on a client terminal 2300. Remote real-time monitoring and analysis become possible through this connection.

[0040] The 360-degree streaming device 1000 may include a 360-degree camera 1100, an environmental sensor module 1200, an embedded module 1300, a data analysis unit 1400 having an AI computing module 1410 and a sensor analysis module 1420, an encryption module 1500, a touch display 1600, and an alarm module 1700. The network 2000 may include a security module 2100, a web server 2200, and a client terminal 2300.

[0041] The 360-degree camera 1100 is responsible for obtaining 360-degree video of the work site in real time and transmitting it to the embedded module 1300. For example, a high-performance camera such as the Ricoh 360 camera simultaneously captures images from various angles through multiple lenses, providing omnidirectional 360-degree video, thereby enabling real-time monitoring of worker activities and changes in the surrounding environment. This method of capture is used to quickly identify potential hazards that may occur at the work site and is also advantageous for verifying workers and comparing them before and after work.

[0042] The 360-degree camera 1100 supports a variety of resolutions such as 1K, 2K, and 4K, allowing users to select the appropriate image quality according to their work environment and network conditions. 4K high-resolution video provides a clear view of details in the work area, making it effective for security monitoring and safety management. 1K low-resolution video reduces network bandwidth and provides smoother streaming, making it suitable for real-time monitoring.

[0043] The collected video is transmitted in real time to the embedded module 1300 via the UVC (USB Video Class) protocol, where the data can be processed and analyzed. The 360-degree camera 1100 can monitor all angles of the work site in all directions, providing a crucial foundation for real-time analysis of potential hazards that may arise while workers are moving or performing specific tasks. It can also be integrated with Jetson Nano to work with deep learning-based AI solutions, enabling advanced functions such as biometric data analysis and work situation recording.

[0044] The 360-degree camera 1100 system can operate via power connection or battery pack, and its water-resistant design ensures stable operation in various work environments. Through a router, client terminals such as smartphones and tablets (2300) can immediately monitor the work site via real-time streaming, and the audible warning function allows for quick response when problems occur on-site.

[0045] The environmental sensor module 1200 plays a role in maintaining workplace safety by collecting various environmental data from the work site in real time. The environmental sensor module 1200 continuously monitors data such as gas concentration, temperature, humidity, and pressure, and collects data in real time to detect potential hazards that may occur in the work environment at an early stage.

[0046] The collected environmental data is first transmitted to the embedded module 1300, and then to the sensor analysis module 1420. The sensor analysis module 1420 analyzes the environmental data transmitted via the embedded module 1300 in real time and quickly detects abnormal values ​​and hazardous situations. For example, if the gas concentration exceeds a pre-set critical value, the sensor analysis module 1420 immediately generates a warning signal, which can be used to alert workers or managers via the alarm module 1700.

[0047] The environmental sensor module 1200 is particularly useful in factories and workplaces where hazardous materials are used, providing the ability to analyze data in real time to prevent workers from being put in danger and to send immediate warnings when necessary. This helps to maintain a safer workplace.

[0048] Furthermore, the collected environmental data is encrypted and transmitted to the web server 2200 via the embedded module 1300 and the sensor analysis module 1420, enabling real-time monitoring even from remote locations. Remote managers and workers can check the environmental conditions of the workplace in real time through this system and take immediate action if any abnormalities are detected.

[0049] The embedded module 1300 is responsible for processing and transmitting 360-degree video and environmental data in real time. The embedded module 1300 integrates and processes data collected from the 360-degree camera 1100 and the environmental sensor module 1200, and then transmits it to the AI ​​computation module 1410 and the sensor analysis module 1420.

[0050] First, the embedded module 1300 receives and processes 1K, 2K, and 4K resolution video transmitted from the 360-degree camera 1100 via the UVC (USB Video Class) protocol. The embedded module 1300 can select the optimal resolution according to the network bandwidth and processing requirements. It also receives and monitors environmental data such as gas concentration, temperature, humidity, and pressure collected by the environmental sensor module 1200 in real time.

[0051] After processing the collected data, the embedded module 1300 transmits it to the sensor analysis module 1420, which analyzes the data in real time. If the gas concentration exceeds a preset critical value or if an abnormal condition in the work environment is detected, the sensor analysis module 1420 generates a warning signal, which is then alerted to the worker and manager via the alarm module 1700.

[0052] In this process, the embedded module 1300 is responsible for data transmission and processing, and ultimately, the AI ​​computing module 1410 and the sensor analysis module 1420 analyze the data to detect hazardous situations. Furthermore, all data is encrypted and transmitted to the web server 2200, allowing for real-time monitoring of the work site conditions even from remote locations. Remote managers and workers can then check the environmental conditions of the work site and respond immediately if any abnormalities are detected. As a result, the embedded module 1300 plays a crucial role in maintaining work site safety by processing data in real time and transmitting it to the data analysis unit 1400.

[0053] The data analysis unit 1400 analyzes video and environmental data collected at the work site in real time and plays a role in detecting dangerous situations. The data analysis unit 1400 consists of an AI calculation module 1410 and a sensor analysis module 1420, which analyze the worker's condition and environmental data respectively, and can immediately generate a warning signal if danger is detected.

[0054] Figure 2 shows the path by which an alarm is transmitted to an operator or remote user when an abnormal value occurs in the results analyzed by the AI ​​computing module according to the present invention. Figure 3 shows each process of the real-time biometric information estimation technology applied to the AI ​​computing module according to the present invention.

[0055] Referring to Figures 2 and 3, the AI ​​computing module 1410 plays a crucial role in estimating the worker's biometric information by analyzing video transmitted in real time from the 360-degree camera 1100. This module detects faces through deep learning-based algorithms and extracts key facial features using Face Mesh and Face Detector technologies (s10). This allows for the estimation of blood volume pulse (BVP) and respiratory cycle, and the analysis of the worker's biological state.

[0056] In this process, the Motion Branch (s20) and the Appearance Branch (s30) work together. The Motion Branch captures patterns that change subtly over time in real-time video, measuring subtle changes on the facial surface to analyze pulse and respiration. Specifically, pulse is analyzed based on subtle changes in skin color caused by changes in blood flow, and respiratory rate can be inferred by analyzing the expansion and contraction patterns of the facial skin over time. The Appearance Branch is used to extract visual static information in each frame, extracting common features from the input cumulative frames to analyze the position of the face and skin within the frame.

[0057] An attention mask selects the main area to be analyzed, filters out unwanted background, and highlights important spatial and temporal features. This allows the model to extract features primarily from key skin areas rather than the entire image, improving the accuracy of biosignal analysis.

[0058] After face detection, a region of interest is defined, and the face region is aligned in each frame using the face rotation values. This alignment process improves the performance of the biometric analyzer, and the face rotation values ​​are calculated horizontally and vertically based on the coordinates of the eyes, nose, and mouth. In this process, the rotation of the face region is calculated so that the eyes are horizontal and the nose and mouth are vertical. Subsequently, the skin region is calculated again within the face region of interest, using the landmark coordinates obtained after the detection of the face mesh to define the remaining skin region excluding the eyes, eyebrows, and mouth, and removing the background to improve the accuracy of the biometric analyzer.

[0059] The overall process shown in Figure 3 involves image acquisition, face detection, skin region calculation, pulse and respiratory waveform analysis, waveform noise reduction filtering, quantification (number of steps per minute), and stress index analysis.

[0060] During the waveform analysis process, waveform noise reduction filtering (s40) is performed. A moving average filter and a convolutional filter are applied here. First, the moving average filter processes the data through the following formula to remove fundamental noise.

[0061]

number

[0062] Each x represents a value of the input signal. These values ​​are signal samples arranged in chronological order, with x1 being the oldest sample and x10 being the most recent sample. These values ​​are continuously measured data, representing 10 samples taken from the original signal before filtering. The 10 in the formula is the value divided when calculating the average; in this case, since we are averaging 10 samples, we divide by 10. That is, we add up all 10 sample values ​​and then divide that value by 10 to obtain the average. This allows us to average out short-term signal fluctuations, reduce noise in areas with high signal fluctuations, and smooth the signal.

[0063] The Conv filter removes noise while preserving the peaks and trends of a waveform through the following formula. The Conv filter operates on a principle similar to the convolution operation in artificial intelligence, and is applied to one-dimensional arrays, making it a suitable method for processing time-series data.

[0064]

number

[0065] Each x represents a value of the input signal. These values ​​are signal samples arranged in chronological order, with x1 being the oldest sample and x10 being the newest sample. In other words, the 10 most recent samples of the input signal are used in the filtering process. The number preceding each signal sample indicates the weighting value assigned to that sample. The weighting value determines how important the sample is processed when filtering.

[0066] The 1 / 10 before the equation serves to scale the result. This is the process of dividing the sum of the weighted samples by 10 and averaging them. This value adjusts the overall value so that the filtered result does not become too large. In equation 2 above, the weights consist of -3, -2, -1, 3, and 7, and such a pattern is effective in capturing changes in the signal in a way that further emphasizes or smooths specific patterns in the signal.

[0067] Subsequently, the pulse rate per minute is converted by analyzing the number of peaks in the waveform (s50). A peak is indicated in the waveform when the current value is two values ​​greater than the previous and subsequent values, and it forms the apex of the waveform. The following formula is used in this process.

[0068]

number

[0069] The respiratory rate is converted to respiratory rate per minute by calculating the zero crossing point (s60). The zero crossing is calculated based on the number of times the waveform aligns with 0, and the zero crossing point is calculated using the following formula.

[0070]

number

[0071] Furthermore, the stress index is calculated by analyzing the diversity of pulse rate and the correlation between pulse rate and respiration. When under stress, the pulse rate becomes constant, and under normal conditions, the pulse rate changes drastically; therefore, the more diverse the pulse waveform, the lower the stress index is considered to be. Also, if the balance between pulse rate and respiration is not maintained, this may indicate a state of stress.

[0072] The stress index is calculated using the following formula (s70). The following formula is RMSSD (Root Mean Square of Successive Differences), which is used to evaluate pulse variability.

[0073]

number

[0074] In the above formula, N represents the total number of RR intervals. An RR interval is the time interval between two consecutive electrocardiogram signals (i.e., two heartbeats) that generate a heartbeat. This value is the total number of heartbeat intervals in the formula, which adjusts the average value calculated in the formula.

[0075] R i+2 , R i+1 , R i These represent the intervals between consecutive heartbeats. This value is the time interval between each heartbeat and is called the RR interval. For all intervals from i=1 to N-1, the above (R i+2 -R i+1 )-(R i+1 -R i The difference between ) is calculated and the results are summed up. This formula calculates a value indicating heart rate variability by squaring the difference between consecutive heart rate intervals (RR intervals), finding the average, and then converting that value to a square root.

[0076] A higher RMSSD value means that heart rates occur at more varied intervals, indicating that the heart is responding more appropriately. Conversely, a lower RMSSD value means less heart rate variability, leading to more consistent heart rate intervals, which may be a result of high stress or poor health.

[0077] Furthermore, the correlation between pulse rate and respiration is analyzed using the following formula (s80).

[0078]

number

[0079] In the above formula, Pulse Value refers to the pulse rate. This value represents the number of heartbeats per minute and is derived based on the pulse rate data measured above. Generally, it is calculated based on the number of pulse peaks. Breath Value refers to the respiratory rate. This value represents the number of breaths per minute and is derived by measuring the respiratory cycle through zero crossing on the respiratory curve. That is, it is the value calculated based on the intersection of inspiration and expiration, representing the number of breaths per minute.

[0080] The above formula divides the difference between pulse rate and respiration by the respiration value. This is a process of normalizing the difference between the two values ​​using the respiration value as a baseline in order to evaluate the relative difference between pulse rate and respiration. This value shows the correlation between pulse rate and respiration, and is useful in evaluating stress levels. A large value means that there is a large discrepancy between pulse rate and respiration.

[0081] Finally, the stress index is calculated through the following formula (s90):

[0082]

number

[0083] In the above formula, the Stress Value is calculated by assigning weighted values ​​to both the a value (pulse rate variability) and the b value (discrepancy between pulse rate and respiration). a × 0.7 plays an even more important role in pulse rate variability, while b × 0.3 plays a supporting role, taking into account the correlation between pulse rate and respiration. A higher Stress Value indicates higher stress and a greater likelihood of physical instability. Conversely, a lower value indicates lower stress and a potentially more stable physical state.

[0084] As a result, the AI ​​calculation module 1410 of the present invention can evaluate and analyze the worker's stress level in real time.

[0085] This video-based biometric analysis technology is non-contact and can monitor workers' health without additional sensors. This technology is effective not only for safety management but also for stress management and understanding workers' health conditions. In particular, its real-time streaming and analysis capabilities allow for immediate on-site response, making it useful in various industrial settings and security systems.

[0086] Figure 4 shows the path by which an alarm is transmitted to an operator or remote user when an abnormal value occurs in the results analyzed by the sensor analysis module according to the present invention.

[0087] Referring to Figure 4, the sensor analysis module 1420 analyzes environmental data such as gas concentration, temperature, humidity, and pressure collected from the environmental sensor module 1200 in real time. The sensor analysis module 1420 generates an immediate warning signal if an environmental change occurs that exceeds a preset critical value. For example, if the concentration of a hazardous gas reaches a dangerous level, the sensor analysis module immediately generates a warning signal to notify workers and managers. This warning is also transmitted to the site via the alarm module 1700 and simultaneously transmitted to the web server 2200 in an encrypted form, allowing remote users to monitor the environmental conditions at the work site in real time.

[0088] The encryption module 1500 is a security element of the present invention and is used to securely transmit 360-degree video data and environmental data. In the present invention, data collected by various sensors and the 360-degree camera 1100 is processed by the embedded module 1300 and the data analysis unit 1400, and then transmitted to a remote user via the web server 2200. In this process, the encryption module 1500 encrypts the data to prevent sensitive data from being leaked to the outside or accessed without permission, ensuring secure transmission.

[0089] The encryption module 1500 encrypts data analyzed by the AI ​​computing module 1410 and the sensor analysis module 1420 in real time. For example, all data is encrypted to securely protect not only sensitive information such as worker facial recognition, respiration, and pulse rate, but also environmental data such as gas concentration, temperature, and humidity. This encrypted data is transmitted over the network, thereby securely protecting it from external threats.

[0090] Furthermore, the encryption module 1500 also ensures security in communication between the web server 2200 and the client terminal 2300. When the client terminal 2300 remotely monitors data or receives notifications, this data is securely transmitted via an encryption protocol such as SSL / TLS. For example, if the AI ​​calculation module 1410 detects a dangerous situation involving a worker, or if the sensor analysis module 1420 detects an anomaly in environmental data, an alarm signal is generated, and this signal is also transmitted in an encrypted form, allowing a remote user to receive a secure warning.

[0091] As a result, the encryption module 1500 securely protects all data generated in this invention, maintaining data integrity and confidentiality while it is transmitted over the network, and blocking external intrusions and threats. This ensures security while allowing for real-time monitoring and alert reception from remote locations.

[0092] The touch display 1600 provides a user interface that allows for real-time monitoring of the work site conditions. Through the touch display 1600, users can immediately check system status, streaming video, AI analysis results, environmental sensor data, and more, with results based on the analyzed data displayed in real time. Users can quickly identify abnormal situations via the touch display 1600 and take necessary immediate action. For example, when the AI ​​calculation module detects abnormal breathing or hazardous elements in a worker, this information is visually displayed on the touch display 1600, allowing the worker to recognize and respond accordingly. The touch display 1600 also provides an intuitive user experience, supporting real-time management of the work environment without complex system configurations.

[0093] The alarm module 1700 is an element that communicates abnormal situations occurring at the work site through visual or auditory warnings. The alarm module 1700 consists of a processor, an LED, or a buzzer, and activates immediately when a hazardous situation is detected by the AI ​​calculation module 1410 or the sensor analysis module 1420. For example, if a gas sensor detects a dangerous concentration of hazardous gas, or if the AI ​​calculation module 1410 detects abnormal behavior by a worker, the alarm module 1700 is activated to emit a warning sound or illuminate a warning light so that workers and site managers can recognize the situation quickly. This alarm plays an essential role in ensuring the safety of workers on site and enables immediate response within the site.

[0094] The security module 2100 is included in the network 2000 and is responsible for securely protecting data transmitted from the 360-degree streaming device 1000 to the web server 2200. The security module 2100 prevents external attacks and data leaks through firewall and encryption technology, and maintains integrity and confidentiality by encrypting all data transmitted over the network.

[0095] This allows remote users accessing the system to securely view data and monitor streaming video, while simultaneously minimizing security risks by blocking attempts to illegally manipulate or steal data. During data transmission, the security module 2100 encrypts and processes the data, protecting it so that it can reach the client terminal 2300 securely.

[0096] The web server 2200 is responsible for transmitting and streaming 360-degree video data and environmental data processed by the embedded module 1300 and data analysis unit 1400 to remote users in real time via a web page. This invention maximizes efficiency because it is immediately accessible on various devices such as smartphones, tablets, and PCs without the need for separate dedicated equipment. Furthermore, it is easy to use and highly accessible, as it can be accessed using only a web browser without the need for app installation.

[0097] The web server 2200 is designed to securely receive data collected at the work site via the encryption module 1500 and provide it to many clients simultaneously. Data transmitted to the web server 2200 is protected through the security module 2100, which includes a firewall, and all data is securely encrypted. This minimizes security risks when accessing the data from external sources.

[0098] First, the web server receives 360-degree video data processed by the embedded module 1300 and data analyzed by the AI ​​computation module 1410 and sensor analysis module 1420 in real time. At this time, the data is protected via the encryption module 1500, and the web server 2200 securely processes it and transmits it to the remote user's client terminal 2300. The client terminal 2300 may include a PC, smartphone, tablet, etc. Users can connect to the web server via a web browser or dedicated application and view 360-degree video in real time at various resolutions such as 1K, 2K, and 4K.

[0099] The web server 2200 not only streams data but also transmits abnormal values ​​and danger signals detected by the AI ​​computing module 1410 and the sensor analysis module 1420. For example, if a worker enters a hazardous area or if the gas concentration exceeds a dangerous level, the corresponding warning signal is transmitted in real time to a remote user via the web server 2200. This supports not only on-site warning notification but also the ability to recognize warnings and respond immediately from a remote location.

[0100] Furthermore, the 2200 web server is designed to provide a stable streaming environment that allows numerous users to connect simultaneously, and to process data smoothly without user conflicts. It enables efficient system operation with minimal personnel, real-time monitoring of work sites regardless of physical distance, and relatively low-cost construction and operation over large geographical areas. Data can be reviewed and addressed in a secure environment, and feature additions and updates can be easily implemented.

[0101] The client terminal 2300 is a device that receives 360-degree video data and environmental data transmitted in real time from the web server 2200, and may include a variety of devices such as PCs, smartphones, and tablets. This device connects to the web server via a web browser or dedicated application, allowing the user to monitor the conditions of the work site in real time and respond to warning signals as needed.

[0102] In particular, the client terminal 2300 can be accessed using only a web browser without the need to install any applications, allowing for easy connection to the system without the installation of any additional software, thus offering excellent accessibility and ease of use. Through the client terminal, users can view streaming video in real time at various resolutions such as 1K, 2K, and 4K, and can also view abnormal values ​​and danger signals detected by the AI ​​calculation module 1410 and the sensor analysis module 1420 in real time.

[0103] The client terminal 2300 provides a smooth streaming environment even when many users connect simultaneously, enabling monitoring of the work site and rapid response from anywhere.

[0104] The scope of protection in this field is not limited to the descriptions and expressions of the embodiments explicitly described above. Furthermore, it is reiterated that the scope of protection of the present invention is not limited by obvious modifications or substitutions in the art to which the present invention pertains. [Explanation of symbols]

[0105] 1000 ···360-degree streaming devices 1100 ···360-degree camera 1200 ···Environmental Sensor Module 1300 ···Embedded Module 1400 ···Data Analysis Department 1410 ···AI Computation Module 1420 ···Sensor Analysis Module 1500 ···Encryption Module 1600 ···Touch display 1700 ··· Alarm Module 2000 ···Network 2100 ···Security Module 2200 ···Web server 2300 ···Client terminal

Claims

1. A 360-degree camera that captures 360-degree video of the work site, An environmental sensor module that measures environmental data in real time, including at least one of the following: gas concentration, temperature, humidity, and pressure data of the work environment, An embedded module that processes the video of the work site obtained from the 360-degree camera, It includes a data analysis unit that analyzes the video and environmental data processed from the embedded module, A 360-degree video real-time streaming and environmental analysis system characterized by providing data analyzed by the aforementioned data analysis unit to field workers or remote managers in real time, thereby supporting safety management and response.

2. The aforementioned data analysis unit, An AI computing module that tracks workers in video processed from the embedded module and performs real-time AI analysis on worker data including at least one of face detection, human tracking, respiration and pulse analysis, and area intrusion detection. The 360-degree video real-time streaming and environmental analysis system according to claim 1, characterized in that it includes a sensor analysis module that analyzes environmental data received from the environmental sensor module in real time.

3. The 360-degree video real-time streaming and environmental analysis system according to claim 2, further comprising a web server that receives the data analyzed by the data analysis unit and provides it in real time to a remote user together with the video processed by the embedded module.

4. Includes an alarm module that provides a warning notification signal so that workers or managers can visually or audibly recognize the warning, The 360-degree video real-time streaming and environmental analysis system according to claim 3, characterized in that the data analysis unit analyzes the worker data and the environmental data in real time to generate a warning notification signal, transmits it to the alarm module, and the alarm module transmits the warning notification signal to the web server.

5. The 360-degree video real-time streaming and environmental analysis system according to claim 4, characterized in that the AI ​​computation module detects the face of a worker in the video processed from the embedded module, tracks the worker's state through human recognition, senses abnormal behavior or dangerous elements of the worker through image analysis, and generates the warning notification signal.

6. The 360-degree video real-time streaming and environmental analysis system according to claim 5, characterized in that the AI ​​calculation module analyzes subtle hue changes occurring in the worker's face in the video processed from the embedded module to sense changes in blood volume pulse (BVP), and thereby estimates the worker's pulse state.

7. The 360-degree video real-time streaming and environmental analysis system according to claim 6, characterized in that the AI ​​calculation module senses the worker's movements using a motion branch and removes the influence of those movements on the analysis of biological information.

8. The 360-degree video real-time streaming and environmental analysis system according to claim 6, characterized in that the AI ​​calculation module tracks changes in the worker's skin hue over a certain period of time using an Appearance Branch to extract changes in blood flow and breathing patterns.

9. The 360-degree video real-time streaming and environmental analysis system according to claim 5, characterized in that the AI ​​calculation module selects the main areas to be analyzed using an attention mask and filters out irrelevant areas to improve the accuracy of biometric information estimation.

10. The 360-degree video real-time streaming and environmental analysis system according to claim 5, characterized in that the AI ​​calculation module measures the respiratory cycle based on zero crossing in the breath wave and analyzes the worker's respiratory state in real time.

11. The 360-degree video real-time streaming and environmental analysis system according to claim 5, characterized in that the AI ​​calculation module calculates the stress level (stress value) based on the worker's biometric information and generates a warning notification signal when the stress exceeds a set value.

12. The 360-degree video real-time streaming and environmental analysis system according to claim 4, characterized in that the sensor analysis module generates a warning notification signal based on the environmental data when the environmental data value exceeds a predetermined critical value and transmits it to the alarm module.

13. The 360-degree video real-time streaming and environmental analysis system according to claim 3, characterized in that it includes an encryption module that encrypts worker data and environmental data analyzed by the AI ​​calculation module and the sensor analysis module, and then transmits them to the web server.

14. A method for real-time streaming of 360-degree video and environmental analysis, performed via a system including a 360-degree camera, an environmental sensor module, an embedded module, and a data analysis unit, a) The 360-degree camera obtains a 360-degree image of the work site, b) The environmental sensor module measures environmental data in real time, including at least one of the following: gas concentration, temperature, humidity, and pressure data of the work environment. c) The embedded module processes the video of the work site obtained from the 360-degree camera, d) The data analysis unit analyzes the video and environmental data processed from the embedded module, e) A method for real-time streaming of 360-degree video and environmental analysis, comprising the step of providing the data analyzed by the data analysis unit to field workers or remote managers in real time to support them in carrying out safety management and response.

15. Step d) above is, The AI ​​computing module tracks the worker within the video processed by the embedded module and performs real-time AI analysis on the worker data, including at least one of face detection, human tracking, respiration and pulse analysis, and area intrusion detection. The method for real-time streaming of 360-degree video and environmental analysis according to claim 14, characterized in that the sensor analysis module analyzes environmental data received from the environmental sensor module in real time.

16. Step e) above is, The method for real-time streaming of 360-degree video and environmental analysis according to claim 15, characterized in that it includes the step of a web server receiving data analyzed by the data analysis unit and providing it in real time to a remote user together with video processed by the embedded module.

17. Step d) above is, The data analysis unit analyzes the worker data and the environmental data in real time, generates a warning notification signal and transmits it to the alarm module, and the alarm module transmits the warning notification signal to the web server. Step e) above is, The method for real-time streaming of 360-degree video and environmental analysis according to claim 16, further comprising the step of providing a warning notification signal so that the alarm module can visually or audibly recognize the warning.

18. Step d) above is, The method for real-time streaming of 360-degree video and environmental analysis according to claim 17, characterized in that the AI ​​computing module includes the steps of detecting a worker's face in the video processed from the embedded module, tracking the worker's state through human recognition, sensing abnormal behavior or dangerous elements of the worker through image analysis, and generating the warning notification signal.

Images