Remote control command method and system based on panoramic video

Abstract

The invention discloses a remote control command method and system based on a panoramic video, and relates to the technical field of monitoring and risk control, and the method comprises the steps: dividing an open-air working area, obtaining an area boundary, configuring a high-definition camera according to the area boundary, and obtaining an area image collection device; the device is connected to a panoramic video fusion system, during construction, the system receives and aligns region video streams to generate a real-time panoramic monitoring video, pre-constructs an operation risk identification network, transmits the video to obtain a risk identification result and a region identifier, traverses a risk coefficient of an operation risk table by using the result and the region, and executes remote management and control command according to the risk coefficient. The technical problems of incomplete, inaccurate and untimely information and difficulty in effectively supporting remote scientific decision in existing open-air working area monitoring and risk management and control are solved, and the technical effects of improving scientificity and effectiveness of remote management and control command and guaranteeing operation safety and high efficiency to the greatest extent are achieved.

Description

Technical Field

[0001] The present application relates to battery management related fields, and in particular to a remote control and command method and system based on panoramic video. Background Art

[0002] In the operation scenario of open-air work areas, due to the vast working area and complex and changeable working environment, efficient monitoring and risk control, and accurate adaptation to the control needs of different areas have become the key points to ensure the normal operation of open-air work areas. Traditional open-air work area monitoring and risk control methods have many shortcomings. For example, relying only on a single camera or local monitoring, it is difficult to achieve comprehensive coverage of the entire work area, the overall situation of the work area is not fully grasped, and there is a lack of in-depth analysis and application of past operation data. It is difficult to accurately assess the risk status of each area. There are unreasonable aspects in resource allocation, resulting in insufficient control resources in key areas and excess resources in some areas. The formulation of control strategies is relatively simple and rigid, and it is unable to effectively respond to the changeable actual working conditions in open-air work areas, resulting in incomplete, inaccurate, and untimely information, and it is difficult to effectively support remote scientific decision-making.

[0003] Among the current relevant technologies, there are technical problems such as incomplete, inaccurate, and untimely information in open-air work area monitoring and risk management, as well as difficulty in effectively supporting remote scientific decision-making. Summary of the invention

[0004] The present application provides a remote control and command method and system based on panoramic video, which divides the open-air work area, obtains the regional boundaries, configures high-definition cameras accordingly, obtains regional image acquisition devices, and connects the devices to a panoramic video fusion system. During construction, the system receives and aligns the regional video streams to generate real-time panoramic monitoring videos, pre-builds an operation risk identification network, transmits videos to obtain risk identification results and regional identifications, uses the results and the risk coefficients of the operation risk table of the region to perform remote control and command according to the risk coefficients, and provides comprehensive, accurate, timely and quantitative monitoring and risk control support for the open-air work area, achieving the technical effect of improving the scientificity and effectiveness of remote control and command, and ensuring the safety and efficiency of operations to the greatest extent.

[0005] The present application provides a remote control and command method based on panoramic video, including:

[0006] Interactive open-air work area is divided into areas to obtain multiple area boundaries of multiple open-air work areas; high-definition cameras are configured according to the multiple area boundaries to obtain multiple area image acquisition devices; the multiple area image acquisition devices are connected to a panoramic video fusion system; during the construction of the open-air work area, multiple area video streams of the multiple area image acquisition devices are received and aligned via the panoramic video fusion system to generate a real-time panoramic monitoring video; a pre-constructed operation risk identification network is constructed, and the real-time panoramic monitoring video is transmitted to the operation risk identification network to obtain a real-time risk identification result, wherein the real-time risk identification result has a real-time operation area identifier; the real-time risk identification result and the real-time operation area are used to traverse a pre-constructed operation risk table to obtain a real-time risk coefficient; remote management and control is performed according to the real-time risk coefficient.

[0007] The present application also provides a remote control and command system based on panoramic video, including:

[0008] An area boundary acquisition module, the area boundary acquisition module is used to interactively divide the open-air work area into areas and obtain multiple area boundaries of multiple open-air work areas; an image acquisition device acquisition module, the image acquisition device acquisition module is used to configure high-definition cameras according to the multiple area boundaries and obtain multiple area image acquisition devices; a connection module, the connection module is used to connect the multiple area image acquisition devices to a panoramic video fusion system; a panoramic monitoring video generation module, the panoramic monitoring video generation module is used to receive and align multiple area video streams of the multiple area image acquisition devices via the panoramic video fusion system during the construction of the open-air work area to generate a real-time panoramic monitoring video; an identification module, the identification module is used to pre-construct an operation risk identification network, transmit the real-time panoramic monitoring video to the operation risk identification network to obtain a real-time risk identification result, wherein the real-time risk identification result has a real-time operation area identifier; a risk coefficient acquisition module, the risk coefficient acquisition module is used to use the real-time risk identification result and the real-time operation area to traverse a pre-constructed operation risk table to obtain a real-time risk coefficient; a command and control module, the command and control module is used to perform remote control and command according to the real-time risk coefficient.

[0009] The present application proposes a remote control and command method and system based on panoramic video. First, the open-air work area is interactively divided into regions to obtain multiple region boundaries of multiple open-air work areas. High-definition cameras are configured according to the multiple region boundaries to obtain multiple region image acquisition devices. The multiple region image acquisition devices are connected to a panoramic video fusion system. During the construction of the open-air work area, the panoramic video fusion system receives and aligns multiple region video streams of the multiple region image acquisition devices to generate real-time panoramic monitoring video, pre-constructs an operation risk identification network, and transmits the real-time panoramic monitoring video to the operation risk identification network to obtain a real-time risk identification result, wherein the real-time risk identification result has a real-time operation area identifier. The real-time risk identification result and the real-time operation area are used to traverse the pre-constructed operation risk table to obtain a real-time risk coefficient. Remote control and command are performed according to the real-time risk coefficient, thereby achieving the technical effect of improving the scientificity and effectiveness of remote control and command and ensuring operation safety and efficiency to the greatest extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solution of the embodiment of the present invention, the accompanying drawings of the embodiment of the present invention will be briefly introduced below. A flow chart is used in the present application to illustrate the operations performed by the system according to the embodiment of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps can be processed in reverse order or simultaneously as needed. At the same time, other operations can also be added to these processes, or one or more operations can be removed from these processes.

[0011] Figure 1 A flowchart of a remote control and command method based on panoramic video provided in an embodiment of the present application;

[0012] Figure 2 A schematic diagram of the structure of a remote control and command system based on panoramic video provided in an embodiment of the present application.

[0013] Explanation of the accompanying drawings: area boundary acquisition module 10, image acquisition device acquisition module 20, connection module 30, panoramic monitoring video generation module 40, identification module 50, risk coefficient acquisition module 60, command and control module 70. DETAILED DESCRIPTION

[0014] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below.

[0015] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0016] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict, and the terms "first\second" involved are merely to distinguish similar objects and do not represent a specific ordering of objects. The terms "including" and "having" and any variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or modules that are not clearly listed or inherent to these processes, methods, products, or devices. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by technicians in the technical field of this application. The terms used herein are for the purpose of describing the embodiments of the present application only.

[0017] The present application embodiment provides a remote control and command method based on panoramic video, such as Figure 1 As shown, the method includes:

[0018] Step S100, dividing the interactive open-air working area into regions to obtain multiple region boundaries of multiple open-air working areas.

[0019] Specifically, the open-air work area division refers to interactively determining the boundaries of multiple operating areas in the open-air work area. Select the open-air work area to be divided, convene relevant personnel, such as project planners, construction managers, safety supervisors, etc., set considerations for regional division, such as the nature of work, topography, functional requirements, etc., perform regional division operations, and obtain multiple regional boundaries. The determination of multiple regional boundaries represents the process of reasonably dividing the open-air work area into different parts, which is characterized by precise division based on multiple factors. Continue to improve the regional division operation, evaluate and adjust the division results, and obtain the final accurate multiple regional boundaries. These regional boundaries represent the boundaries that clearly define the scope of each operating area. Record relevant information of the entire regional division process.

[0020] Step S200: configuring high-definition cameras according to the multiple area boundaries to obtain multiple area image acquisition devices.

[0021] Specifically, the high-definition camera configuration refers to the arrangement of cameras according to the boundaries of multiple open-air work areas to obtain multiple regional image acquisition devices. Select the open-air work area where the camera is to be configured, preset the panoramic monitoring height constraint, interactively obtain the standard parameter information of the standard camera, set relevant parameters such as camera viewing angle, resolution, focal length, etc., perform analysis operations, and obtain the standard field of view. Based on multiple regional boundaries and standard fields of view, analyze and determine the camera deployment strategy. For a certain regional boundary, determine whether it falls into the standard field of view. If it falls, locate the center of the regional boundary as the camera deployment point; if it does not fall, synchronize it to the pre-built camera deployment analysis network to obtain the deployment analysis results. Process multiple regional boundaries in this way to determine all camera deployment strategies. Configure the high-definition camera according to the deployment strategy to obtain multiple regional image acquisition devices. Record relevant information of the entire high-definition camera configuration process.

[0022] In a possible implementation, high-definition cameras are configured according to the multiple regional boundaries to obtain multiple regional image acquisition devices, and step S200 further includes step S210, presetting panoramic monitoring height constraints. Specifically, the preset panoramic monitoring height constraint refers to the restriction condition on the monitoring height that is pre-set before the high-definition camera configuration is performed, the open-air working area to be monitored is selected, relevant measurement and analysis tools are prepared, specific constraint values ​​such as maximum monitoring height, minimum monitoring height, and acceptable height deviation are set, and the height constraint setting operation is performed to obtain a clear panoramic monitoring height restriction range. This panoramic monitoring height constraint range represents the reasonable height range that the camera can reach under specific monitoring requirements. Its characteristic is that the camera installation height is strictly defined. Based on this constraint range, standard parameter information of standard cameras is interactively obtained, such as the camera's viewing angle, resolution, focal length, etc. Continuously combine the boundaries of multiple open-air working areas, analyze and determine the camera's deployment strategy, and record relevant information of the entire preset panoramic monitoring height constraint process.

[0023] Step S220, interactively obtain standard parameter information of the standard camera, and obtain the standard field of view according to the panoramic monitoring height constraint and the standard parameter information analysis. Specifically, the standard field of view acquisition process refers to interactively obtaining the parameter information of the standard camera, and analyzing the panoramic monitoring height constraint and these parameters to determine the field of view range. Select the monitoring scene to be analyzed for the field of view, and interactively obtain the standard parameter information of the standard camera such as viewing angle, resolution, focal length, etc. by communicating with relevant equipment suppliers, consulting technical information, etc., and prepare the data basis for analysis. Then, combined with the pre-set panoramic monitoring height constraint, set specific values ​​such as maximum monitoring height and minimum monitoring height, perform analysis operations, and use professional mathematical algorithms and models to comprehensively process the obtained parameter information and height constraints. The obtained standard field of view represents the area range that the camera can effectively monitor under specific conditions, and its characteristics are that the restrictions on camera parameters and monitoring height are comprehensively considered. Continue to evaluate and optimize the analysis results to ensure the accuracy and effectiveness of the field of view. Record the relevant information of the entire standard field of view acquisition process.

[0024] Step S230 obtains multiple camera deployment strategies based on the analysis of the multiple area boundaries and the standard field of view. Specifically, the camera deployment strategy acquisition process refers to a series of deployment plans determined based on the analysis of multiple area boundaries and the standard field of view, selecting an open-air working area for deployment strategy analysis, and obtaining relevant data of multiple area boundaries, including shape, size, position and other information. At the same time, clarify the determined standard field of view range and related parameters. For the first area boundary, determine whether it falls into the standard field of view. If it does, locate the center of the area boundary and use it as the deployment strategy for the first camera, because the area center can better achieve comprehensive monitoring. If it does not fall into the standard field of view, it will be transmitted to the pre-built camera deployment analysis network. This network will Using complex algorithms and models, and taking into account various characteristics of regional boundaries and surrounding environmental factors, one or more alternative deployment plans are given. The alternative plans are evaluated and screened from multiple aspects such as monitoring effect, cost, and installation difficulty. The optimal plan is selected as the first camera deployment strategy, and other regional boundaries are handled in the same way. The relationship with the standard field of view is judged in turn, and the respective camera deployment strategies are determined. Finally, through comprehensive and detailed analysis, the situations of multiple regional boundaries and standard fields of view are combined to obtain multiple camera deployment strategies that can effectively achieve the monitoring objectives, and the relevant information of the entire camera deployment strategy acquisition process is recorded.

[0025] Step S240 configures the high-definition camera according to the multiple camera deployment strategies to obtain the multiple regional image acquisition devices. Specifically, the multiple regional image acquisition device acquisition process refers to the result obtained by configuring the high-definition camera according to the multiple camera deployment strategies, selecting the open-air working area where the image acquisition device is to be configured, sorting out multiple determined camera deployment strategies, which include detailed information such as the installation position, angle, focal length, etc. of each camera, and then, according to these strategies, selecting the appropriate high-definition camera model, considering the camera's resolution, frame rate, low-light performance and other parameters to meet the image quality requirements of the monitoring area, and professional technicians implement the installation operation of the high-definition camera according to the position in the deployment strategy, using appropriate installation tools and fixtures to ensure that the camera is installed firmly, and during the installation process, adjust the angle and focal length of the camera so that it is accurately aligned with the predetermined monitoring area to achieve the best monitoring effect, and after the installation is completed, connect and debug each camera, connect the camera to the data transmission line properly, and ensure that the image signal can be transmitted stably and smoothly, and after this series of configuration and debugging operations, finally successfully obtain multiple devices that can effectively collect images of each area.

[0026] In a possible implementation, multiple camera deployment strategies are obtained based on the multiple region boundaries and the standard field of view, and step S230 further includes step S231, determining whether a first region boundary falls within the standard field of view, wherein the first region boundary is any one of the multiple region boundaries. Specifically, a first region boundary is selected from the multiple region boundaries for determination to see whether it falls within the standard field of view, and a range of the region boundary is accurately compared with a range of the standard field of view.

[0027] Step S232: If the first area boundary falls within the standard field of view, the first area center of the first area boundary is located, and the first area center is used as the first camera deployment strategy. Specifically, if the first area boundary falls within the standard field of view, in order to achieve a good monitoring effect, the center position of the area boundary will be located, because the area center can usually cover the entire area to the greatest extent, so the first area center is determined as the first camera deployment strategy.

[0028] Step S233: If the boundary of the first area does not fall within the standard field of view, the boundary of the first area is synchronized to the pre-constructed camera deployment analysis network to obtain a first deployment analysis result, and the first deployment analysis result is used as the first camera deployment strategy. Specifically, if the boundary of the first area does not fall within the standard field of view, it is synchronously transmitted to the pre-constructed camera deployment analysis network, and a variety of factors such as the shape of the area, the surrounding environment, and the focus of monitoring are comprehensively considered to obtain the first deployment analysis result, and the first deployment analysis result is the optimal camera deployment strategy for the area, which is used as the first camera deployment strategy.

[0029] Step S234, and so on, the multiple camera deployment strategies are obtained according to the multiple regional boundaries and the standard field of view. Specifically, according to the same method and steps, other regions in the multiple regional boundaries are processed in turn, and the above judgment, positioning or analysis process is repeated continuously. According to the relationship between each regional boundary and the standard field of view, the corresponding camera deployment strategies are determined respectively, and finally, according to the multiple regional boundaries and the standard field of view, multiple camera deployment strategies suitable for each region can be successfully obtained, thereby achieving effective monitoring coverage of the entire area.

[0030] In a possible implementation, if the first area boundary does not fall within the standard field of view, the first area boundary is synchronized to the pre-built camera deployment analysis network to obtain a first deployment analysis result, and the first deployment analysis result is used as the first camera deployment strategy, and step S233 further includes step S2331, interactively obtaining multiple sample fields of view, multiple sample area boundaries, and multiple sample deployment strategies. Specifically, through an interactive method, multiple sample fields of view, multiple sample area boundaries, and multiple sample deployment strategies are widely collected, and the sample data comes from various actual monitoring scenarios and previous deployment experience, and is rich in diversity and representativeness.

[0031] Step S2332, obtain K data analysis models from the random operator library. Specifically, obtain K data analysis models from the random operator library, each of which has different characteristics and advantages. For example, some models are good at processing large-area fields of view, some models are more accurate in analyzing the boundaries of complex-shaped regions, some models focus on considering the impact of environmental factors on deployment strategies, some models focus on cost optimization, and some models comprehensively consider multiple factors.

[0032] Step S2333, using the multiple sample fields of view, multiple sample area boundaries and multiple sample layout strategies as training data to perform output optimization of the K data analysis models, and obtain K camera layout analysis branches. Specifically, a large amount of sample data obtained is input into each of the above models as training data, and each model will learn and optimize the output based on these data. For example, when processing field of view data, model 1 will calculate the deviation between the area of ​​each sample field of view and the average area, and by continuously adjusting the internal parameters, the deviation between the predicted field of view area and the field of view area of ​​the actual sample is gradually reduced. After multiple iterations of training, model 1 can output more accurate field of view analysis results. Other models will also process and optimize the input data in a similar manner according to their respective characteristics and priorities, and after their respective training processes, obtain optimized camera layout analysis branches.

[0033] Step S2334, connect the K camera deployment analysis branches in parallel to generate the camera deployment analysis network. Specifically, connect the camera deployment analysis branches in parallel, assign weights to each branch, and calculate the final comprehensive result by weighted average. By connecting in parallel, a powerful camera deployment analysis network is generated, which can provide accurate and effective camera deployment strategies for various complex monitoring scenarios.

[0034] In a possible implementation, step S233 further includes step S2335, synchronizing the first area boundary to a pre-built camera layout analysis network, and obtaining K candidate layout analysis results through the camera layout analysis network analysis. Specifically, when it is determined that the first area boundary does not fall within the standard field of view, the area boundary is synchronously transmitted to the pre-built camera layout analysis network, and the network can analyze the input area boundary data. Within the network, various factors such as the shape, size, surrounding environment, and monitoring requirements of the area are comprehensively considered to generate K candidate layout analysis results for the first area boundary.

[0035] Step S2336, perform repetition frequency analysis on the K candidate layout analysis results to obtain the first layout analysis result. Specifically, perform repetition frequency analysis on the K candidate layout analysis results, count the number of occurrences of each candidate result under different conditions or in different simulation operations, and through frequency statistics and analysis, understand the probability and stability of each candidate result. Finally, based on the repetition frequency analysis results, determine the candidate result with the highest frequency of occurrence or the most stable performance under specific conditions, and determine it as the first layout analysis result. The first layout analysis result is the optimal camera layout strategy for the first area boundary. By inputting the area boundary data into a special analysis network, multiple candidate results are generated, and these candidate results are subjected to in-depth repetition frequency analysis, so as to obtain the most suitable first layout analysis result.

[0036] Step S300: connecting the multiple regional image acquisition devices to a panoramic video fusion system.

[0037] Specifically, prepare the hardware equipment and cables required for connection, determine the output interface type of each regional image acquisition device, select the appropriate cable according to the interface type to connect the regional image acquisition device to the corresponding input interface of the panoramic video fusion system, and connect multiple regional image acquisition devices one by one in a certain order or number. You can start with the device that is closer to the panoramic video fusion system, or connect according to the importance or priority of the area. After the connection is completed, each connection needs to be checked and tested. By observing the system indicator light, checking the system device connection status display, or performing a simple image acquisition test, confirm whether the connection is successful and whether the image signal can be transmitted normally. When all regional image acquisition devices are successfully connected to the panoramic video fusion system and confirmed to be correct after testing.

[0038] Step S400, during the construction process of the open-air work area, the panoramic video fusion system receives and aligns multiple regional video streams of the multiple regional image acquisition devices to generate a real-time panoramic monitoring video.

[0039] Specifically, when construction is in progress in an open-air work area, the panoramic video fusion system is in a startup and running state, ready to receive video stream data from multiple regional image acquisition devices. Each regional image acquisition device begins to acquire video images of its area, and encodes and transmits these video streams in a specific format and protocol. The panoramic video fusion system receives video stream signals from multiple regional image acquisition devices. Since the positions, angles, and acquisition times of different regional image acquisition devices may be different, the received video streams may not be aligned in time and space. The system analyzes and processes the video streams, for example, by extracting key feature points in the video, such as the edges of buildings and roads. The system uses the time stamp information of the video stream to synchronize the videos in different areas to ensure their consistency on the timeline. After completing the space and time calibration, the system fuses the video streams of multiple areas, involving image stitching, processing of overlapping areas, and adjustment of color and brightness, so that the transition of the fused video images is natural and there is no obvious stitching trace. By continuously performing the above processing and fusion operations on the received video streams, the system finally generates a real-time panoramic monitoring video. The video can comprehensively and clearly display the real-time situation of each area of ​​the open-air work area, providing intuitive and accurate information for construction management and monitoring.

[0040] Step S500, pre-constructing an operation risk identification network, transmitting the real-time panoramic monitoring video to the operation risk identification network to obtain a real-time risk identification result, wherein the real-time risk identification result has a real-time operation area identification.

[0041] Specifically, before the implementation of the plan, it is first necessary to use a large amount of relevant data and advanced machine learning algorithms to pre-build an operation risk identification network. The data includes various types of construction scene videos, known risk cases and corresponding feature information. When a real-time panoramic monitoring video is generated during the construction of the open-air work area, this real-time panoramic monitoring video is transmitted to the pre-built operation risk identification network through a high-speed and stable data transmission channel. After the operation risk identification network receives the real-time panoramic monitoring video, it immediately starts the analysis and processing program to perform detailed feature extraction and analysis on each frame of the video, including but not limited to personnel behavior, equipment operation status, environmental changes, etc. Using its internal complex neural network structure and calculation model, the extracted features are compared and judged with the pre-trained risk model. For each area in the video, a comprehensive evaluation and analysis will be carried out, and finally a real-time risk identification result will be obtained. The result can not only point out the current risk type, but also identify the specific area where the risk is located, that is, with real-time operation area identification, it can timely and accurately discover potential risks in the construction process of the open-air work area and clarify its specific location, providing strong support for taking corresponding prevention and response measures.

[0042] In a possible implementation, step S500 further includes step S510, interactively obtaining multiple sample risk monitoring videos of multiple operational risk behaviors, wherein the multiple sample risk monitoring videos have multiple sample risk behavior type identifiers. Specifically, through multiple interactive channels, such as cooperation with various construction companies, safety regulatory agencies, scientific research institutions, or collection through network platforms, multiple sample risk monitoring videos of multiple operational risk behaviors are widely obtained. While obtaining these videos, each video is equipped with multiple sample risk behavior type identifiers. These identifiers are marked by professionals after careful observation and analysis, and clearly and accurately indicate the specific risk behavior types contained in the video.

[0043] Step S520, construct the operation risk identification network based on the multiple sample risk monitoring videos and multiple sample risk behavior types. Specifically, the operation risk identification network is constructed using the multiple sample risk monitoring videos with clear identification and the corresponding sample risk behavior types. The construction process involves using advanced machine learning algorithms and deep learning technologies to pre-process the sample videos, including image enhancement, denoising and other operations, and then extracting key features in the video, such as personnel action features, environmental features, equipment status features, etc. Based on these features and the corresponding risk behavior type identification, the neural network model is trained and the parameters of the model are continuously adjusted so that it can accurately learn and identify different types of operation risk behavior patterns.

[0044] Step S530, the real-time panoramic monitoring video is transmitted to the operation risk identification network to obtain real-time risk identification results. Specifically, during the construction process of the open-air work area, real-time panoramic monitoring video is continuously generated, and this real-time video is transmitted to the already constructed operation risk identification network through a high-speed data transmission channel. After the network receives the real-time video, it quickly analyzes and processes each frame of the image, compares and judges it with the risk behavior pattern learned before, and thus obtains real-time risk identification results.

[0045] Step S540, reversely locate the real-time operation area in the panoramic video fusion system according to the real-time risk identification result. Specifically, reverse positioning is performed in the panoramic video fusion system according to the obtained real-time risk identification result, and the real-time operation area where the risk occurs is accurately determined by analyzing the relevant features and parameters in the risk identification result, combining the position and feature information of each area in the panoramic video, and using specific algorithms and logic. It can timely and accurately discover and locate the risk area in the construction process, providing a key basis for taking corresponding safety measures and intervention means, and forming a complete operation risk monitoring and response system by acquiring samples through multiple channels, building a powerful identification network, transmitting and analyzing videos in real time, and accurately locating risk areas.

[0046] Step S600, using the real-time risk identification result and the real-time operation area to traverse a pre-constructed operation risk table to obtain a real-time risk coefficient.

[0047] Specifically, a work risk table is pre-constructed. The table should include various work risk behaviors and their corresponding risk factors. The risk factors can be determined based on historical data, expert evaluations or other relevant information. During the construction process in the open-air work area, the panoramic video fusion system receives and aligns multiple regional video streams of multiple regional image acquisition devices to generate real-time panoramic monitoring videos. The real-time panoramic monitoring videos are transmitted to the work risk identification network to obtain real-time risk identification results. The real-time risk identification results should have real-time work area identification for subsequent positioning. According to the real-time work area identification in the real-time risk identification results, the real-time work area is reversely located in the panoramic video fusion system. Based on the real-time risk identification results and real-time operation areas, traverse the pre-built operation risk table, find the corresponding risk coefficient by matching the real-time operation area with the risk behavior in the operation risk table, and obtain the corresponding real-time risk coefficient from the operation risk table according to the matching result. The coefficient reflects the risk level of the current operation area. According to the real-time panoramic monitoring video and the operation risk table, the real-time risk coefficient can be obtained quickly and accurately, which provides an important basis for construction safety management. Construction management personnel can take corresponding measures according to the real-time risk coefficient to reduce risks and ensure construction safety. With the accumulation and analysis of more data, the operation risk table can be continuously optimized to improve the accuracy and reliability of risk assessment.

[0048] In a possible implementation, step S600 further includes step S610, interactively obtaining multiple standard risk coefficients of multiple standard risk behavior types in multiple open-air working areas. Specifically, multiple standard risk coefficients of multiple standard risk behavior types in multiple open-air working areas are obtained through various interactive methods, such as cooperating with professional safety assessment agencies, communicating with management departments of multiple open-air working areas, collecting relevant industry reports and research results, etc. In the acquisition process, it involves the sorting and screening of a large amount of data. For example, risk coefficient information about multiple open-air working areas such as mining, construction, road maintenance, etc. is collected from different data sources, including but not limited to coefficient values ​​corresponding to multiple standard risk behavior types such as falling from heights, being hit by objects, and mechanical injuries.

[0049] Step S620, construct and generate the operation risk table based on the multiple open-air operation areas, multiple standard risk behavior types and multiple standard risk coefficients. Specifically, the operation risk table is constructed based on the collected multiple open-air operation areas, multiple standard risk behavior types and multiple standard risk coefficients, and the data is structured and classified. For example, the data is classified according to different open-air operation areas, such as classifying the risk coefficients related to mining operations into one category and those related to construction into another category. Under each category, it is further subdivided according to the standard risk behavior type, such as listing the coefficients of the risk of falling from height in mining and construction separately.

[0050] Step S630, using the real-time risk identification result and the real-time operation area to traverse the pre-constructed operation risk table to obtain the real-time risk coefficient. Specifically, after the operation risk table is constructed, when the real-time risk identification result and the real-time operation area are obtained, start traversing the pre-constructed operation risk table, determine the corresponding classification in the table according to the real-time operation area, and search for entries matching the risk behavior type in the real-time risk identification result under the classification. Assuming that the real-time operation area is a certain construction site, and the real-time risk identification result is a "fall from height" risk, then search for the standard risk coefficient corresponding to the "fall from height" risk behavior type under the construction classification, so as to obtain the final real-time risk coefficient, effectively integrate and utilize complex risk information, and provide a reliable basis for accurately assessing real-time risks.

[0051] Step S700, executing remote control and command according to the real-time risk coefficient.

[0052] Specifically, after obtaining the real-time risk coefficient, a dedicated risk assessor will interpret and analyze the coefficient and compare the real-time risk coefficient with the preset risk threshold. If the real-time risk coefficient is lower than the risk threshold, it means that the current operation risk is at a low level. The remote control and command center may maintain normal monitoring status, but still maintain a certain level of attention. If the real-time risk coefficient is between the low risk threshold and the high risk threshold, it indicates that there is a certain degree of risk. The remote control and command personnel will immediately issue an early warning message to the on-site workers, possibly through communication tools such as walkie-talkies and mobile applications, to remind them of potential risks and require them to strengthen self-protection and standardize operations. If the real-time risk coefficient exceeds the high risk threshold, it means that the situation is critical, and the remote control and command center will quickly take a series of emergency measures. For example, immediately stop related high-risk operations and instruct on-site personnel to evacuate along the predetermined emergency evacuation routes. At the same time, mobilize emergency rescue resources, such as rescue vehicles and medical teams, to prepare for possible accidents. During the entire process, remote control and command personnel will continue to track changes in real-time risk factors, dynamically adjust control strategies and command instructions, maintain close communication with on-site operation managers, and promptly understand the actual situation on site to ensure the effective implementation of control measures. The remote control and command center will also record relevant risk information and measures taken for subsequent summary and analysis, providing a reference for improving future risk control work. Executing remote control and command based on real-time risk factors is a dynamic, timely and precise process designed to maximize the safety and smooth progress of open-air operations.

[0053] The embodiment of the present application divides the open-air work area, obtains the regional boundary, configures a high-definition camera accordingly, obtains a regional image acquisition device, and connects the device to a panoramic video fusion system. During construction, the system receives and aligns the regional video stream to generate a real-time panoramic monitoring video, pre-builds an operation risk identification network, transmits the video to obtain the risk identification result and the regional identification, uses the result and the risk coefficient of the operation risk table of the region to traverse, and performs remote control and command according to the risk coefficient, thereby providing comprehensive, accurate, timely and quantitative monitoring and risk control support for the open-air work area, achieving the technical effect of improving the scientificity and effectiveness of remote control and command, and maximizing the safety and efficiency of operations.

[0054] In the above, refer to Figure 1 A remote control and command method based on panoramic video according to an embodiment of the present invention is described in detail. Figure 2 A panoramic video-based remote control and command system according to an embodiment of the present invention is described.

[0055] A remote control and command system based on panoramic video according to an embodiment of the present invention is used to solve the technical problems of incomplete, inaccurate, untimely information and difficulty in effectively supporting remote scientific decision-making in existing open-air work area monitoring and risk control, so as to improve the scientificity and effectiveness of remote control and command, and maximize the technical effect of ensuring the safety and efficiency of operations. A remote control and command system based on panoramic video includes: an area boundary acquisition module 10, an image acquisition device acquisition module 20, a connection module 30, a panoramic monitoring video generation module 40, an identification module 50, a risk coefficient acquisition module 60, and a command and control module 70.

[0056] The area boundary acquisition module 10 is used for regional division of the interactive open-air working area to obtain multiple area boundaries of multiple open-air working areas;

[0057] The image acquisition device acquisition module 20 is used to configure high-definition cameras according to the multiple regional boundaries to obtain multiple regional image acquisition devices;

[0058] The connection module 30 is used to connect the multiple regional image acquisition devices to the panoramic video fusion system;

[0059] The panoramic monitoring video generation module 40 is used to receive and align multiple regional video streams of the multiple regional image acquisition devices via the panoramic video fusion system during the construction process of the open-air work area to generate a real-time panoramic monitoring video;

[0060] The identification module 50 is used to pre-build an operation risk identification network, transmit the real-time panoramic monitoring video to the operation risk identification network to obtain a real-time risk identification result, wherein the real-time risk identification result has a real-time operation area identification;

[0061] The risk coefficient acquisition module 60 is used to obtain a real-time risk coefficient by using the real-time risk identification result and the pre-constructed operation risk table of the real-time operation area traversal;

[0062] The command and control module 70 is used to perform remote control and command according to the real-time risk coefficient.

[0063] The specific configuration of the image acquisition device acquisition module 20 will be described in detail below. As described above, the area division of the interactive open-air work area is performed to obtain multiple area boundaries of multiple open-air working areas. The image acquisition device acquisition module 20 may further include: a constraint unit for presetting a panoramic monitoring height constraint; a standard field of view unit for interactively obtaining standard parameter information of a standard camera, and obtaining a standard field of view based on the panoramic monitoring height constraint and the standard parameter information analysis; a deployment strategy unit for obtaining multiple camera deployment strategies based on the multiple area boundaries and the standard field of view analysis; and an image acquisition device unit for performing high-definition camera configuration according to the multiple camera deployment strategies to obtain the multiple regional image acquisition devices.

[0064] Among them, multiple camera deployment strategies are obtained according to the multiple area boundaries and the standard field of view analysis, and the deployment strategy unit may further include: a standard field of view judgment subunit is used to judge whether the first area boundary falls into the standard field of view, wherein the first area boundary is any one of the multiple area boundaries; a first area center subunit is used to locate the first area center of the first area boundary if the first area boundary falls into the standard field of view, and use the first area center as the first camera deployment strategy; a first deployment analysis result acquisition subunit is used to synchronize the first area boundary to a pre-built camera deployment analysis network if the first area boundary does not fall into the standard field of view, obtain a first deployment analysis result, and use the first deployment analysis result as the first camera deployment strategy; multiple camera deployment strategy acquisition subunits are used to, by analogy, obtain the multiple camera deployment strategies according to the multiple area boundaries and the standard field of view analysis.

[0065] Among them, if the boundary of the first area does not fall within the standard field of view, the boundary of the first area is synchronized to the pre-built camera layout analysis network to obtain a first layout analysis result, and the first layout analysis result is used as the first camera layout strategy. The first layout analysis result acquisition subunit may further include: a sample layout strategy micro-unit is used to interactively obtain multiple sample fields of view, multiple sample area boundaries and multiple sample layout strategies; a data analysis model micro-unit is used to call and obtain K data analysis models from a random operator library; a layout analysis branch micro-unit is used to use the multiple sample fields of view, multiple sample area boundaries and multiple sample layout strategies as training data to perform output optimization of the K data analysis models to obtain K camera layout analysis branches; an analysis network micro-unit is used to connect the K camera layout analysis branches in parallel to generate the camera layout analysis network.

[0066] Among them, the first layout analysis result acquisition subunit may further include: an alternative result micro-unit is used to synchronize the first area boundary to a pre-built camera layout analysis network, and obtain K alternative layout analysis results through analysis of the camera layout analysis network; the first layout analysis result acquisition micro-unit is used to perform repetition frequency analysis on the K alternative layout analysis results to obtain the first layout analysis result.

[0067] The specific configuration of the identification module 50 will be described in detail below. As described above, a pre-constructed operation risk identification network is used to transmit the real-time panoramic monitoring video to the operation risk identification network to obtain a real-time risk identification result, wherein the real-time risk identification result has a real-time operation area identification, and the identification module 50 may further include: a risk monitoring video acquisition unit for interactively obtaining a plurality of sample risk monitoring videos of a plurality of operation risk behaviors, wherein the plurality of sample risk monitoring videos have a plurality of sample risk behavior type identifications; an operation risk identification network construction unit for constructing the operation risk identification network based on the plurality of sample risk monitoring videos and the plurality of sample risk behavior types; a real-time risk identification result acquisition unit for transmitting the real-time panoramic monitoring video to the operation risk identification network to obtain a real-time risk identification result; and a positioning unit for reversely positioning the real-time operation area in the panoramic video fusion system according to the real-time risk identification result.

[0068] The specific configuration of the risk coefficient acquisition module 60 will be described in detail below. As described above, the real-time risk coefficient is obtained by using the real-time risk identification result and the real-time operation area to traverse the pre-constructed operation risk table. The risk coefficient acquisition module 60 may further include: a standard risk coefficient acquisition unit for interactively obtaining multiple standard risk coefficients of multiple standard risk behavior types of multiple open-air operation areas; an operation risk table acquisition unit for generating the operation risk table based on the multiple open-air operation areas, multiple standard risk behavior types and multiple standard risk coefficients; a real-time risk coefficient acquisition unit for obtaining the real-time risk coefficient by using the real-time risk identification result and the real-time operation area to traverse the pre-constructed operation risk table.

[0069] A remote control and command system based on panoramic video provided in an embodiment of the present invention can execute a remote control and command method based on panoramic video provided in any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method.

[0070] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules may be used and run on the user terminal and / or server, and the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0071] The above specific implementation manner does not constitute a limitation to the protection scope of the present application. It should be understood by those skilled in the art that various modifications, combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application should be included in the protection scope of the present application. In some cases, the actions or steps recorded in the present application can be performed in an order different from that in the embodiment and can still achieve the desired results. In addition, the process depicted in the accompanying drawings does not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Claims

1. A remote control and command method based on panoramic video, characterized in that: The method comprises: Interactive open-air working area area division, to obtain multiple area boundaries of multiple open-air working areas; Performing high-definition camera configuration according to the multiple regional boundaries to obtain multiple regional image acquisition devices; Connecting the multiple regional image acquisition devices to a panoramic video fusion system; During the construction process of the open-air work area, the panoramic video fusion system receives and aligns multiple regional video streams of the multiple regional image acquisition devices to generate a real-time panoramic monitoring video; Pre-constructing an operation risk identification network, transmitting the real-time panoramic monitoring video to the operation risk identification network to obtain a real-time risk identification result, wherein the real-time risk identification result has a real-time operation area identification; Using the real-time risk identification result and the real-time operation area to traverse a pre-constructed operation risk table, a real-time risk coefficient is obtained; Remote control and command are performed according to the real-time risk factor.

2. The method according to claim 1, characterized in that High-definition cameras are configured according to the multiple regional boundaries to obtain multiple regional image acquisition devices, and the method further includes: Preset panoramic monitoring height constraints; Interactively obtain standard parameter information of a standard camera, and obtain a standard field of view according to the panoramic monitoring height constraint and the standard parameter information analysis; Obtaining multiple camera deployment strategies according to the multiple area boundaries and the standard field of view analysis; High-definition cameras are configured according to the multiple camera deployment strategies to obtain the multiple regional image acquisition devices.

3. The method according to claim 2, characterized in that A plurality of camera deployment strategies are obtained according to the plurality of area boundaries and the standard field of view analysis, the method further comprising: Determining whether a first area boundary falls within the standard field of view, wherein the first area boundary is any one of the multiple area boundaries; If the first area boundary falls within the standard field of view, locating a first area center of the first area boundary, and using the first area center as a first camera deployment strategy; If the first area boundary does not fall within the standard field of view, synchronizing the first area boundary to a pre-constructed camera layout analysis network to obtain a first layout analysis result, and using the first layout analysis result as the first camera layout strategy; By analogy, the multiple camera deployment strategies are obtained according to the multiple area boundaries and the standard field of view analysis.

4. The method according to claim 3, characterized in that The method further comprises: Interactively obtain multiple sample fields of view, multiple sample area boundaries, and multiple sample layout strategies; Obtain K data analysis models from the random operator library; Using the multiple sample fields of view, multiple sample area boundaries, and multiple sample layout strategies as training data to perform output optimization of the K data analysis models, and obtain K camera layout analysis branches; The K camera layout analysis branches are connected in parallel to generate the camera layout analysis network.

5. The method according to claim 4, characterized in that If the first area boundary does not fall within the standard field of view, synchronizing the first area boundary to a pre-built camera layout analysis network, analyzing to obtain the first layout analysis result, the method further includes: Synchronizing the boundary of the first area to a pre-constructed camera layout analysis network, and obtaining K candidate layout analysis results through analysis of the camera layout analysis network; Perform repetition frequency analysis on the K candidate layout analysis results to obtain the first layout analysis result.

6. The method according to claim 1, characterized in that Pre-constructing an operation risk identification network, transmitting the real-time panoramic monitoring video to the operation risk identification network to obtain a real-time risk identification result, wherein the real-time risk identification result has a real-time operation area identifier, and the method further includes: Interactively obtaining a plurality of sample risk monitoring videos of a plurality of operational risk behaviors, wherein the plurality of sample risk monitoring videos have a plurality of sample risk behavior type identifiers; Building the operation risk identification network based on the multiple sample risk monitoring videos and the multiple sample risk behavior types; Transmitting the real-time panoramic monitoring video to the operation risk identification network to obtain real-time risk identification results; The real-time operation area is reversely located in the panoramic video fusion system according to the real-time risk identification result.

7. The method according to claim 1, characterized in that The real-time risk identification result and the real-time operation area traverse the pre-constructed operation risk table to obtain a real-time risk coefficient. The method further includes: Interactively obtain multiple standard risk coefficients of multiple standard risk behavior types in multiple open-air working areas; Constructing and generating the operation risk table based on the multiple open-air operation areas, multiple standard risk behavior types and multiple standard risk coefficients; The real-time risk coefficient is obtained by using the real-time risk identification result and the pre-constructed operation risk table of the real-time operation area traversal.

8. A remote control and command system based on panoramic video, characterized in that: The system is used to implement a remote control and command method based on panoramic video according to any one of claims 1 to 7, and the system includes: An area boundary acquisition module, the area boundary acquisition module is used to interactively divide the open-air working area into regions and obtain multiple area boundaries of multiple open-air working areas; An image acquisition device acquisition module, the image acquisition device acquisition module is used to configure high-definition cameras according to the multiple regional boundaries to obtain multiple regional image acquisition devices; A connection module, the connection module is used to connect the multiple regional image acquisition devices to the panoramic video fusion system; A panoramic monitoring video generation module, wherein the panoramic monitoring video generation module is used to receive and align multiple regional video streams of the multiple regional image acquisition devices via the panoramic video fusion system during the construction process of the open-air work area to generate a real-time panoramic monitoring video; An identification module, the identification module is used to pre-build an operation risk identification network, transmit the real-time panoramic monitoring video to the operation risk identification network to obtain a real-time risk identification result, wherein the real-time risk identification result has a real-time operation area identification; A risk coefficient acquisition module, the risk coefficient acquisition module is used to obtain a real-time risk coefficient by using the real-time risk identification result and the real-time operation area to traverse a pre-constructed operation risk table; A command and control module, wherein the command and control module is used to perform remote control and command according to the real-time risk coefficient.

Images