360-degree panoramic vision digital twinning system, related method and device

Abstract

The invention provides a 360-degree panoramic visual digital twinning system, a related method and equipment, relates to the technical field of visual simulation, and is suitable for a mine mining working face. The system comprises a physical space data acquisition module which is used for acquiring physical entity data of a target mine mining working face, and the physical entity data comprises environment characteristic data, equipment characteristic data and production rule characteristic data of a target working condition; the digital space model construction module is used for constructing a plurality of visual digital models according to the physical entity data so as to fuse and generate a target visual digital model; and the twin data cloud storage module is used for storing physical entity data, simulation data of various visual digital models and driving data. According to the invention, the visual simulation intelligent level of the coal mining working face operation equipment can be improved, and the environment perception capability of the operation equipment is improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent coal mines, and in particular to a 360-degree panoramic visual digital twin system, related methods and equipment. Background Art

[0002] The intelligentization of coal mines aims to comprehensively upgrade and optimize the production, management, and safety of coal mines by applying modern technologies such as information technology, automation technology, the Internet of Things, and artificial intelligence.

[0003] Although mine mining has entered the intelligent stage at present, it is limited by the harsh working conditions of high dust and low illumination in the mine mining working face, and the environmental perception of the coal mine excavation working face still has problems such as poor perception accuracy and low intelligence level. Summary of the invention

[0004] The embodiments of the present application provide a 360-degree panoramic visual digital twin system, related methods and equipment, which are suitable for mine mining faces. By constructing a panoramic visual digital model based on the physical entity data of the mining face, the visual simulation accuracy and visual simulation intelligence level of the operating equipment can be improved. By storing the physical entity data, simulation data and drive data in the cloud, it is beneficial to improve the data interaction response speed and operation stability of the visual digital twin system.

[0005] In the first aspect, the present application proposes a 360-degree panoramic visual digital twin system, which is suitable for mine mining working faces. The system includes: a physical space data acquisition module, which is used to collect physical entity data of the target mine mining working face, wherein the physical entity data includes: environmental characteristic data, equipment characteristic data, and production rule characteristic data of the target working conditions; a digital space model construction module, which is used to construct multiple visual digital models based on the physical entity data to fuse and generate a target visual digital model; a twin data cloud storage module, which is used to store physical entity data, simulation data of multiple visual digital models, and drive data.

[0006] In some feasible embodiments, the physical space data acquisition module includes: multiple image data acquisition units, which are respectively set at target monitoring points and are used to collect physical entity image data of the target mine mining face; a picture control unit, which is used to control the posture states of multiple image data acquisition units to form a target picture acquisition field of view; wherein the visual coverage of the target picture acquisition field of view for the target mine mining face is greater than or equal to a preset visual coverage.

[0007] In some feasible embodiments, the digital space model construction module includes: an image correction unit, which is used to correct the target image so that the distortion rate of the target image captured by the target image acquisition unit is less than or equal to the preset distortion rate according to the parameter information of the target image acquisition unit and / or the images captured by at least two other image acquisition units when it is determined that the distortion rate of the target image captured by the target image acquisition unit is greater than the preset distortion rate, wherein the distance between the at least two other image acquisition units and the target image acquisition unit is less than or equal to the preset distance; an image fusion unit, which is used to perform feature matching on the captured image and / or the corrected image to generate a target fused image of the target mine mining face to construct a target visual digital model.

[0008] In some feasible embodiments, the digital space model construction module also includes: an environment model construction unit for constructing a target environment model based on environment feature data; a geometry model construction unit for constructing a target geometry model based on environment feature data and equipment feature data; an operation model construction unit for constructing a target operation model based on environment feature data and equipment feature data; a state model construction unit for constructing a target state model based on environment feature data and equipment feature data; a rule model construction unit for constructing a target rule model based on production rule feature data; and a model fusion unit for generating a target visual digital model based on the target environment model, target geometry model, target operation model, target state model and / or target rule model to simulate the actual production environment of the target mine mining face.

[0009] In some feasible implementations, the twin data cloud storage module includes: a data fusion unit, used to perform data fusion operations on physical entity data, simulation data, and driving data to generate a target data information set, wherein the data fusion operation includes: data conversion operation, data preprocessing operation, data classification operation, data association operation, and data integration operation.

[0010] In some feasible implementations, independent multi-protocol data transmission channels are formed between the physical space data acquisition module, the digital space model construction module and the twin data cloud storage module, wherein the data fusion unit is provided with multiple data interfaces for receiving physical entity data, simulation data and drive data of multiple protocols.

[0011] In some feasible implementations, the system is configured for the target mining equipment, and the system also includes: a twin application service module, which is used to obtain the continuous motion control data of the target mining equipment within a preset time, and generate mining hazard warning information and corresponding hazard emergency strategies based on physical entity data, simulation data and continuous motion control data.

[0012] On the second aspect, the present application proposes a 360-degree panoramic vision digital twin model generation method, which is applicable to the 360-degree panoramic vision digital twin system as described above. The method includes: collecting physical entity data of the target mine mining face, wherein the physical entity data includes: environmental characteristic data, equipment characteristic data, and production rule characteristic data of the target working conditions; based on the physical entity data, constructing multiple visual digital models to fuse and generate a target visual digital model; storing physical entity data, simulation data of multiple visual digital models, and driving data.

[0013] In a third aspect, the present application proposes an electronic device comprising a processor and a memory, wherein the memory stores computer program instructions, and the computer program instructions are used by the processor to execute the 360 ​​panoramic vision digital twin model generation method as described above when the processor is running.

[0014] In a fourth aspect, the present application proposes a storage medium on which program instructions are stored, and the program instructions are used to execute the 360 ​​panoramic vision digital twin model generation method as described above during runtime.

[0015] The 360-degree panoramic visual digital twin system, related methods and equipment proposed in this application are applicable to mine mining faces. Among them, the system includes: a physical space data acquisition module, which is used to collect the physical entity data of the target mine mining face, wherein the physical entity data includes: environmental characteristic data, equipment characteristic data and production rule characteristic data of the target working condition; a digital space model construction module, which is used to construct a variety of visual digital models based on the physical entity data to fuse and generate a target visual digital model; a twin data cloud storage module, which is used to store physical entity data, simulation data of a variety of visual digital models and drive data. This application can improve the accuracy of visual simulation and the level of visual simulation intelligence by constructing a visual digital model based on physical entity data. By storing physical entity data, simulation data and drive data in the cloud, it is beneficial to improve the data interaction response speed and operation stability of the visual digital twin system. This application can improve the level of visual simulation intelligence of coal mine mining face operating equipment and improve the environmental perception ability of operating equipment.

[0016] Other advantages, objectives and features of the present application will be reflected in part through the following description, and in part will be understood by those skilled in the art through study and practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the exemplary embodiments below. The accompanying drawings are only for the purpose of illustrating exemplary embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0018] Figure 1 A structural schematic diagram of a 360-degree panoramic visual digital twin system provided in an embodiment of the present application;

[0019] Figure 2 A structural schematic diagram of a physical space data acquisition module provided in an embodiment of the present application;

[0020] Figure 3 A schematic diagram of the layout position of a fisheye camera provided in an embodiment of the present application;

[0021] Figure 4 A structural schematic diagram of a digital space model building module provided in an embodiment of the present application;

[0022] Figure 5 A schematic diagram of a process for generating a 360-degree panoramic visual digital twin model provided in an embodiment of the present application; and

[0023] Figure 6 A structural schematic diagram of an electronic device provided in an embodiment of the present application.

[0024] in, Figures 1 to 6 The corresponding relationship between the reference numerals and the structure names is as follows:

[0025] 100. 360-degree panoramic visual digital twin system; 110. Physical space data acquisition module; 120. Digital space model building module; 130. Twin data cloud storage module;

[0026] 111. Image data acquisition unit; 112. Picture control unit;

[0027] 121. Image correction unit; 122. Image fusion unit; 123. Environment model building unit; 124. Geometric model building unit; 125. Operation model building unit; 126. State model building unit; 127. Rule model building unit; 128. Model fusion unit;

[0028] 300. Electronic device; 310. Processor; 320. Memory. DETAILED DESCRIPTION

[0029] In order to better understand the technical solutions provided by the embodiments of this specification, the technical solutions of the embodiments of this specification are described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of this specification and the specific features in the embodiments are detailed descriptions of the technical solutions of the embodiments of this specification, rather than limitations on the technical solutions of this specification. In the absence of conflict, the embodiments of this specification and the technical features in the embodiments can be combined with each other.

[0030] In this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or equipment. In the absence of more restrictions, the elements limited by the statement "comprise one..." do not exclude the existence of other identical elements in the process, method, article or equipment including the elements. The term "more than two" includes two or more than two situations.

[0031] The intelligentization of coal mines aims to comprehensively upgrade and optimize the production, management, and safety of coal mines by applying modern technologies such as information technology, automation technology, the Internet of Things, and artificial intelligence.

[0032] Although mine mining has entered the intelligent stage at present, it is limited by the harsh working conditions of high dust and low illumination at the mine mining face. The operating equipment has insufficient perception of the environment, poor intelligence level, slow data interaction response speed and poor stability.

[0033] In view of this, the embodiments of the present application provide a 360-degree panoramic visual digital twin system, related methods and equipment, which are suitable for mine mining working faces. By constructing a visual digital model based on physical entity data, the visual simulation accuracy and the level of visual simulation intelligence can be improved. By storing physical entity data, simulation data and drive data in the cloud, it is beneficial to improve the data interaction response speed and operation stability of the visual digital twin system.

[0034] A first aspect of an embodiment of the present application provides a 360-degree panoramic visual digital twin system suitable for mine excavation working faces. Figure 1 A structural schematic diagram of a 360-degree panoramic visual digital twin system 100 provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the system 100 includes: a physical space data acquisition module 110, a digital space model construction module 120 and a twin data cloud storage module 130.

[0035] The physical space data acquisition module 110 is used to acquire the physical entity data of the target mine excavation working face, wherein the physical entity data includes: environmental characteristic data, equipment characteristic data and production rule characteristic data of the target working condition.

[0036] Exemplarily, the above-mentioned environmental characteristic data may include: excavation face environmental characteristics, temperature characteristic data, humidity characteristic data, pressure characteristic data, lighting characteristic data, noise characteristic data, disturbance behavior characteristic data, etc.

[0037] Specifically, the above-mentioned excavation face environmental characteristics may include: tunnel environmental characteristic data, surrounding rock state characteristic data, etc.

[0038] Exemplarily, the above-mentioned equipment characteristic data include: equipment shape characteristic data, equipment performance characteristic data, equipment structure characteristic data, equipment posture characteristic data, equipment operation behavior characteristic data and equipment working status characteristic data, etc.

[0039] Specifically, the above equipment may include: integrated digging and anchoring machines, anchor transfer machines for coal mines, belt transfer machines, walking-type self-moving machine tails, underground centralized control centers, and other sensor devices installed on production equipment that can provide information.

[0040] Exemplarily, the production rule characteristic data of the target working condition may include: production process control procedure characteristic data of the target working condition, production process execution rule characteristic data of the target working condition, and implicit data association rule characteristic data of the target working condition, etc.

[0041] It should be noted that, in some feasible implementations, the physical space data acquisition module 110 can also be used to collect relevant data of various information systems. For example, the information systems can include: electromechanical systems, transportation systems, ventilation systems, and drainage systems.

[0042] The digital space model building module 120 is used to build multiple visual digital models according to the physical entity data to fuse and generate a target visual digital model.

[0043] It should be noted that the digital space model construction module 120 can construct multiple visual digital models based on the data mirror of the physical entity data collected by the physical space data acquisition module 110 and based on multiple modeling technologies to fuse and generate a target visual digital model, thereby achieving accurate simulation and prediction of the physical space.

[0044] Exemplarily, the above-mentioned multiple visual digital models may include: a production environment model, a geometric shape model, a production rule model, and the like.

[0045] In some feasible implementations, multiple visual digital models can be constructed based on multiple modeling technologies according to the actual needs of the user and the corresponding physical entity data to fuse and generate a target visual digital model that meets the actual needs of the user.

[0046] The twin data cloud storage module 130 is used to store physical entity data, simulation data of various visual digital models, and drive data.

[0047] Exemplarily, the above-mentioned simulation data may include: model operation mechanism data, model operation physical process related data, model process simulation data, model behavior simulation data, model operation process verification data, model simulation effect evaluation data, model performance analysis data and model operation status prediction data.

[0048] In some feasible implementations, the twin data cloud storage module 130 can also be used to store basic knowledge data. Exemplarily, the basic knowledge data can include: industry general standard knowledge data, industry rule constraint knowledge data, and equipment maintenance knowledge data.

[0049] Based on this, the 360 ​​panoramic visual digital twin system 100 proposed in this application is suitable for mine mining working faces, wherein the system includes: a physical space data acquisition module 110, used to collect physical entity data of the target mine mining working face, wherein the physical entity data includes: environmental characteristic data, equipment characteristic data and production rule characteristic data of the target working conditions; a digital space model construction module 120, used to construct a variety of visual digital models based on the physical entity data to fuse and generate a target visual digital model; a twin data cloud storage module 130, used to store physical entity data, simulation data of a variety of visual digital models and drive data. By configuring a physical space data acquisition module 110, a digital space model construction module 120 and a twin data cloud storage module 130, the present application can realize the accurate construction of multiple visual digital models based on physical entity data, and integrate and generate target visual digital models that meet the actual needs of users, thereby getting rid of the influence of harsh working conditions of high dust and low illumination in the mine mining face on the construction of visual models, and improving the simulation accuracy and intelligence level of visual digital models; storing physical entity data, simulation data of multiple visual digital models and driving data in the twin data cloud storage module 130 is conducive to facilitating subsequent data retrieval, thereby improving the data interaction response speed and operation stability of the 360 ​​panoramic visual digital twin system 100.

[0050] For example, Figure 2 A structural diagram of a physical space data acquisition module 110 provided in an embodiment of the present application is shown as follows: Figure 2 As shown, in some feasible implementations, the physical space data acquisition module 110 includes: a plurality of image data acquisition units 111, which are respectively arranged at target monitoring points and are used to acquire physical entity image data of the target mine excavation working face.

[0051] Exemplarily, the image data acquisition unit 111 may include: image acquisition devices such as a fisheye camera and an industrial camera.

[0052] In some feasible implementations, the above-mentioned target monitoring points can be set according to the user's actual monitoring needs, so as to collect physical entity image data of the target mine excavation working face that meets the user's monitoring needs.

[0053] Exemplarily, the above-mentioned target monitoring points may include: fixed points around the target mining equipment, fixed points on the target mine mining face, etc.

[0054] In some feasible embodiments, the physical space data acquisition module 110 also includes: an image control unit 112, which is used to control the posture states of multiple image data acquisition units 111 to form a target image acquisition field of view; wherein the visual coverage of the target image acquisition field of view for the target mine mining working face is greater than or equal to a preset visual coverage.

[0055] It should be noted that the above-mentioned visual coverage is the ratio of the area that can be observed by the multiple image data acquisition units 111 within the target mine excavation working face to the above-mentioned target mine excavation working face. Among them, the above-mentioned preset visual coverage can be set by the user according to the monitoring accuracy requirements of the user, and no specific limitation is made here. It should be noted that the above-mentioned preset visual coverage is positively correlated with the monitoring accuracy requirements of the above-mentioned user, that is, the higher the user's monitoring accuracy requirements, the greater the value of the preset visual coverage. Specifically, the value of the above-mentioned preset visual coverage can be 95%.

[0056] In some feasible implementations, the posture states of multiple image data acquisition units 111 can be adjusted so that the visual coverage of the target image acquisition field of view for the target mine mining face is greater than or equal to 95%, thereby achieving full coverage acquisition of physical entity image data of the target mine mining face.

[0057] Based on this, the above-mentioned system can realize the accurate collection of physical entity image data of the target mine mining face by setting multiple image data acquisition units 111 at the target monitoring points respectively. By configuring the picture control unit 112, it can realize the accurate control of the posture states of multiple image data acquisition units 111 according to actual monitoring needs to form a target picture acquisition field of view, so that the visual coverage rate of the target picture acquisition field of view for the target mine mining face is greater than or equal to the preset visual coverage rate, thereby realizing panoramic perception of the target mine mining face, providing an image data basis for the accurate construction of the visual digital model, and promoting the accurate simulation of the actual working conditions of the target mine mining face.

[0058] In some feasible implementations, a preset number of fisheye cameras can be arranged around the intelligent excavation working face operation equipment to facilitate the visual panoramic perception of the surrounding environment of the intelligent excavation working face operation equipment. Among them, the above preset number can be set according to the user's monitoring accuracy requirements, and no specific limitation is made here. It should be noted that the above preset number is positively correlated with the above user's monitoring accuracy requirements, that is, the higher the user's monitoring accuracy requirements, the larger the preset number.

[0059] For example, Figure 3 A schematic diagram of the layout position of a fisheye camera provided in an embodiment of the present application, such as Figure 3 As shown, 7 fisheye cameras can be arranged around the intelligent excavation working face operating equipment to form a target image acquisition field of view in the rectangular frame, so that the visual coverage of the target image acquisition field of view for the target mine excavation working face is greater than or equal to the preset visual coverage.

[0060] For example, Figure 4 A structural diagram of a digital space model building module 120 provided in an embodiment of the present application is shown in FIG. Figure 4 As shown, in some feasible implementations, the digital space model construction module 120 includes: an image correction unit 121, which is used to correct the target image so that the distortion rate of the target image captured by the target image acquisition unit is less than or equal to the preset distortion rate according to the parameter information of the target image acquisition unit and / or the images captured by at least two other image acquisition units when it is determined that the distortion rate of the target image captured by the target image acquisition unit is greater than the preset distortion rate, wherein the distance between the at least two other image acquisition units and the target image acquisition unit is less than or equal to the preset distance.

[0061] It should be noted that the distortion rate of the target image can be determined according to the Brown-Conrady model. The above-mentioned preset distortion rate and the above-mentioned preset distance can be set according to the user's image correction accuracy requirements, and are not specifically limited here. Among them, the above-mentioned preset distortion rate is negatively correlated with the user's image correction accuracy requirements, that is, the higher the user's image correction accuracy requirements, the smaller the above-mentioned preset distortion rate. The above-mentioned preset distance is negatively correlated with the user's image correction accuracy requirements, that is, the higher the user's image correction accuracy requirements, the smaller the above-mentioned preset distance.

[0062] In some feasible implementations, when it is determined that the distortion rate of the target image acquired by the target image acquisition unit is greater than the preset distortion rate, the image correction unit 121 may correct the target image based on the parameter information of the image data acquisition unit 111, in combination with the radial distortion correction technology and / or axial geometric correction technology of the image data acquisition unit 111, so that the distortion rate of the target image is less than or equal to the preset distortion rate. And / or, retrieve the corresponding data of the overlapping area with the target image in the image data acquired by the image acquisition unit whose distance from the target image acquisition unit is less than or equal to the preset distance, and correct the target image so that the distortion rate of the target image is less than or equal to the preset distortion rate.

[0063] In some feasible implementations, the digital space model construction module 120 also includes: an image fusion unit 122, which is used to perform feature matching on the collected images and / or corrected images, generate a target fused image of the target mine mining face to construct a target visual digital model.

[0064] Exemplarily, the image fusion unit 122 may perform feature matching on the captured image and / or the corrected image based on a four-point rectangle correction matching method, and fuse and splice the image to generate a target fused image of the target mine excavation face to construct a target visual digital model.

[0065] Based on this, the above system can realize, by configuring the image correction unit 121, when it is determined that the distortion rate of the target image collected by the target image acquisition unit is greater than the preset distortion rate, according to the parameter information of the target image acquisition unit and / or the images collected by at least two other image acquisition units, correcting the target image so that the distortion rate of the target image is less than or equal to the preset distortion rate. By limiting the distance between at least two other image acquisition units and the target image acquisition unit to be less than or equal to the preset distance, the image accuracy of the target image can be further improved. By configuring the image fusion unit 122, the collected image and / or the corrected image are feature matched, and the target fusion image of the target mine excavation working face can be accurately generated, thereby improving the construction accuracy of the target visual digital model. The present application constructs a panoramic visual model based on the actual production environment image of the target mine excavation working face, and mirrors the panoramic visual model to the digital space and combines it with the three-dimensional virtual twin of the equipment, so as to realize the formation of a panoramic environment visual digital twin model with real environment and virtual equipment, which is conducive to improving the visual simulation accuracy and the level of visual simulation intelligence in the model construction process.

[0066] In some feasible embodiments, the digital space model construction module 120 also includes: an environment model construction unit 123 for constructing a target environment model based on environment feature data; a geometry model construction unit 124 for constructing a target geometry model based on environment feature data and equipment feature data; an operation model construction unit 125 for constructing a target operation model based on environment feature data and equipment feature data; a state model construction unit 126 for constructing a target state model based on environment feature data and equipment feature data; a rule model construction unit 127 for constructing a target rule model based on production rule feature data; and a model fusion unit 128 for fusing and generating a target visual digital model based on the target environment model, target geometry model, target operation model, target state model and / or target rule model to simulate the actual production environment of the target mine mining face.

[0067] Exemplarily, the above-mentioned environment model building unit 123 can build a target environment model according to: temperature characteristic data, humidity characteristic data, pressure characteristic data, lighting characteristic data, and noise characteristic data, etc. It should be noted that the digitization of the environmental characteristics of the target mine excavation working face by the target environment model is conducive to ensuring the safe production of the target mine excavation working face.

[0068] Exemplarily, the geometric model building unit 124 can build a target geometric model based on: tunnel environment geometric feature data, equipment posture feature data, equipment shape feature data, and equipment structure feature data, etc. It should be noted that the target geometric model is a digital representation of the target mine excavation working face environmental features, equipment posture features, equipment shape features, and equipment structure features, which is conducive to achieving real-time monitoring of the target mine excavation working face.

[0069] Exemplarily, the operation model building unit 125 can build a target operation model based on operation factor characteristic data such as equipment operation behavior characteristic data, environmental disturbance behavior characteristic data, and scheduling instruction behavior characteristic data. It should be noted that the target operation model is a data-based representation of the operation characteristics of the working face, which is conducive to ensuring the normal operation of the target mine excavation working face.

[0070] Exemplarily, the state model building unit 126 can build a target state model based on: environmental feature data and equipment feature data of the target mine excavation working face, etc. It should be noted that the target state model is a digital representation of the working face environment, equipment status, operation operation status, fault warning status and operation process status, which is conducive to improving the reliability of the target mine excavation working face.

[0071] Exemplarily, the rule model building unit 127 can build a target rule model based on: production process control procedure feature data, production process execution rule feature data, and implicit data association rule feature data, etc. It should be noted that the target rule model is a digital representation of the working face rules, which is useful for ensuring the standardization of the target mine excavation working face.

[0072] It should be noted that by configuring the model fusion unit 128, it is possible to select the corresponding model according to the actual needs of the user, fuse and generate a target visual digital model, and use the digital space to conduct a comprehensive, in-depth and detailed simulation and prediction of the physical space, thereby improving the intelligence and efficiency of the management and optimization of the mining working face of the target mine.

[0073] Based on this, the above-mentioned system can realize a comprehensive simulation of the actual production environment, equipment status and production rules of the physical space by configuring the environment model construction unit 123, the geometric model construction unit 124, the operation model construction unit 125, the state model construction unit 126 and the rule model construction unit 127. By configuring the model fusion unit 128, it can realize the precise fusion and generation of the target visual digital model according to the actual needs of the user to simulate the actual production environment of the target mine mining face, get rid of the adverse effects of the harsh working conditions of high dust and low illumination of the mine mining face on the production of the target mine mining face, and ensure the stable production of the target mine mining face.

[0074] In some feasible embodiments, the twin data cloud storage module 130 includes: a data fusion unit, used to perform data fusion operations on physical entity data, simulation data, and driving data to generate a target data information set, wherein the data fusion operation includes: data conversion operation, data preprocessing operation, data classification operation, data association operation, and data integration operation.

[0075] In some feasible implementations, the above-mentioned data fusion unit can be used to perform data conversion operations, data preprocessing operations, data classification operations, data association operations and data integration operations on physical entity data, simulation data and driving data to generate a target data information set, which is stored in the above-mentioned twin data cloud storage module 130 to improve the scientific nature of the data storage structure and facilitate the subsequent rapid retrieval of data, so as to improve the data interaction response speed and operation stability of the 360 ​​panoramic vision digital twin system 100.

[0076] It should be noted that, in some feasible implementations, the twin data cloud storage module 130 can store the full chain data in the 360 ​​panoramic vision digital twin system 100. Accordingly, the above-mentioned data fusion unit can be used to perform the above-mentioned data fusion operation on the full chain data in the 360 ​​panoramic vision digital twin system 100 to generate a full chain target data information set, so as to further improve the data interaction response speed and operation stability of the 360 ​​panoramic vision digital twin system 100. Among them, the above-mentioned full chain data may include: perception data in physical space, simulation data in digital space, model-driven data, twin application service data, knowledge data, and fusion derivative data, etc. Among them, the above-mentioned twin application service data can be used as the driving chain of digital twins. The above-mentioned twin application service data may include: functional service data, such as: algorithms, data models, data processing methods, etc.; service service data, such as: user interaction interface data, information sorting data, and screening data, etc.

[0077] In some feasible implementations, independent multi-protocol data transmission channels are formed between the physical space data acquisition module 110, the digital space model construction module 120 and the twin data cloud storage module 130, wherein the data fusion unit is provided with multiple data interfaces for receiving physical entity data, simulation data and drive data of multiple protocols.

[0078] Exemplarily, the above-mentioned multiple protocol data transmission channels may include: MQTT data transmission channel, HTTP / HTTPS data transmission channel, WebSocket data transmission channel, etc. The above-mentioned multiple data interfaces may include: MQTT client, HTTP / HTTPS interface, Socket interface, etc.

[0079] Based on this, the above system forms independent multi-protocol data transmission channels between the physical space data acquisition module 110, the digital space model construction module 120 and the twin data cloud storage module 130. The data fusion unit is provided with multiple data interfaces for receiving physical entity data, simulation data and drive data of multiple protocols. It can realize the transmission of multiple protocol data to the data fusion unit, improve the data fusion unit's ability to process multiple data fusions, improve the data compatibility of the data fusion unit, and thereby improve the data compatibility of the 360 ​​panoramic vision digital twin system 100.

[0080] In some feasible implementations, the above-mentioned multiple protocol data transmission channels can be applied to multiple communication networks for transmitting twin data. Exemplarily, the above-mentioned multiple communication networks may include: Internet of Things, 5G network, ubiquitous network and WiFi, etc. Among them, the Internet of Things can collect the equipment information and environmental information that need to be monitored in the mining face of the target mine in real time based on visual perception technology, radar scanning technology, etc. Among them, the 5G network can improve the information interaction speed and accuracy of the 360 ​​panoramic visual digital twin system 100. Specifically, the above-mentioned 5G network may include the following application scenarios: enhanced mobile broadband (eMBB), ultra-high reliability and low latency communication (uRLLC), massive machine communication (mMTC), etc. Among them, the ubiquitous network can improve the connection tightness of sensors, transmitters, actuators, drivers and controllers at each end. Among them, the above-mentioned WiFi may include: the sixth-generation wireless network technology IEEE 802.11ax, to improve the compatibility of the 360 ​​panoramic vision digital twin system 100, reduce the data transmission cost of the 360 ​​panoramic vision digital twin system 100, and improve the adaptability of the 360 ​​panoramic vision digital twin system 100 to long-distance data transmission.

[0081] In some feasible implementations, the system is configured for the target mining equipment, and the system also includes: a twin application service module, which is used to obtain the continuous motion control data of the target mining equipment within a preset time, and generate mining hazard warning information and corresponding hazard emergency strategies based on physical entity data, simulation data and continuous motion control data.

[0082] For example, the twin application service module can retrieve the full chain data in the twin data cloud storage module 130 to obtain the continuous motion control data of the target mining equipment within the preset time, so as to generate mining hazard warning information and corresponding emergency strategies based on the physical entity data, simulation data and continuous motion control data. The preset time can be set according to the actual needs of the user and is not specifically limited here.

[0083] In some feasible implementations, when the twin application service module generates mining hazard warning information and corresponding hazard emergency response strategies, the above-mentioned mining hazard warning information and corresponding hazard emergency response strategies can be voice broadcast based on the loudspeaker or speaker of the target mining equipment, or displayed in text on the control screen of the target mining equipment.

[0084] In some feasible implementations, the above-mentioned twin application service module can also be installed on the user's mobile terminal to facilitate sending mining hazard warning information and corresponding hazard emergency strategies directly to the user terminal.

[0085] Based on this, the above-mentioned system can improve the safety of the mining process by generating mining hazard warning information and corresponding hazard emergency strategies through the provision of a twin application service module.

[0086] In some feasible implementations, the twin application service module may also include functions such as environmental perception, state monitoring, motion control, process monitoring, intelligent maintenance, and performance optimization. In some feasible implementations, the twin application service module directly retrieves the full chain data in the twin data cloud storage module 130, or performs data simulation and recycling in combination with the driving digital space model to achieve the optimization function of various applications.

[0087] It should be noted that, in some feasible implementations, the 360-degree panoramic visual digital twin system 100 provided by the present application can realize a panoramic display of the tunnel environment of the target mine excavation working face. In some feasible implementations, the 360-degree panoramic visual digital twin system 100 provided by the present application can support free-viewpoint roaming display, stand-alone device free-viewpoint display, first-person perspective roaming and third-person perspective roaming, and realize the simulation inspection of the three-dimensional scene of the intelligent fast excavation working face. At the same time, the 360-degree panoramic visual digital twin system 100 provided by the present application has the function of switching between automatic and manual inspection.

[0088] In some feasible implementations, the 360-degree panoramic vision digital twin system 100 provided in this application can simulate the scene interface of a fixed perspective to achieve dynamic tracking and display of the integrated digging and anchoring machine, the anchor transfer unit for coal mines, the retractable belt conveyor, and the cutting curve of the integrated digging and anchoring machine. The 360-degree panoramic vision digital twin system 100 provided in this application can be used to display the current equipment operation status, the relative position and posture of the integrated digging and anchoring machine to the tunnel, the relative position and posture of the anchor transfer unit for coal mines to the tunnel, the overlap distance between the belt transfer machine and the tail of the walking self-moving machine, and the distance between the units, and leave an interface for the introduction of other equipment status information, including but not limited to the operation status of key components of each unit, such as: temporary support operation status, walking status, front and rear support, dust removal fan status, cutting feed times and footage display, pump station pressure, equipment temperature, cutting current, etc.

[0089] In some feasible implementations, the 360-degree panoramic visual digital twin system 100 provided by the present application also supports real-time monitoring of human-machine environment management. For example, the present application can realize real-time monitoring of the working status of underground personnel, equipment operating status, geological status, environmental situation and other element information, and fully display the positional relationship between personnel, equipment and environment through a three-dimensional display screen, and perform real-scene digital online display. Real-time alarm or early warning of abnormal behavior, abnormal status, abnormal disturbance or potential risks to ensure the safety of the working environment; centralized recording and management of detailed information such as personnel status, equipment operating conditions, and environmental information.

[0090] In some feasible implementations, the 360-degree panoramic visual digital twin system 100 provided in this application also supports historical data statistical analysis. For example, historical data and real-time data are combined in the digital space to statistically classify the status and behavior of personnel, equipment, and environment, analyze the environmental model, geometric model, operation model, state model, and rule model, and provide decision-making basis for the issuance of instructions for intelligent excavation working face equipment and the optimization of process system through digital twin simulation tests, thereby improving excavation efficiency.

[0091] In some feasible implementations, the 360-degree panoramic visual digital twin system 100 provided by the present application also supports intelligent maintenance of production equipment. For example, the present application can use artificial intelligence algorithms to analyze the real-time status data and historical maintenance information of the integrated digging and anchoring machine, the coal mine anchor transfer unit, the belt transfer machine, and the walking self-moving tail, so as to achieve data consistency and process coordination, predict potential failures and performance degradation, and formulate a predictive maintenance plan to determine the best time for maintenance, so as to reduce unexpected downtime, ensure the reliability and production efficiency of equipment operation, and minimize maintenance costs.

[0092] In some feasible implementations, the 360-degree panoramic vision digital twin system 100 provided in this application also supports production optimization control and prediction. For example, this application can achieve in-depth optimization and intelligent scheduling of tunneling processes through integrated data analysis and artificial intelligence technology, use advanced optimization algorithms such as genetic algorithms, intelligently adjust production plans, and combine real-time monitoring and predictive maintenance to warn of potential risks, reduce downtime, and improve production continuity and stability. Intelligently adjust production strategies to achieve flexible production, and use digital twin technology to simulate production processes in a virtual environment, verify optimization plans, and reduce trial and error costs.

[0093] According to a second aspect of an embodiment of the present application, a method for generating a 360-degree panoramic vision digital twin model is proposed, which is applicable to the 360-degree panoramic vision digital twin system as described above. Figure 5 A schematic diagram of a 360-degree panoramic visual digital twin model generation method 200 provided in an embodiment of the present application. Figure 5 As shown, the method 200 may include the following steps:

[0094] Step S210, collecting physical entity data of the target mine excavation working face, wherein the physical entity data includes: environmental characteristic data, equipment characteristic data and production rule characteristic data of the target working condition.

[0095] Step S220, constructing multiple visual digital models based on the physical entity data to fuse and generate a target visual digital model.

[0096] Step S230, storing physical entity data, simulation data of multiple visual digital models, and driving data.

[0097] In a third aspect, the present application proposes an electronic device. Figure 6 FIG. 1 is a structural diagram of an electronic device 300 provided in an embodiment of the present application. Figure 6 As shown, the electronic device 300 includes a processor 310 and a memory 320, wherein the memory 320 stores computer program instructions, and the computer program instructions are used by the processor 310 to execute the 360 ​​panoramic visual digital twin model generation method described in the second aspect when the processor 310 is running.

[0098] In a fourth aspect, the present application further provides a storage medium on which program instructions are stored, and the program instructions are used to execute the 360 ​​panoramic visual digital twin model generation method described in the second aspect when running. The storage medium may include, for example, a storage component of a tablet computer, a hard disk of a computer, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disk read-only memory (CD-ROM), a USB memory, or any combination of the above storage media. The computer-readable storage medium may be any combination of one or more computer-readable storage media.

[0099] A person of ordinary skill in the art can understand the specific details and beneficial effects of the 360-degree panoramic vision digital twin model generation method, electronic device and storage medium by reading the above description of the 360-degree panoramic vision digital twin system, which will not be elaborated here for the sake of brevity.

[0100] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0101] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

[0102] Although the preferred embodiments of this specification have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of this specification.

[0103] Obviously, those skilled in the art can make various changes and modifications to this specification without departing from the spirit and scope of this specification. Thus, if these modifications and variations of this specification fall within the scope of the claims of this specification and their equivalents, this specification is also intended to include these modifications and variations.

Claims

1. A 360-degree panoramic visual digital twin system, characterized in that: Applicable to the mining face of a mine, the system comprises: A physical space data acquisition module is used to acquire physical entity data of the target mine excavation working face, wherein the physical entity data includes: environmental characteristic data, equipment characteristic data and production rule characteristic data of the target working condition; A digital space model building module, used to build multiple visual digital models according to the physical entity data to fuse and generate a target visual digital model; The twin data cloud storage module is used to store the physical entity data, the simulation data of the multiple visual digital models, and the driving data.

2. The system according to claim 1, characterized in that The physical space data acquisition module includes: A plurality of image data acquisition units are respectively arranged at target monitoring points and used for acquiring physical entity image data of the target mine excavation working face; A picture control unit, used to control the positions and postures of the plurality of image data acquisition units to form a target picture acquisition field of view; Among them, the visual coverage of the target image acquisition field of view to the target mine mining working face is greater than or equal to a preset visual coverage.

3. The system according to claim 2, characterized in that The digital space model building module includes: an image correction unit, configured to, when determining that the distortion rate of the target image acquired by the target image acquisition unit is greater than a preset distortion rate, correct the target image so that the distortion rate of the target image is less than or equal to the preset distortion rate according to parameter information of the target image acquisition unit and / or images acquired by at least two other image acquisition units, wherein the distance between the at least two other image acquisition units and the target image acquisition unit is less than or equal to the preset distance; The image fusion unit is used to perform feature matching on the collected images and / or the corrected images to generate a target fused image of the target mine excavation working face to construct a target visual digital model.

4. The system according to claim 1, characterized in that The digital space model building module also includes: The environment model building unit is used to build a target environment model according to the environment feature data; The geometric model building unit is used to build a target geometric model according to the environmental feature data and the device feature data; The operation model building unit is used to build a target operation model according to the environmental characteristic data and the device characteristic data; The state model building unit is used to build a target state model according to the environment feature data and the device feature data; The rule model building unit is used to build a target rule model according to the production rule feature data; The model fusion unit is used to generate a target visual digital model based on the target environment model, the target geometric model, the target operation model, the target state model and / or the target rule model to simulate the actual production environment of the target mine mining face.

5. The system according to claim 1, characterized in that The twin data cloud storage module includes: A data fusion unit is used to perform a data fusion operation on the physical entity data, the simulation data and the driving data to generate a target data information set, wherein the data fusion operation includes: a data conversion operation, a data preprocessing operation, a data classification operation, a data association operation and a data integration operation.

6. The system according to claim 5, characterized in that Independent multi-protocol data transmission channels are formed between the physical space data acquisition module, the digital space model construction module and the twin data cloud storage module, wherein the data fusion unit is provided with multiple data interfaces for receiving the physical entity data, the simulation data and the drive data of multiple protocols.

7. The system according to any one of claims 1 to 6, characterized in that The system is configured on the target mining equipment, and the system further comprises: The twin application service module is used to obtain the continuous motion control data of the target mining equipment within a preset time, and generate mining hazard warning information and corresponding hazard emergency strategies based on the physical entity data, the simulation data and the continuous motion control data.

8. A method for generating a 360-degree panoramic visual digital twin model, characterized in that: Applicable to the system of claim 1, the method comprising: Collecting physical entity data of the target mine excavation working face, wherein the physical entity data includes: environmental characteristic data, equipment characteristic data and production rule characteristic data of the target working condition; According to the physical entity data, a plurality of visual digital models are constructed to fuse and generate a target visual digital model; The physical entity data, the simulation data of the multiple visual digital models and the driving data are stored.

9. An electronic device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the processor executes the computer program, the method according to claim 8 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to claim 8 is implemented.

Images