Hybrid mine data acquisition and transmission method and device, electronic equipment and medium

Abstract

The invention provides a mixed mine data acquisition and transmission method and device, electronic equipment and a medium, and relates to the technical field of data acquisition and transmission, and the method comprises the steps: synchronously receiving mixed mine data through a target communication interface; wherein the mixed mine data comprises sensor sensing data and object file type data; creating a Hash consistency check code based on the hybrid mine data, separating the hybrid mine data, and obtaining a structured data packet corresponding to the sensor sensing data and an object file stream corresponding to the object file type data based on the Hash consistency check code; and transmitting the structured data packet and the object file stream to a superior platform, so that the superior platform performs collaborative fusion storage of the sensor sensing data and the object file type data based on the hash consistency check codes of the structured data packet and the object file stream. According to the method and the system, the sensor sensing data and the object file type data can be ensured to be consistently and cooperatively collected and transmitted in the same service scene.

Description

Technical Field

[0001] The present invention relates to the technical field of data acquisition and transmission, and in particular to a method, device, electronic equipment and medium for hybrid mine data acquisition and transmission. Background Art

[0002] In order to ensure the safety of mine production, all kinds of monitoring systems and object file data in mines need to be integrated, collected and transmitted to provide data support for various intelligent and information systems in mines. The mixed mine data collection and transmission in mines mainly targets monitoring sensors, industrial centralized control systems, etc., involving unstructured data such as IoT OT structured data, IT structured data and object file type data (including binary files). In order to ensure the consistency of data use and business attributes, single batch structured data and unstructured data in the same business scenario need to be collaboratively collected, transmitted and related to each other. However, there is currently no solution to this problem. Summary of the invention

[0003] In view of this, the purpose of the present invention is to provide a method, device, electronic device and medium for hybrid mine data collection and transmission, which can ensure the consistent collaborative collection and transmission of sensor perception data and object file type data in the same business scenario.

[0004] In order to achieve the above purpose, the technical solution adopted by the embodiment of the present invention is as follows: In the first aspect, an embodiment of the present invention provides a method for collecting and transmitting hybrid mine data, comprising: synchronously receiving hybrid mine data through a target communication interface; wherein the hybrid mine data comprises: sensor perception data and object file type data; creating a hash consistency check code based on the hybrid mine data, and separating the hybrid mine data, and obtaining a structured data packet corresponding to the sensor perception data and an object file stream corresponding to the object file type data based on the hash consistency check code; transmitting the structured data packet and the object file stream to an upper-level platform, so that the upper-level platform performs collaborative fusion storage of the sensor perception data and the object file type data based on the hash consistency check code of the structured data packet and the object file stream.

[0005] In one embodiment, the mixed mining data is separated, and a structured data packet corresponding to the sensor perception data and an object file stream corresponding to the object file type data are obtained based on the hash consistency check code, including: serializing the sensor perception data and injecting the hash consistency check code into the serialized sensor perception data; encrypting the sensor perception data injected with the hash consistency check code to obtain a structured data packet corresponding to the sensor perception data; after receiving the stream terminator of the object file type data, injecting the hash consistency check code into the tail of the stream of the object file type data to obtain the object file stream corresponding to the object file type data.

[0006] In one embodiment, structured data packets and object file streams are transmitted to an upper-level platform, including: transmitting structured data packets to the upper-level platform via a structured data transmission channel; wherein the structured data transmission channel is implemented based on a high-throughput timing message queue middleware; transmitting object file streams to the upper-level platform via an unstructured data transmission channel; wherein the unstructured data transmission channel is implemented based on an object storage middleware.

[0007] In one embodiment, a structured data packet is transmitted to an upper-level platform via a structured data transmission channel, including: publishing the structured data packet to a message queue; after receiving subscription information from the upper-level platform, pulling the structured data packet subscribed by the upper-level platform from the message queue, and transmitting the structured data packet to the upper-level platform.

[0008] In one embodiment, an object file stream is transmitted to an upper-level platform via an unstructured data transmission channel, including: pushing an object file stream of object file type data to an object storage of the upper-level platform; constructing an object receiving monitor via the upper-level platform, executing a Hook call when a new object file stream transmission is detected, calling an unstructured data receiving service to receive the object file stream, and packaging the object file when a stream terminator of the object file stream is received to obtain an object file package.

[0009] In one embodiment, collaborative fusion storage of sensor perception data and object file type data is performed based on hash consistency check codes of structured data packets and object file streams, including: when the monitoring service of the upper-level platform monitors the object file stream transmission, a Hook call is executed to call the consistency data collaborative fusion service.

[0010] In one embodiment, the consistent data collaborative fusion service includes: decrypting and unpacking the structured data packet to obtain the hash consistency check code of the sensor perception data, and pushing the hash consistency check code of the sensor perception data to the data receiving secondary cache of the upper-level platform; unpacking the object file packet of the object file stream to obtain the hash consistency check code of the object file type data; retrieving the hash consistency check code of the sensor perception data that is identical to the hash consistency check code of the object file type data in the data receiving secondary cache of the upper-level platform; if the hash consistency check code of the sensor perception data that is identical to the hash consistency check code of the object file type data is retrieved, then pulling the sensor perception data and the object file type data at the same time, and simultaneously storing the sensor perception data and the object file type data on the disk persistently.

[0011] In the second aspect, an embodiment of the present invention provides a device for collecting and transmitting hybrid mine data, including: a data collection module, used to synchronously receive hybrid mine data through a target communication interface; wherein the hybrid mine data includes: sensor perception data and object file type data; a data separation module, used to create a hash consistency check code based on the hybrid mine data, and separate the hybrid mine data, and obtain a structured data packet corresponding to the sensor perception data and an object file stream corresponding to the object file type data based on the hash consistency check code; a data transmission module, used to transmit the structured data packet and the object file stream to an upper-level platform, so that the upper-level platform can perform collaborative fusion storage of the sensor perception data and the object file type data based on the hash consistency check code of the structured data packet and the object file stream.

[0012] In a third aspect, an embodiment of the present invention provides an electronic device, including a processor and a memory, wherein the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement the steps of any one of the methods provided in the first aspect above.

[0013] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of any one of the methods provided in the first aspect are executed.

[0014] The embodiments of the present invention bring the following beneficial effects: The above-mentioned method, device, electronic device and medium for collecting and transmitting mixed mine data provided by the embodiment of the present invention first synchronously receives mixed mine data (sensor perception data and object file type data) through the target communication interface; then creates a hash consistency check code based on the mixed mine data, and separates the mixed mine data, and obtains the structured data packet corresponding to the sensor perception data and the object file stream corresponding to the object file type data based on the hash consistency check code; finally, the structured data packet and the object file stream are transmitted to the upper-level platform, so that the upper-level platform performs collaborative fusion storage of the sensor perception data and the object file type data based on the hash consistency check code of the structured data packet and the object file stream. The above-mentioned method first synchronously receives mixed mine data through the target communication interface, which can ensure the consistency of data collection access; secondly, based on the hash consistency check code, the structured and unstructured mixed mine data are collaboratively transmitted and collaboratively fused and stored, which ensures the consistent collaborative transmission of sensor perception data and object file type data in the same business scenario.

[0015] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0016] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 A flow chart of a hybrid mine data collection and transmission method provided by an embodiment of the present invention; Figure 2 A schematic diagram of a hybrid mine data consistency acquisition and transmission system provided by an embodiment of the present invention; Figure 3 A schematic diagram of the structure of a hybrid mine data acquisition and transmission device provided by an embodiment of the present invention; Figure 4 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] In order to ensure the consistency of data usage and business attributes, it is required that single batches of structured data and unstructured data in the same business scenario need to be collected, transmitted and associated with each other. However, there is currently no solution to this problem.

[0021] Based on this, the embodiments of the present invention provide a method, device, electronic device and medium for hybrid mine data collection and transmission, which can ensure the consistent collaborative collection and transmission of sensor perception data and object file type data in the same business scenario.

[0022] To facilitate understanding of this embodiment, a hybrid mine data collection and transmission method disclosed in an embodiment of the present invention is first introduced in detail. The method can be executed by an electronic device, such as a smart phone, a computer, a tablet computer, etc. Figure 1 The flowchart of a hybrid mine data collection and transmission method shown in FIG. 1 illustrates that the method mainly includes the following steps S101 to S103: Step S101: synchronously receiving mixed mine data through a target communication interface.

[0023] Among them, mixed mining data includes: sensor perception data and object file type data. Sensor perception data can be structured data of sensor perception type collected by various sensors in the mining intelligent system, such as: environmental monitoring sensor data (methane, carbon monoxide, dust, oxygen, water level, pressure, displacement, etc.), equipment operation monitoring sensor data (current, voltage, power, vibration, harmonics, temperature, etc.); object file type data includes unstructured data such as audio and video AI intelligent recognition of structured data streams (for example: business alarm data for identifying people, objects, and environment, intelligent recognition of identification videos, recognition of voice pulses, etc.) and object file stream data (for example: video files, CAD drawings, telemetry data set documents, binary files, etc.).

[0024] In one implementation, the collection of mixed mine data needs to ensure the consistency of collection access, so the sensor perception data (structured data) and the object file type data (unstructured data) need to be collected and accessed through the same communication. In the embodiment of the present invention, the target communication interface can be used to synchronously receive the mixed mine data. The target communication interface can be a RESTFul-style API interface, that is, the software collection component provides an HTTP / TCP Server, and the interface realizes the synchronous reception of sensor perception data and object file type data.

[0025] Step S102: Create a hash consistency check code based on the mixed mine data, separate the mixed mine data, and obtain a structured data packet corresponding to the sensor perception data and an object file stream corresponding to the object file type data based on the hash consistency check code.

[0026] In one embodiment, the received mixed mine data is used to create a hash consistency check code through a consistency algorithm, and the sensor perception data and the object file type data are separated. When the data is separated, the hash consistency check code is injected into the sensor perception data and the object file type data respectively, and a structured data packet corresponding to the sensor perception data and an object file stream corresponding to the object file type data are obtained, and the structured data packet and the object file stream are pushed into the system cache.

[0027] Step S103: Transmit the structured data packet and the object file stream to the upper-level platform, so that the upper-level platform performs collaborative fusion storage of the sensor perception data and the object file type data based on the hash consistency check code of the structured data packet and the object file stream.

[0028] In one embodiment, the sensor perception data and object file type data pushed into the system cache are extracted from the cache and transmitted to the superior platform respectively. During data transmission, a structured data transmission channel and an unstructured data transmission channel are constructed to transmit the sensor perception data and the object file type data respectively. After uploading the data to the superior platform, the superior platform can obtain the hash consistency check code of the structured data packet and the object file stream respectively, match the sensor perception data and the object file type data according to the hash consistency check code, and perform collaborative fusion storage on the successfully matched sensor perception data and the object file type data. Among them, the superior platform can be a system platform for data analysis, calculation and decision-making, for example: the disaster warning monitoring platform system provides risk warnings to users after fusion analysis of mixed data, and the data governance platform classifies and governs the data to provide users with data knowledge graphs, etc.; when performing collaborative fusion storage, different storage media are first used according to the characteristics of various types of data, such as relational databases, document databases, time series databases, object storage, streaming media, etc., and then the identification and fusion of data in multiple storage media are realized through software algorithms. Unified access.

[0029] The above-mentioned method for hybrid mine data collection and transmission provided by an embodiment of the present invention first synchronously receives hybrid mine data through a target communication interface, thereby ensuring the consistency of data collection access; secondly, based on a hash consistency check code, collaborative transmission and collaborative fusion storage of structured and unstructured hybrid mine data are performed, thereby ensuring the consistent collaborative transmission of sensor perception data and object file type data in the same business scenario.

[0030] In one embodiment, for the aforementioned step S102, that is, when the mixed mining data is separated and the structured data packet corresponding to the sensor perception data and the object file stream corresponding to the object file type data are obtained based on the hash consistency check code, the following methods may be used, including but not limited to: For sensor perception data, first, the sensor perception data is serialized and the hash consistency check code is injected into the serialized sensor perception data; then, the sensor perception data injected with the hash consistency check code is encrypted to obtain a structured data packet corresponding to the sensor perception data.

[0031] In the specific implementation, the sensor perception data is first serialized, and the hash consistency check code is injected into the sensor perception data. Then, the sensor perception data injected with the hash consistency check code is encrypted and packaged as a whole package to obtain the structured data packet corresponding to the sensor perception data. Finally, the entire structured data packet is pushed into the system cache.

[0032] For object file type data, after receiving the stream terminator of the object file type data, the hash consistency check code is injected into the tail of the stream of the object file type data to obtain the object file stream corresponding to the object file type data.

[0033] In the specific implementation, the object file type data is received in a streaming manner. After receiving the stream terminator of the object file type data, the hash consistency check code is injected into the end of the stream to obtain the object file stream corresponding to the object file type data. The object file stream is then saved and the object file body and file handle are pushed into the system cache.

[0034] In one embodiment, different types of data need to use different communication technologies based on their data characteristics to realize the collection and transmission of their respective data. Therefore, in the embodiment of the present invention, independent transmission channels for different data types are constructed, namely, structured data transmission channels and unstructured data transmission channels, which are used to transmit sensor perception data and object file type data, respectively. Based on this, for the aforementioned step S103, that is, when transmitting the structured data packet and the object file stream to the upper-level platform, the following methods can be adopted, including but not limited to: (1) Transmit structured data packets to the upper-level platform through the structured data transmission channel.

[0035] Among them, the structured data transmission channel is implemented based on high-throughput timing message queue middleware.

[0036] In the specific implementation, the structured data packet is first published to the message queue; then after receiving the subscription information of the upper-level platform, the structured data packet subscribed by the upper-level platform is pulled from the message queue, and the structured data packet is transmitted to the upper-level platform.

[0037] Specifically, the sensor perception data in the system cache is published to the message queue on the lower-level platform, and the sensor perception data in the message queue is subscribed to on the upper-level platform. After receiving the subscription information of the upper-level platform, the subscribed sensor perception data is pulled and pushed into the data receiving cache of the upper-level platform.

[0038] (2) The object file stream is transmitted to the upper-level platform through the unstructured data transmission channel.

[0039] Among them, the unstructured data transmission channel is implemented based on object storage middleware.

[0040] In the specific implementation, the object file stream of the object file type data is first pushed to the object storage of the upper-level platform; then, the object receiving monitor is built through the upper-level platform, and the Hook call is executed when a new object file stream transmission is monitored, and the unstructured data receiving service is called to receive the object file stream, and the object file is packaged when the stream terminator of the object file stream is received to obtain the object file package.

[0041] Specifically, the lower-level platform actively pushes the object file type data in the system cache to the upper-level platform object storage; the upper-level platform builds an object receiving monitoring service, executes a Hook call when a new object file stream is detected, and calls a special unstructured data receiving service to receive the received object file stream. When the stream end character is received, the object file is packaged to obtain an object file package, and it is pushed to the upper-level platform data receiving cache.

[0042] In another embodiment, when transmitting data to a higher-level platform, a packet transmission method can be adopted to divide a large file or large data into multiple small data packets for transmission, thereby improving transmission efficiency and reducing transmission delay.

[0043] In one embodiment, for the aforementioned step S103, that is, when the sensor perception data and object file type data are collaboratively fused and stored based on the hash consistency check code of the structured data packet and the object file stream, the following methods can be adopted, including but not limited to: when the monitoring service of the upper-level platform monitors the object file stream transmission, a Hook call is executed to call the consistency data collaborative fusion service.

[0044] In the specific implementation, the upper platform builds a data consistency reception fusion monitoring service to monitor the data reception cache pushed to the upper platform. Since the sensor perception data transmission rate is much higher than the object file type data transmission rate, when the monitoring service monitors and receives the object file stream, the Hook call is executed, and the Hook calls the consistency data collaborative fusion service.

[0045] In one implementation, the consistent data collaborative fusion service includes: First, the structured data packet is decrypted and unpacked to obtain the hash consistency check code of the sensor perception data, and the hash consistency check code of the sensor perception data is pushed to the data receiving secondary cache of the upper-level platform.

[0046] In the specific implementation, the consistent data collaborative fusion service first decrypts and unpacks the received structured data packet, obtains the hash consistency check code of the sensor perception data, and pushes the hash consistency check code of the sensor perception data into the data receiving secondary cache, waiting for the consistency match after the object file type data object file packet arrives.

[0047] Then, the object file package of the object file stream is unpacked to obtain a hash consistency check code of the object file type data.

[0048] In specific implementation, the consistency data collaborative fusion service simultaneously unpacks the received object file package to obtain the hash consistency check code of the object file type data.

[0049] Finally, the hash consistency check code of the sensor perception data that is identical to the hash consistency check code of the object file type data is retrieved in the data receiving secondary cache of the upper platform; if the hash consistency check code of the sensor perception data that is identical to the hash consistency check code of the object file type data is retrieved, the sensor perception data and the object file type data are pulled at the same time, and the sensor perception data and the object file type data are simultaneously persistently stored on the disk.

[0050] In the specific implementation, the hash consistency check code of the sensor perception data that is identical to the hash consistency check code of the object file type data is retrieved in the data receiving secondary cache of the upper platform. When the sensor perception data with the same hash consistency check code is matched in the data receiving secondary cache, the sensor perception data and the object file type data are pulled out from the cache at the same time and persistently stored on the disk at the same time, thereby realizing the coordinated fusion storage of the sensor perception data and the object file type data for use by the mining business system.

[0051] The embodiment of the present invention provides a multi-dimensional data consistency acquisition and transmission technology based on the integration of Internet of Things data communication technology and object data communication technology. Based on high-throughput cache message queues and object storage middleware as data cache carriers, independent transmission channels for different data types are constructed, and consistency algorithms and multi-dimensional data collaborative algorithms are implemented through software coding. Various types of data are organized and collaboratively transmitted, realizing consistent mapping association, collaborative interception and information fusion management of data batches.

[0052] For ease of understanding, the present invention also provides a hybrid mine data consistency collection and transmission system, see Figure 2 As shown, it includes: a data collection access layer 201, a transmission channel layer 202 for various types of data, and a collaborative fusion layer 203 for multiple data types.

[0053] The data collection access layer 201 adopts the RESTFul style API interface, that is, the software collection component provides HTTP / TCP Server, and the interface realizes the synchronous reception of sensor perception data and object file type data; then the received mixed mining data is used to create a hash consistency check code through a consistency algorithm, and the sensor perception data and object file type data are separated. During separation, the sensor perception data line is serialized, the hash consistency check code is injected into the sensor perception data, and then the entire package of data is encrypted and packaged before the entire structured data packet is pushed into the system cache. Object file type data is received in a stream mode. After receiving the stream terminator, the hash consistency check code is injected into the end of the stream, and then the object file stream is saved and the object file body and file handle are pushed into the system cache.

[0054] The transmission channel layer 202 of various data extracts the sensor perception data and object file type data pushed into the system cache by the data acquisition access layer from the cache and transmits them to the upper system platform respectively. The transmission layer constructs structured data transmission channels and unstructured data transmission channels.

[0055] The structured data transmission channel is implemented based on the high-throughput time-series message queue middleware. The sensor perception data in the cache is published to the message queue on the lower-level platform, and the data in the message queue is subscribed to on the upper-level platform. The subscribed sensor perception data is pulled and pushed into the upper-level platform data receiving cache.

[0056] The unstructured data transmission channel is implemented based on the object storage middleware. The object file type data in the cache is actively pushed to the object storage of the upper-level platform on the lower-level platform. The upper-level platform builds an object receiving monitor, and executes a Hook call when a new object transmission stream is detected. The received object transmission stream is received by a special object file type data receiving service. When the stream terminator is found, the object file is packaged and pushed to the upper-level platform data receiving cache.

[0057] The collaborative fusion layer 203 of multiple data types is used for the collaborative fusion storage of mixed mining data. Specifically, the upper platform constructs a data consistency reception fusion listening service, listens and pushes it into the data reception cache of the upper platform, and executes a Hook call when the listening service listens and receives the object transmission stream, and the Hook calls the consistency data collaborative fusion service. First, the received structured data packet is decrypted and unpacked, and its hash consistency check code is obtained and pushed into the data reception secondary cache, waiting for the consistency match after the object file type data object file package arrives; at the same time, the received object file package is unpacked, and its hash consistency check code is obtained, and then the sensor perception data with the same hash consistency check code as its sensor perception data is retrieved in the upper platform data reception secondary cache. When it is found that the data with the same hash consistency check code is matched in the data reception secondary cache, the sensor perception data and the unstructured object data are pulled out from the cache at the same time and stored persistently on the disk at the same time. In this way, the collaborative fusion storage of sensor perception data and object file type data is realized for use by the mining business system.

[0058] The embodiment of the present invention realizes the collaborative fusion transmission of structured and unstructured mixed mining data through the collaborative processing of the above-mentioned data access layer, transmission layer, and collaborative fusion layer, and based on the consistency verification algorithm and the multi-dimensional data collaborative algorithm, providing data fusion support for the intelligent mining system. At the same time, the present invention is oriented to a variety of data usage scenarios, opens up the collaborative collection and transmission technology of structured and unstructured data, and provides comprehensive data support for artificial intelligence analysis and multi-dimensional data fusion analysis in the intelligent mining system.

[0059] For the hybrid mine data acquisition and transmission method provided in the above embodiment, the embodiment of the present invention also provides a hybrid mine data acquisition and transmission device, see Figure 3 The schematic diagram of the structure of a hybrid mine data acquisition and transmission device is shown, indicating that the device mainly includes the following parts: The data acquisition module 301 is used to synchronously receive mixed mine data through the target communication interface; wherein the mixed mine data includes: sensor perception data and object file type data; The data separation module 302 is used to create a hash consistency check code based on the mixed mine data, separate the mixed mine data, and obtain a structured data packet corresponding to the sensor perception data and an object file stream corresponding to the object file type data based on the hash consistency check code; The data transmission module 303 is used to transmit the structured data packet and the object file stream to the upper-level platform, so that the upper-level platform can perform collaborative fusion storage of the sensor perception data and the object file type data based on the hash consistency check code of the structured data packet and the object file stream.

[0060] The above-mentioned hybrid mine data collection and transmission device provided by the embodiment of the present invention first synchronously receives the hybrid mine data through the target communication interface, so as to ensure the consistency of data collection access; secondly, based on the hash consistency check code, the structured and unstructured hybrid mine data are collaboratively transmitted and collaboratively integrated and stored, so as to ensure the consistent collaborative transmission of sensor perception data and object file type data in the same business scenario.

[0061] In one embodiment, the above-mentioned data separation module 302 is specifically used to: serialize the sensor perception data and inject the hash consistency check code into the serialized sensor perception data; encrypt the sensor perception data injected with the hash consistency check code to obtain a structured data packet corresponding to the sensor perception data; after receiving the stream terminator of the object file type data, inject the hash consistency check code into the end of the stream of the object file type data to obtain the object file stream corresponding to the object file type data.

[0062] In one embodiment, the above-mentioned data transmission module 303 is specifically used to: transmit structured data packets to the upper-level platform through a structured data transmission channel; wherein the structured data transmission channel is implemented based on a high-throughput timing message queue middleware; transmit object file streams to the upper-level platform through an unstructured data transmission channel; wherein the unstructured data transmission channel is implemented based on an object storage middleware.

[0063] In one embodiment, the data transmission module 303 is specifically used to: publish a structured data packet to a message queue; after receiving subscription information from the upper-level platform, pull the structured data packet subscribed by the upper-level platform from the message queue, and transmit the structured data packet to the upper-level platform.

[0064] In one embodiment, the data transmission module 303 is specifically used to: push the object file stream of the object file type data to the object storage of the upper-level platform; construct an object receiving monitor through the upper-level platform, execute a Hook call when a new object file stream transmission is monitored, call the unstructured data receiving service to receive the object file stream, and package the object file when the stream terminator of the object file stream is received to obtain an object file package.

[0065] In one implementation, the data transmission module 303 is specifically used to: execute a Hook call when the monitoring service of the upper-level platform monitors the object file stream transmission, and call the consistency data collaborative fusion service.

[0066] In one embodiment, the above-mentioned consistency data collaborative fusion service includes: decrypting and unpacking the structured data packet to obtain the hash consistency check code of the sensor perception data, and pushing the hash consistency check code of the sensor perception data to the data receiving secondary cache of the upper-level platform; unpacking the object file packet of the object file stream to obtain the hash consistency check code of the object file type data; retrieving the hash consistency check code of the sensor perception data that is identical to the hash consistency check code of the object file type data in the data receiving secondary cache of the upper-level platform; if the hash consistency check code of the sensor perception data that is identical to the hash consistency check code of the object file type data is retrieved, then pulling the sensor perception data and the object file type data at the same time, and simultaneously storing the sensor perception data and the object file type data on the disk.

[0067] It should be noted that the implementation principle and technical effects of the device provided in the embodiment of the present invention are the same as those of the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding contents in the aforementioned method embodiment.

[0068] An embodiment of the present invention further provides an electronic device. Specifically, the electronic device includes a processor and a storage device. The storage device stores a computer program, and when the computer program is executed by the processor, it executes the method described in any one of the above implementation methods.

[0069] Figure 4 A structural diagram of an electronic device provided in an embodiment of the present invention, the electronic device 100 includes: a processor 40, a memory 41, a bus 42 and a communication interface 43, wherein the processor 40, the communication interface 43 and the memory 41 are connected via the bus 42; the processor 40 is used to execute an executable module stored in the memory 41, such as a computer program.

[0070] The memory 41 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 43 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.

[0071] The bus 42 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0072] Among them, the memory 41 is used to store programs, and the processor 40 executes the program after receiving the execution instruction. The method executed by the device for flow process definition disclosed in any embodiment of the above-mentioned embodiment of the present invention can be applied to the processor 40 or implemented by the processor 40.

[0073] The processor 40 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 40. The above processor 40 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiment of the present invention can be directly embodied as a hardware decoding processor to be executed, or the hardware and software modules in the decoding processor can be executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 41, and the processor 40 reads the information in the memory 41 and completes the steps of the above method in combination with its hardware.

[0074] The computer program product of the readable storage medium provided in the embodiment of the present invention includes a computer-readable storage medium storing program code, and the instructions included in the program code can be used to execute the method described in the previous method embodiment. The specific implementation can be referred to the previous method embodiment, which will not be repeated here.

[0075] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0076] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A method for hybrid mine data collection and transmission, characterized in that: include: Synchronously receiving mixed mine data through a target communication interface; wherein the mixed mine data includes: sensor perception data and object file type data of the mine; Creating a hash consistency check code based on the mixed mine data, separating the mixed mine data, and obtaining a structured data packet corresponding to the sensor perception data and an object file stream corresponding to the object file type data based on the hash consistency check code; The structured data packet and the object file stream are transmitted to an upper-level platform, so that the upper-level platform performs collaborative fusion storage of the sensor perception data and the object file type data based on the hash consistency check code of the structured data packet and the object file stream.

2. The method according to claim 1, characterized in that The mixed mining data is separated, and a structured data packet corresponding to the sensor perception data and an object file stream corresponding to the object file type data are obtained based on the hash consistency check code, including: Serializing the sensor perception data and injecting the hash consistency check code into the serialized sensor perception data; Encrypting the sensor perception data injected with the hash consistency check code to obtain a structured data packet corresponding to the sensor perception data; After receiving the stream terminator of the object file type data, the hash consistency check code is injected into the tail of the stream of the object file type data to obtain the object file stream corresponding to the object file type data.

3. The method according to claim 1, characterized in that Transmitting the structured data packet and the object file stream to a superior platform includes: The structured data packet is transmitted to the upper platform through a structured data transmission channel; wherein the structured data transmission channel is implemented based on a high-throughput timing message queue middleware; The object file stream is transmitted to the upper-level platform via an unstructured data transmission channel; wherein the unstructured data transmission channel is implemented based on object storage middleware.

4. The method according to claim 3, characterized in that: Transmitting the structured data packet to the upper-level platform through a structured data transmission channel includes: Publishing the structured data packet to a message queue; After receiving the subscription information of the upper-level platform, the structured data packet subscribed by the upper-level platform is pulled from the message queue, and the structured data packet is transmitted to the upper-level platform.

5. The method according to claim 3, characterized in that: Transmitting the object file stream to the upper-level platform through an unstructured data transmission channel includes: Pushing the object file stream of the object file type data to the object storage of the upper-level platform; The object receiving monitor is constructed through the upper-level platform, and a Hook call is executed when a new object file stream transmission is monitored, and the unstructured data receiving service is called to receive the object file stream, and the object file is packaged when the stream terminator of the object file stream is received to obtain an object file package.

6. The method according to claim 5, characterized in that The method comprises: performing collaborative fusion storage of the sensor perception data and the object file type data based on the hash consistency check code of the structured data packet and the object file stream, comprising: When the monitoring service of the upper-level platform monitors the object file stream transmission, a Hook call is executed to call the consistency data collaborative fusion service.

7. The method according to claim 6, characterized in that The consistent data collaborative fusion service includes: Decrypting and unpacking the structured data packet to obtain a hash consistency check code of the sensor-perceived data, and pushing the hash consistency check code of the sensor-perceived data to a data receiving secondary cache of the upper-level platform; Unpacking the object file package of the object file stream to obtain a hash consistency check code of the object file type data; Retrieving a hash consistency check code of the sensor perception data identical to the hash consistency check code of the object file type data in the data receiving secondary cache of the upper platform; If a hash consistency check code of the sensor perception data that is identical to the hash consistency check code of the object file type data is retrieved, the sensor perception data and the object file type data are simultaneously pulled and persistently stored on disk.

8. A hybrid mine data collection and transmission device, characterized in that: include: A data acquisition module, used for synchronously receiving mixed mine data through a target communication interface; wherein the mixed mine data includes: sensor perception data and object file type data; A data separation module, used to create a hash consistency check code based on the mixed mine data, separate the mixed mine data, and obtain a structured data packet corresponding to the sensor perception data and an object file stream corresponding to the object file type data based on the hash consistency check code; A data transmission module is used to transmit the structured data packet and the object file stream to the upper-level platform, so that the upper-level platform can perform collaborative fusion storage of the sensor perception data and the object file type data based on the hash consistency check code of the structured data packet and the object file stream.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are performed.

Images