A multi-database fusion access method, device and system applied to a scientific research ship

Abstract

The application relates to the technical field of databases, and provides a multi-database fusion access method, device and system applied to a scientific research ship. The application maps the characteristics of collected data of the scientific research ship into data tags respectively, the data tags include FB-Type of frequency variation type, MB-Type of slow variation type and ST-Type of stability type, and different data channel strategies are matched for the data tags respectively; the collected data of different data tags will go to respective database engines along preset strategies; wherein the collected data with the FB-Type data tag is imported into a time series database; the collected data with the MB-Type data tag is imported into a document type database; and the collected data with the ST-Type data tag is imported into a relational database, so that the problem that heterogeneous data stored in a single database architecture is poor in practicability is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of database, in particular to a multi-database fusion access method, device and system applied to a scientific research ship. BACKGROUND

[0002] In intelligent ships, industrial control and multi-source perception systems, data types often exhibit characteristics of high-frequency collection, structural heterogeneity and diverse transmission. For example, in the context of a marine scientific research ship, the system needs to handle continuous time series data from on-board sensors, structured instruction data from task scheduling, log data from device health monitoring, and cache state data for temporary response. These data are significantly heterogeneous in structure, access strategy and storage method.

[0003] Current systems typically use a single database to manage all types of data, which has the following problems: (1) structural constraints do not match; (2) access efficiency is limited; (3) multi-dimensional index coupling is difficult; (4) storage resource configuration is redundant, etc. Therefore, the existing data access method is difficult to support intelligent scheduling of data flow in complex scenarios.

[0004] Therefore, it is urgent to overcome the defects of the prior art in the technical field. SUMMARY

[0005] The technical problem to be solved by the present application is to provide a multi-database fusion access method, device and system applied to a scientific research ship.

[0006] The present application adopts the following technical solutions:

[0007] In a first aspect, the present application provides a multi-database fusion access method applied to a scientific research ship. The characteristics of the collected data of the scientific research ship are mapped to data tags, wherein the data tags include: FB-Type, MB-Type and ST-Type, and different data path strategies are matched for each data tag, including:

[0008] The collected data of different data tags will go to their respective database engines along the preset strategy;

[0009] Among them, the collected data with FB-Type data tag is imported into time series database to support high-frequency write and window aggregation query; the collected data with MB-Type data tag is imported into document type database to support flexible structure and version update; the collected data with ST-Type data tag is imported into relational database to store tableable task information and index analysis results.

[0010] Further, the frequency-varying data includes collected data generated by one or more of an acoustic Doppler current profiler, a conductivity-temperature-depth device, and a weather station;

[0011] The slowly-varying data includes one or more of a ship's attitude, an autonomous underwater vehicle's operating trajectory, and a tidal variation;

[0012] The steady data includes one or more of an engine operating state, a station operation task start and end time, a station operation task coordinate, and a station operation task type.

[0013] Further, the frequency-varying data is not synchronized remotely by default between the research vessel and the roadbed server, and is only cached in a time series database of a local node, and the method further includes:

[0014] When a deviation between the currently collected frequency-varying data and the first callable data is greater than a deviation threshold, a remote synchronization request is triggered;

[0015] The first callable data is specifically device detection data made before the research vessel leaves the station, and a dynamic interval value of the frequency-varying data generated in advance according to a to-be-completed research task.

[0016] Further, the method further includes:

[0017] In combination with one or more of a task priority, a sensor correlation strength, and a load limiting factor, a data tag of the collected data is switched from the frequency-varying FB-Type to the slowly-varying MB-Type.

[0018] Further, the method specifically includes:

[0019] For collected data with a low task priority and involving a strongly correlated sensor, when a difference between a network bandwidth load and a load limiting factor is less than a first preset value, the data tag of the collected data is switched from the frequency-varying FB-Type to the slowly-varying MB-Type.

[0020] The collected data involving the strongly correlated sensor is collected data with a degree of dependence on other sensors greater than a second preset value.

[0021] Further, the slowly-varying MB-Type data is projected on a steady structure, and the method further includes:

[0022] A trajectory envelope line is formed for a ship's trajectory, and the envelope line is used as an input parameter of a steady ST-Type data tag.

[0023] Further, the method further includes:

[0024] The stable data is pre-played template comparison, and the deviation is encoded into the data packet.

[0025] In a second aspect, the application further provides a multi-database fusion access system applied to a scientific research ship, comprising:

[0026] The ship collects conductivity temperature depth continuous data, which is identified as frequency variable data and is temporarily stored in a local time series database;

[0027] When the ship returns to a station point, the system writes stable data into a relational database for calling by a dispatching center station; wherein the stable data comprises a station number and an operation plan;

[0028] If energy consumption of the ship along a line is abnormal, whether corresponding frequency variable data is dispatched is judged according to a deviation between structural type operation and environmental data, so as to feed back the frequency variable data to a roadbed server;

[0029] If continuous multiple task results are convergent, part of the frequency variable data is written into a document type database as an analysis trajectory envelope to participate in subsequent analysis and re-calibration.

[0030] Further, the frequency variable data is not remotely synchronized between the scientific research ship and the roadbed server by default, and is only cached in a time series database of a local node, and further comprising:

[0031] When a deviation between currently collected frequency variable data and first callable data is greater than a deviation threshold, a remote synchronization request is triggered;

[0032] The first callable data is specifically equipment detection data made before the scientific research ship leaves a station, and a dynamic interval value of the frequency variable data generated in advance according to a scientific research task to be completed.

[0033] In a third aspect, the application further provides a multi-database fusion access device applied to a scientific research ship, used for realizing the multi-database fusion access method applied to the scientific research ship in the first aspect, and the device comprises:

[0034] At least one processor; and a memory connected with the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the processor to execute the multi-database fusion access method applied to the scientific research ship in the first aspect.

[0035] In a fourth aspect, the application further provides a non-volatile computer storage medium, which stores computer executable instructions, and the computer executable instructions are executed by one or more processors to complete the multi-database fusion access method applied to the scientific research ship in the first aspect.

[0036] The application provides a multi-database fusion access mechanism, which respectively maps data characteristics collected in a scientific research ship into multiple data tags, and respectively matches different data channel strategies for each data tag according to the data tag, so that the collected data of different data tags will go to respective database engines along the preset strategy, that is, the data is respectively stored in the corresponding database according to the type of the data, thereby avoiding the problems of structure constraint mismatch, limited access efficiency, difficulty in coupling of multi-dimensional indicators, and storage resource configuration redundancy caused by storing all types of data in a single database. By identifying the data behavior type through the data tag, constructing a multi-path access strategy and a dynamic tag mapping mechanism, the differential scheduling, optimized storage and intelligent routing of heterogeneous data in the task execution period are realized, so that the system can perform multi-strategy heterogeneous storage of the collected data of different characteristics according to specific logic during the running process. BRIEF DESCRIPTION OF DRAWINGS

[0037] In order to more clearly illustrate the technical solutions of the embodiments of the application, the drawings needed to be used in the embodiments of the application will be briefly introduced. Obviously, the drawings described below are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0038] Figure 1 is a flowchart of a multi-database fusion access method applied to a scientific research ship provided by an embodiment of the application;

[0039] Figure 2 is a flowchart of a specific example of writing collected data according to data tags provided by an embodiment of the application;

[0040] Figure 3 is a specific example diagram of path mapping provided by an embodiment of the application;

[0041] Figure 4 is a specific example diagram of generating a path provided by an embodiment of the application;

[0042] Figure 5 is another specific example diagram of generating a path provided by an embodiment of the application;

[0043] Figure 6 is a specific example diagram of comparing first-callable data with real-time collected data provided by an embodiment of the application;

[0044] Figure 7 is another flowchart of a multi-database fusion access method applied to a scientific research ship provided by an embodiment of the application;

[0045] Figure 8is a flow diagram of another application of a multi-database fusion access method applied to a scientific research ship provided by an embodiment of the present application.

[0046] Figure 9 is a schematic diagram of a multi-database fusion access device applied to a scientific research ship provided by an embodiment of the present application. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solutions and advantages of the present application clearer and more apparent, the present application will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0048] In the description of the present application, the terms "inner", "outer", "longitudinal", "transverse", "upper", "lower", "top", "bottom", etc. indicate the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and do not require the present application to be constructed and operated in a particular orientation, therefore should not be understood as limiting the present application.

[0049] In the present application, the terms "first", "second", etc. are only for descriptive purposes and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second", etc. can explicitly or implicitly include one or more of the features. In the description of the present application, unless otherwise specified, the meaning of "multiple" is two or more.

[0050] In the present application, unless otherwise specified and limited, the term "connection" should be understood broadly, for example, "connection" can be fixed connection, or detachable connection, or integral; can be directly connected, or indirectly connected through intermediate medium. In addition, the term "coupling" can be an electrically connected mode for realizing signal transmission.

[0051] In addition, the technical features involved in each embodiment of the present application described below can be combined with each other as long as there is no conflict between them.

[0052] Embodiment 1:

[0053] In the current intelligent system, data shows the characteristics of extensive sources, diverse structures, rapid changes, etc. Especially in the fields of maritime affairs, industrial control, urban Internet of Things, etc., data not only includes high-frequency collected sensor information, but also includes periodic recorded device status, task-oriented analysis results, and instantaneous response data cache, etc. These heterogeneous data coexist in the same system, which brings great challenges to the traditional single database architecture.

[0054] Conventional data systems usually store all data uniformly in one database engine, which, however, causes a series of problems such as data type mismatch, low query efficiency, resource waste, high development cost, etc.

[0055] Therefore, there is an urgent need for a general architecture scheme capable of dynamically fusing multiple database technologies during system operation, so that the system can autonomously optimize data paths and select storage according to the running state, data characteristics and environmental requirements, thereby realizing more intelligent and efficient management capabilities.

[0056] Embodiment 1 of the present application provides a multi-database fusion access method applied to a scientific research ship, as shown in the figure, the characteristics of the collected data of the scientific research ship are mapped into data tags respectively, wherein the data tags include: FB-Type, MB-Type, and ST-Type, and different data path strategies are matched for each data tag respectively, and the method comprises: Figure 1

[0057] The collected data of different data tags will go to their respective database engines along the preset strategy, that is, each data tag corresponds to a respective path.

[0058] In the embodiment of the present application, the collected data of the scientific research ship is mapped into different data tags according to their respective characteristics, specifically including three types: FB-Type, MB-Type, and ST-Type. Among them, the collected data of the scientific research ship refers to the data collected by the equipment in the scientific research ship.

[0059] In one embodiment, the data type needs to be identified first to determine the data tag; in the embodiment of the present application, the collected data is classified according to semantic behavior, and the specific method is as follows:

[0060] The collected data with FB-Type data tag is called FB-type data, which includes one or more of the collected data generated by acoustic Doppler current profiler, conductivity temperature depth device and weather station. FB-type data usually has strong short-period fluctuations and is only used for evaluation in a specific period, for example: high-frequency collected data generated by acoustic Doppler current profiler (ADCP), conductivity temperature depth (CTD) device, weather station, etc.

[0061] ​In an embodiment, the frequency-varying data includes: scientific research equipment data; target data obtained by deploying specific equipment in a laying operation, wherein the equipment in the laying operation includes: active source OBS, passive source OBS, sediment trap, underwater video detection, lander, in-situ station, long baseline, etc.; and marine physical parameters measured by special equipment in a physical ocean operation, wherein the equipment in the physical ocean operation includes: SBE21, ADCP, sea-air flux observation equipment, and weather station.

[0062] The collected data with the slowly-varying MB-Type data tag is referred to as slowly-varying data, and the slowly-varying data includes one or more of ship attitude, AUV running track, and tide level change. The slowly-varying data has a relatively obvious pushing change track and is used for a following operation ship. The slowly-varying data changes slowly but still needs to be tracked. The slowly-varying data includes: ship attitude data, AUV running track data, and tide level change data.

[0063] In an embodiment, the slowly-varying data includes: following operation data, wherein the following operation includes AUV, HOV, Glider, etc.

[0064] The collected data with the stable ST-Type data tag is referred to as stable data, and the stable data includes one or more of engine running state, station operation task start and end time, station operation task coordinate, and station operation task type.

[0065] The stable data has a fixed structure and is stable, is core reference data of a task, and for example, includes engine running state data and station operation task parameters (such as start and end time, coordinate, and operation type).

[0066] In an embodiment, the stable data includes engine running state data, station operation data, research ship deployment and recovery / non-recovery operation and task state data, etc., wherein the engine running state data includes the first engine speed, the second engine speed, the third engine speed, etc., of the research ship; the station operation data includes data collected by water taking, sediment taking, rock taking, fine sampling, detection type operation, etc.; the research ship deployment and recovery and / or non-recovery operation includes collected data of active source OBS, passive source OBS, sediment trap, underwater video observation, lander, in-situ experiment station and long baseline operation, etc.; and the task state data includes leg, station number, planned start and end time, station longitude and latitude, operation step, etc., wherein the task state data is used to select a specific deployment operation according to the research task arrangement, add leg, station number, planned start and end time, station longitude and latitude, operation step, etc., confirm the deployment operation task and enter the pending release state.

[0067] In an embodiment of the present application, each data tag is matched with a different data path strategy, and the collected data of different data tags will go to the respective database engine along the preset strategy, i.e., each data tag corresponds to a respective path. The preset strategy is determined by a person skilled in the art according to the specific use scenario. An embodiment of the present application provides a specific example of the preset strategy as follows:

[0068] As shown in Figure 2 , the paths corresponding to the three types of data tags are as follows: the collected data with the FB-Type data tag is imported into a time series database to support high-frequency writing and window aggregation query; the collected data with the MB-Type data tag is imported into a document type database to support flexible structure and version update; and the collected data with the ST-Type data tag is imported into a relational database to store tableable task information and index analysis results.

[0069] The specific implementation of the time series database, the document type database and the relational database is determined by a person skilled in the art according to the specific use scenario; in an embodiment, the time series database can use QuestDB, the document type database can use MongoDB, and the relational database can use MySQL database.

[0070] As shown in Figure 3Fig. 1 is a schematic diagram of a data behavior projection mapping framework according to an embodiment of the present application; in an embodiment, each data stream node collecting data in real time from a research vessel acquires the collected data, and a state identifier maps the collected data to different data labels according to the characteristics of the collected data: frequency-varying type, slowly-varying type, and stable type, and different types of data labels form a multi-level index behavior description; by mapping the running data state to a set of data labels, the data decision intention is driven. And the mapping routing controller matches different data path strategies for each data label, and the collected data of different data labels will go to the respective database engine (i.e. the specific carrying entity of the abstract target storage node) along the preset strategy. Since the database engine storing different types of collected data is determined according to the data label, the system has the ability to reflect the data representation of the generic task without directly controlling the database instruction execution mode.

[0071] In an embodiment, the system can maintain a dynamic path diagram structure, and different data labels will go to different database engines along the preset conditions (e.g. frequency, structure and business urgency). In an optional embodiment, Redis cache can also be used for temporary state maintenance across stages; for example, using Redis cache to store online flag of storage device and / or task quick retrieval field, etc.

[0072] The present application provides a multi-database fusion access mechanism, which maps the data characteristics collected by the research vessel to multiple data labels, and matches different data path strategies for each data label according to the data label, and the collected data of different data labels will go to the respective database engine along the preset strategy, i.e. the data of different types are stored in the corresponding database, avoiding the use of a single database to store all types of data, resulting in problems such as structure constraint mismatch, access efficiency limitation, multi-dimensional index coupling difficulty, and storage resource configuration redundancy. By identifying the data behavior type through data label, constructing multi-path access strategy and dynamic label mapping mechanism, the heterogeneous data is differentially scheduled, optimized stored and intelligently routed in the task execution period, so that the system can perform multi-strategy heterogeneous storage of collected data with different characteristics according to specific logic during operation.

[0073] In an embodiment, in an actual application scenario, before the research vessel performs the task off the coast, the equipment on each research vessel is used to perform a pre-rehearsal of the task execution, i.e. to collect various data, and then to simulate the state of the research vessel performing the task after leaving the coast. For example, Figure 4As shown, after collecting data, each evaluation node classifies and evaluates the collected data according to the data access trigger condition of the collected data of a certain device at each time, and the system will perform two key evaluations: one is to determine whether it is high-frequency access, and the other is to evaluate whether the structure of the collected data is stable. According to the evaluation result of high-frequency access, if it is "yes", candidate path A is selected; if it is "no", candidate path B is selected. At the same time, according to the evaluation result of structural stability, if the structure is stable, candidate path C is selected. Finally, the information of these candidate paths will be transmitted to the fusion strategy to make a comprehensive judgment, so as to complete the path selection, and the selected path will be used for heterogeneous storage call, so as to realize efficient data access and processing. The specific way of path selection by the fusion strategy is selected by those skilled in the art according to the specific use scene. Among them, the collected data of a certain device at each time is selected as candidate path A, which means that the corresponding collected data is frequency-varying data, and the corresponding collected data is stored in the time series database according to the storage method of the frequency-varying data; the collected data of a certain device at each time is selected as candidate path B, which means that the corresponding collected data is slowly-varying data, and the corresponding collected data is stored in the document-type database according to the storage method of the slowly-varying data; the collected data of a certain device at each time is selected as candidate path C, which means that the corresponding collected data is stable data, and the corresponding collected data is stored in the relational database according to the storage method of the stable data. After the different collected data is stored in the heterogeneous database, the corresponding calling method can also change with the characteristics of the corresponding database, which is not limited here.

[0074] In one embodiment, as Figure 5The figure shows the architecture processing flow of the system of the embodiment of the application. First, the collected data flows in from the data access end, then the data behavior identifier analyzes and identifies the collected data to obtain corresponding analysis data; the analysis data enters the data tag mapping module, where the collected data is labeled and classified by different data tags (i.e., the frequency-variable FB-Type, the slow-changing MB-Type, and the stable ST-Type). Subsequently, the corresponding collected data is sent to the path generator to determine the transmission path of each collected data according to the corresponding data tag. Finally, the fusion scheduling gateway integrates and schedules the collected data of the same data tag within a certain period of time, and distributes the collected data to different database adapters (i.e., DB-A, DB-B, and DB-C) according to the transmission path of each data tag, so as to store to the corresponding database (i.e., the time series database, the document database, and the relational database), thereby realizing the final storage or further processing of the collected data. The application realizes archiving and calling by a data semantic driven mode, performs fuzzy parameter evaluation on the time, access frequency, structural complexity, and other dimensions of certain data to obtain data tags, connects different database capability units through a certain dynamic mapping logic at the bottom of the system, and therefore can realize unlimited database types and high compatibility with most modern databases.

[0075] In an embodiment, the data path selection can be adjusted according to the real-time context, and the specific strategy is as follows:

[0076] The frequency-variable data does not perform remote synchronization between the research vessel and the roadbed server by default, and is only cached in the time series database of the local node, and the method further comprises:

[0077] When the deviation between the currently collected frequency-variable data and the first callable data is greater than the deviation threshold, a remote synchronization request is triggered; wherein the first callable data is specifically the device detection data made before the research vessel leaves the station, and the dynamic interval value of the frequency-variable data generated in advance according to the to-be-completed research task.

[0078] Wherein, the research vessel leaving the station refers to the research vessel before it sets sail; the device detection data refers to the collected data when the research vessel rehearses before setting sail; the specific value of the deviation threshold and the specific value of the dynamic interval value are both determined by a person skilled in the art according to the specific use scenario, and are not limited here. The currently collected frequency-variable data is the collected data with the frequency-variable FB-Type data tag.

[0079] The primary callable data includes collected data and corresponding preset dynamic range values ​​obtained from pre-departure rehearsals. In one embodiment, when the research vessel conducts a rehearsal before setting sail, it records the values ​​of data collected by each device at each moment. The thresholds for the primary callable data, etc., are all the corresponding values ​​of each device at each moment, rather than a specific fixed value.

[0080] In this embodiment of the invention, the frequency-varying data is not remotely synchronized by default, but is only cached on the local node.

[0081] After the research vessel departs from shore and begins its mission, the data collected during the actual mission may deviate from the data collected during the rehearsal. For example... Figure 6 As shown, a remote synchronization request is triggered only when there is a significant deviation between the real-time collected data and the first-level callable data, i.e., when the deviation between the real-time collected data and the first-level callable data exceeds the deviation threshold. The data labels of the corresponding real-time collected data are then switched to store these real-time collected data in a document-oriented database (i.e., when the switched data labels are of the slowly changing type) or a relational database (i.e., when the switched data labels are of the stable type). This remote synchronization of the significant deviation significantly reduces bandwidth and cloud storage costs.

[0082] For example, the first-level callable data is a dynamic range value, which includes a maximum range value and a minimum range value, such as... Figure 6 The two dashed lines in the middle show the current frequency variation data as follows: Figure 6 The solid line indicates that before time t1, the deviation between the currently acquired frequency-varying data and the dynamic interval value is less than or equal to the allowable deviation threshold. Starting from time t1, the currently acquired frequency-varying data deviates significantly from the dynamic interval value, and the deviation between the acquired data and the dynamic interval value exceeds the allowable deviation threshold. This deviation only returns to less than or equal to the allowable deviation threshold at time t2 and remains within the range of less than or equal to the allowable deviation threshold.

[0083] In one embodiment, during the pre-departure rehearsal of the research vessel, all collected data are labeled with frequency-varying FB-Type data tags to record all simulated values. This data is stored in a time-series database according to the frequency-varying data storage method. This allows for subsequent comparison with the simulated values ​​before departure, storing and transmitting values ​​that differ significantly from the simulated values, thereby reducing the amount of data to be transmitted and stored, saving bandwidth, and improving the system's usability. Through model building and learning from historical research data, the relevant collected data from the research vessel after departure is transmitted differentiatedly according to three types: frequency-varying FB-Type, slowly varying MB-Type, and stable ST-Type.

[0084] The roadbed server is a non-offshore system platform, and is used for receiving part of the collected data (for example, data needing remote synchronization) of the scientific research ship before and after the scientific research ship sails off the shore; the roadbed server is arranged on the shore and is equivalent to a shore terminal platform. The roadbed server can also store the first callable data; the data needing remote synchronization can be stable data.

[0085] The embodiment of the application flexibly switches the data tag according to the system running state and the environmental requirement (for example, bandwidth limitation), and autonomously performs data path optimization and storage selection adjustment. The method further comprises the following steps:

[0086] The data tag of the collected data is switched from the frequency-varying type FB-Type to the slow-varying type MB-Type in combination with one or more of the task priority, the sensor association strength and the load limitation factor.

[0087] In an embodiment, specifically, the method specifically comprises: Figure 7 As shown in the figure, for the collected data related to a strongly associated sensor, when the difference between the network bandwidth load and the load limitation factor is less than a first preset value, the data tag of the collected data is switched from the frequency-varying type FB-Type to the slow-varying type MB-Type; wherein the collected data related to the strongly associated sensor is the collected data with a dependence degree on other sensors greater than a second preset value.

[0088] The preset level, the first preset value, the load limitation factor and the second preset value are determined by a person skilled in the art according to a specific use scenario, and are not limited herein. In the embodiment of the application, the importance of each task requirement can be represented by the task priority, and the preset level is a pre-specified threshold of the task priority. In an actual application scenario, there are often many task requirements that need to be based on the data collected by other sensors in the completion of other tasks, wherein the other sensors refer to sensors other than the target device performing the current task; the dependence degree on other sensors can be represented by a dependence level, and the second preset value is a pre-specified threshold of the dependence level. The load limitation factor is a threshold of the network bandwidth.

[0089] One task can correspond to multiple collected data; for collected data with low task priority, since it does not need high-frequency writing and does not need to participate in window aggregation query immediately, it does not need to be stored into the time series database immediately. When the network bandwidth is quickly occupied, the system will preferentially reduce the collected data which is neither important (i.e., low task priority) nor dependent on other sensors (i.e., involves strong correlation sensors); because the absence of these collected data will not cause the overall data chain to break, it can be indirectly calculated or compensated by other sensors, so pausing to upload them can free up bandwidth, ensure that critical task data (such as alarms, control instructions) is delivered in real time, and avoid system crashes or delays. Therefore, the embodiment of the present application switches the data tag of the collected data from the frequency-varying data to the slowly-varying data, so as to reduce the collected data with low task priority and involving strong correlation sensors from frequency-varying data to slowly-varying data, and uses the data transmission path of the slowly-varying data to process, so as to reduce the real-time load of the system as much as possible.

[0090] In an embodiment, as shown in FIG. 1, the slowly-varying MB-Type data is projected into a stable structure, and the method further comprises: forming a trajectory envelope for the accompanying trajectory, and taking the envelope as an input parameter of the stable ST-Type data tag. Figure 7

[0091] The accompanying trajectory is usually an ideal or measured relative motion trajectory. The stable structure projection can be: forming a trajectory envelope for the accompanying trajectory (i.e., slowly-varying data), and taking the trajectory envelope as one of the input parameters of the stable data. The stable structure projection refers to: mathematically projecting the accompanying trajectory in a stable coordinate system to obtain a stable form curve that varies with time but has removed high-frequency jitter. A three-dimensional (or two-dimensional) safety envelope is determined around the stable form curve, which can be understood as adding a shell to the trajectory, and the thickness of the shell is the maximum possible deviation of the accompanying trajectory within the safety range.

[0092] The envelope is the boundary line of the accompanying trajectory, indicating the maximum deviation range of the accompanying trajectory. For example, the envelope describes the safety range of the accompanying trajectory of the research vessel; when the actual accompanying trajectory exceeds the envelope, potential risks can be prompted, triggering system control for adjustment. The envelope is a kind of stable data, and the corresponding data tag is the stable ST-Type data tag. In this process, the envelope (i.e., stable data) is input as an input parameter into the controller for real-time control decision, and at this time the accompanying trajectory serves as a safety constraint to provide boundary conditions for the real-time control decision of the controller, thereby ensuring that the accompanying trajectory adjusted by the controller is still within the safety range.

[0093] ​The above mechanism forms an evolutionary dimension reduction path between data types, i.e., from frequent change to slow change, and from slow change to stability, thereby supporting automatic improvement of structural stability and utilization efficiency of data in the process of gradual clarification of data value.

[0094] In an embodiment, in order to improve the efficiency of remote reporting, the method further comprises: performing pre-play template matching on stable data, and encoding the deviation into a data packet.

[0095] Due to the characteristics of the frequent change type data, the data will change frequently according to the environment and business content, so the embodiment of the present application preferentially stores this kind of data only in the time series database of the ship itself, and exists as secondary callable combined data, serving the primary callable data.

[0096] The primary callable data is formed by the above-mentioned stable data as a source, and the characteristics of this kind of data are that it usually stands at a more macroscopic perspective to collect related data, such as collected ocean depth, total weight, and latitude and longitude, etc., and will also contain the release cable length, engine running state during the collection process, etc. This kind of data can obtain a pre-play template through large model data calculation when the research ship leaves the shore and goes to sea to perform tasks, and the corresponding pre-play template data (i.e., the primary callable data in the above) will be carried in the server of the research ship. The pre-play template data and the actual generated current real-time collection data are matched and analyzed in real time, and the analysis result (i.e., the serious deviation generated by matching or not matching) is attached as the optimal parameter in the data packet to be sent to the land platform through the satellite. The specific way of obtaining the pre-play template through large model data calculation and the real-time matching and analysis of the pre-play template data and the primary callable data is selected by a person skilled in the art according to the specific use scene. Since there is a pre-play template, only the secondary callable data generated in real time and deviating from the template needs to be carried in the data packet; the part that is consistent or has minimal deviation does not need to be transmitted, thereby greatly reducing the data transmission amount.

[0097] Since there is no need to occupy the satellite data bandwidth to send the frequent change type data collected during normal navigation to the land platform; and the secondary callable data is stored in the server, the more value and significance of the above-mentioned secondary callable data is that when the deviation exceeds the deviation threshold after analyzing the primary callable data, the research ship's own server will retrieve one or more secondary callable data associated with the current analysis of the primary callable data, form a data packet, and send it to the land platform.

[0098] The application takes a label as a pivot to abstract, takes into account data structure characteristics and task flow logic, constructs a cross-engine and cross-type data running atlas, and provides infrastructure protection for intelligent task management under a heterogeneous database. The data label is determined by dynamically sensing the data attribute, and the label-level routing and differentiated processing are realized. Through multi-database collaborative scheduling, the task scheduling flow no longer depends on a single structure index, and the retrieval efficiency is improved. The frequency-varying data is stored locally by default, and only the key difference is reported remotely, and the bandwidth occupation optimization is significant. For complex task processes (such as accompanying scientific investigation, multi-stage device deployment), the data-driven and link backtracking capabilities are improved.

[0099] On this basis, the embodiment of the application also provides a multi-database fusion access system applied to a scientific investigation ship, comprising:

[0100] The ship collects conductivity temperature depth continuous data, identifies it as frequency-varying data, and only temporarily stores it in a local time series database; when the ship returns to a station point, the system writes stable data into a relational database for calling by a scheduling center; wherein the stable data includes a station number and an operation plan; if energy consumption abnormity occurs along the line of the ship, whether the corresponding frequency-varying data is scheduled is judged according to the deviation between the structure type operation and the environmental data, so as to feed it back to a roadbed server; if continuous multiple task results tend to be the same, part of the frequency-varying data will be written into a document type database as an analysis track envelope to participate in subsequent analysis and re-calibration.

[0101] The embodiment of the application provides a data flow rule from sea collection to shore storage in the application scenario of a marine scientific investigation ship: the conductivity temperature depth continuous data collected on the ship is classified as frequency-varying data by the system; this kind of data is only temporarily cached in the local time series database on the scientific investigation ship, and is not immediately returned, so as to avoid occupying satellite bandwidth. In an embodiment, the scheduling center can be a management system that needs to use the data collected by the scientific investigation ship, and the specific system functions and properties of the scheduling center are determined by those skilled in the art according to the specific use scenario, which is not limited here. The scheduling center realizes the writing and calling of stable data by integrating a relational database, supports real-time decision-making and resource scheduling.

[0102] When the ship reaches the pre-planned station again (i.e., the ship returns to the station point), the system writes the stable data, i.e., the station number that does not change over time and the operation plan of this time, into the MySQL relational database; the scheduling center can subsequently directly query these static metadata through the MySQL relational database without the need to parse massive raw conductivity-temperature-depth data. If energy consumption anomalies occur during the voyage (i.e., energy consumption anomalies occur along the line of the ship), for example, a sudden increase in propulsion power, the system compares the structural operation data (for example, the planned speed, the load curve) and the environmental data (for example, the measured flow rate, the wind and wave) in real time, and calculates the deviation between the two; if the deviation exceeds the threshold value, the corresponding frequency variable data field (i.e., the key abnormal segment) is immediately returned to the shore platform through the satellite link for remote diagnosis; wherein the threshold value is determined by a person skilled in the art according to the specific use scenario. If the deviation analysis results of consecutive multiple tasks tend to be consistent, it means that some high-frequency variables (such as the turbulent dissipation rate) have statistical significance; then the system samples and compresses these frequency variable data into trajectory envelopes (i.e., statistical boundary curves) and writes them into the document database, which can be used for analysis and recalibration in the future, for example: model retraining, sensor drift recalibration, and / or route energy consumption baseline update.

[0103] The embodiment of the present application only temporarily stores the original conductivity-temperature-depth data on the ship; static metadata is written only when it returns to the station point; only the key segment is returned when it is abnormal; and when multiple tasks are verified to be valid, the statistical envelope is deposited into the document database for long-term optimization.

[0104] In addition, in an embodiment, the frequency variable data does not perform remote synchronization between the research ship and the roadbed server by default, and is only cached in the time series database of the local node, and further comprises:

[0105] When the deviation between the currently collected frequency variable data and the first callable data is greater than the deviable threshold, a remote synchronization request is triggered. The first callable data is specifically the device detection data made before the research ship leaves the station, and the dynamic interval value of the frequency variable data generated in advance according to the to-be-completed research task.

[0106] The multi-database fusion access mechanism of the embodiment of the present application is an abstract data fusion operation logic framework, and therefore does not depend on specific scheduling control structures and does not limit the accurate flow details of data, can adapt to multiple types of task processing scenarios, and has an evolutionary data path management capability.

[0107] For example, Figure 8As shown, the embodiment of the present application pre-configures the operation strategy for different types of collected data, and performs a rehearsal after the configuration, determines the data characteristics of each type of collected data according to the behavior feedback of various collected data, and maps the corresponding data tags. The data tags are used to distinguish the paths of various collected data types to the database engine. And in subsequent, flexibly according to the system running state and environmental demand, etc., independently carries out data path optimization and storage selection fine tuning, in order to switch the collected data from one data tag to another data tag, switch the database type of the real-time collected data storage; in one embodiment, when switching, the old database before switching can be interface transformed, and the new database after switching can be adapted according to the characteristics of the collected data.

[0108] The embodiment of the present application improves the adaptability of the system in a heterogeneous data environment, avoids the structural bottleneck caused by database mismatch when using a single database to store various data, realizes fuzzy control and intelligent generation of data processing path by corresponding different databases through data tags, and does not rely on manual configuration or static template; can adapt to various software and hardware platforms, and has universality in maritime dispatch, industrial equipment management, energy monitoring and other systems; has a certain module drift tolerance in design, and can support subsequent engine replacement and strategy update.

[0109] As Figure 9 shown, is the architecture schematic diagram of the multi-database fusion access device applied to the scientific research ship of the embodiment of the present application. The multi-database fusion access device applied to the scientific research ship of the embodiment of the present application includes one or more processors 21 and a memory 22. Among them, Figure 9 The processor 21 is taken as an example in this embodiment.

[0110] The processor 21 and the memory 22 can be connected through a bus or other means, Figure 9 for example, the connection through the bus is taken as an example.

[0111] The memory 22 is a kind of non-volatile computer readable storage medium, which can be used to store non-volatile software programs and non-volatile computer executable programs, such as the multi-database fusion access method applied to the scientific research ship in embodiment 1. The processor 21 executes the non-volatile software programs and instructions stored in the memory 22, thereby executing the multi-database fusion access method applied to the scientific research ship.

[0112] The memory 22 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state memory device. In some embodiments, the memory 22 can optionally include a memory that is remotely located relative to the processor 21, and these remote memories can be connected to the processor 21 through a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0113] The program instructions / modules are stored in the memory 22, and when executed by the one or more processors 21, perform the multi-database fusion access method for a research vessel in Embodiment 1 described above, for example, perform each step of the multi-database fusion access method for a research vessel in Embodiment 1 described above.

[0114] It is worth noting that the information interaction, execution process, etc. between the modules and units in the above-mentioned apparatus and system are based on the same concept as the processing method embodiments of the present application, and the specific content can be referred to the description in the method embodiments of the present application, which will not be described here.

[0115] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the embodiments can be completed by a program instructing relevant hardware, and the program can be stored in a computer-readable storage medium, which can include a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0116] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for multi-database fusion access applied to research vessels, characterized in that, The characteristics of the data collected by the research vessel are mapped to data tags, wherein the data tags include: frequency-varying, slowly varying, and stable types, and different data path strategies are matched for each data tag, including: Data collected under different data tags will be routed to their respective database engines according to preset strategies; Specifically, the collected data with frequently changing data tags will be imported into a time-series database as frequently changing data to support high-frequency writing and window aggregation queries; the collected data with slowly changing data tags will be imported into a document-type database as slowly changing data to support flexible structures and version updates; and the collected data with stable data tags will be imported into a relational database as stable data to store tableable task information and indicator analysis results. Among them, the frequency-varying data includes data collected by one or more of the following: acoustic Doppler current profiler, conductivity temperature depth device and weather station; the slowly varying data includes one or more of the following: the attitude of the escort vessel, the trajectory of the autonomous underwater vehicle and the tide level change; and the stable data includes one or more of the following: engine operating status, start and end time of the station operation task, coordinates of the station operation task, and operation type of the station operation task.

2. The multi-database fusion access method applied to research vessels according to claim 1, characterized in that, By default, frequency-varying data is not remotely synchronized between the research vessel and the land-based server; it is only cached in the time-series database of the local node. The method also includes: When the deviation between the currently collected frequency-varying data and the first-level callable data exceeds the deviation threshold, a remote synchronization request is triggered. Specifically, the first-level callable data includes equipment testing data performed before the research vessel leaves the site, and the dynamic range value of the frequency-varying data generated in advance according to the scientific research tasks to be completed.

3. The multi-database fusion access method applied to research vessels according to claim 1, characterized in that, The method further includes: By combining one or more of the following factors—task priority, sensor correlation strength, and load limiting factor—the data labels of the collected data can be dynamically adjusted from frequency-varying to gradually changing.

4. The multi-database fusion access method applied to research vessels according to claim 3, characterized in that, The method specifically includes: For task priority below a preset level, and involving data collected from strongly correlated sensors, when the difference between the network bandwidth load and the load limiting factor is less than a first preset value, the data label of the collected data is switched from frequency-varying type to slow-varying type. The data collected by the strongly correlated sensors refers to data that is more dependent on other sensors than a second preset value.

5. The multi-database fusion access method applied to research vessels according to claim 1, characterized in that, The method further includes projecting a stable structure onto slowly varying data types. A trajectory envelope is formed for the accompanying trajectory, and the envelope is used as the input parameter for the stable data label.

6. The multi-database fusion access method applied to research vessels according to claim 1, characterized in that, The method further includes: Stable data is compared with pre-simulated templates, and the deviation is encoded into the data packet.

7. A multi-database fusion access system for scientific research vessels, characterized in that, include: The characteristics of the data collected by the research vessel are mapped to data tags, including: frequency-varying, slowly varying, and stable types, and different data path strategies are matched for each data tag; the collected data of different data tags will follow the preset strategy to their respective database engines; The ship collects continuous data on conductivity, temperature, and depth, which is identified as frequency-varying data and is only temporarily stored in the local time-series database. When a ship returns to its destination, the system writes stable data into a relational database for use by the dispatching platform; the stable data includes the destination number and the operation plan. If abnormal energy consumption occurs along the route, the corresponding frequency-varying data will be dispatched based on the deviation between the structural operation data and the environmental data, and then fed back to the land-based server. If the results of multiple consecutive tasks converge, some of the frequently changing data will be written into a document-based database as the analysis trajectory envelope to participate in subsequent analysis and recalibration.

8. The multi-database fusion access system for research vessels according to claim 7, characterized in that, The frequency-varying data is not remotely synchronized between the research vessel and the land-based server by default; it is only cached in the time-series database of the local node. It also includes: When the deviation between the currently collected frequency-varying data and the first-level callable data exceeds the deviation threshold, a remote synchronization request is triggered. Specifically, the first-level callable data includes equipment testing data performed before the research vessel leaves the site, and the dynamic range value of the frequency-varying data generated in advance according to the scientific research tasks to be completed.

9. A multi-database fusion access device for use on a research vessel, characterized in that, include: At least one processor; And a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the processor for performing the multi-database fusion access method for scientific research vessels as described in any one of claims 1-6.

Images