Intelligent retrieval method for ship database
By classifying and preprocessing the ship database, building dynamic indexes and deep learning indexes, optimizing the data storage structure and retrieval paths, the problem of inefficient data retrieval in the ship database is solved, and efficient and accurate data management and improved ship design efficiency are achieved.
Patent Information
- Application Number
- CN202510498400.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-21
- Publication Date
- 2025-07-18
AI Technical Summary
In the prior art, the data retrieval of ship databases is inefficient and insufficient in accuracy, especially when large-scale data processing is difficult to meet the needs of high efficiency and high accuracy.
By classifying and preprocessing the ship database, dynamic indexes are constructed, including B+ tree index, B tree index, table jump and LSM-Tree index, combined with deep learning index, optimize the data storage structure and search path, and adjust the search conditions using hierarchical relationships, match and merge data, and generate target ship data in a fixed format.
It significantly improves the speed and accuracy of data retrieval, reduces data redundancy, enhances the flexibility of data management and system scalability, and improves the intelligence level and efficiency of ship design.
Smart Images
Figure CN120336391A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of ship design, and in particular to an intelligent retrieval method for ship databases. Background Art
[0002] In the field of intelligent ship hull line design, designers often need to quickly and accurately retrieve similar parent ships from a large ship database as design references. The data of these parent ships includes dimension parameters, performance data, historical design data, etc., which are usually stored in a relational database. Due to the lack of effective index support and data structure optimization, traditional data retrieval methods often result in slow retrieval speed and difficulty in accurately matching the data of parent ships similar to the design target. This not only increases the design cycle but also reduces the design efficiency.
[0003] With the continuous increase in data volume, traditional data retrieval methods have been difficult to meet the requirements of high efficiency and high accuracy, especially when dealing with complex queries and large data volumes, the efficiency often drops significantly. Therefore, there is an urgent need for an intelligent retrieval method for ship databases to solve the above problems. Summary of the Invention
[0004] The present invention aims to solve the problems of low efficiency and insufficient accuracy in large-scale data retrieval in the prior art, and proposes an intelligent retrieval method for ship databases, which is particularly suitable for processing the needs of large-scale ship design data, and significantly improves the intelligent level and efficiency of ship hull line design. The technical solution of the present invention is as follows:
[0005] An intelligent retrieval method for ship databases includes the following steps:
[0006] Classify and preprocess the data in the ship database to obtain a ship basic information table and a ship department parameter table;
[0007] Construct a dynamic index for the ship basic information table and the ship department parameter table;
[0008] Based on the retrieval conditions, screen the corresponding parent ships and their basic information from the ship basic information table to form a temporary intermediate table;
[0009] Perform a joint retrieval on the temporary intermediate table and the ship department parameter table to obtain the complete data of the target ship;
[0010] Process the complete data of the target ship to obtain a target ship data file in a fixed format;
[0011] Delete the temporary intermediate table.
[0012] A further technical solution thereof is that constructing a dynamic index for the ship basic information table and the ship department parameter table includes:
[0013] Dynamically select the indexing strategies for the ship basic information table and the ship department parameter table according to the query mode. The indexing strategies include B+ tree index, B tree index, skip list, and LSM-Tree.
[0014] A further technical solution thereof is to dynamically select the indexing strategy for the ship basic information table according to the query mode, including:
[0015] If the data in the table is frequently accessed, select the B+ tree index; if the data in the table is queried in a range, select the skip list; if a large amount of data is written into the table, select the LSM-Tree;
[0016] Dynamically select the indexing strategy for the ship department parameter table according to the query mode, including:
[0017] If the data in the table is frequently accessed, select the B tree index; if the data in the table is queried in a range, select the skip list; if a large amount of data is written into the table, select the LSM-Tree.
[0018] A further technical solution thereof is that the method further includes:
[0019] If the combined retrieval duration exceeds the set threshold, construct an index based on deep learning for the ship basic information table and the ship department parameter table, and re-perform the combined retrieval on the temporary intermediate table and the ship department parameter table.
[0020] A further technical solution thereof is that the index based on deep learning adopts a hierarchical model. The first layer uses a convolutional neural network to extract relevant features of the input data and input them to the second layer, and the second layer uses an MLP to predict the specific location of the target data.
[0021] A further technical solution thereof is that the method further includes optimizing the retrieval condition logic, including:
[0022] Based on the hierarchical relationship between ship data, adjust the elements in the retrieval conditions according to the master-slave relationship, and narrow the search range in combination with the ship navigation environment and ship attribute constraints.
[0023] A further technical solution thereof is to perform a combined retrieval on the temporary intermediate table and the ship department parameter table to obtain the complete data of the target ship, including:
[0024] Match the target ship number, target propeller model number, and main working conditions in the temporary intermediate table and the ship department parameter table, and use them as the primary key to merge the matched data to obtain the complete data of the target ship.
[0025] A further technical solution thereof is to process the complete data of the target ship, including:
[0026] Complete the missing values in the complete data of the target ship and ensure consistent data types;
[0027] Convert the processed target ship data into a fixed format and standardize the data structure.
[0028] A further technical solution thereof is that the processing of the complete data of the target ship further includes:
[0029] Based on the navigation log, expand the time series data of the specified data in the target ship data file.
[0030] A further technical solution thereof is that the data in the ship database is classified and preprocessed to obtain a ship basic information table and a ship department parameter table, including:
[0031] Utilize the differences in the attributes and uses of ship data to classify the data in the ship database, including the basic information of the ship, design parameters, historical design data, and test data;
[0032] For the classified data, complete the missing values therein and screen out abnormal data values, thereby forming a ship basic information table, a ship department parameter table, a offsets table, and a test information table;
[0033] Among them, the ship basic information table and the ship department parameter table are linked to the offsets table and the test information table, and the data in the linked tables corresponds based on the ship number and the propeller model number.
[0034] The beneficial technical effects of the present invention are:
[0035] First, a dynamic index is constructed for the ship basic information table and the ship department parameter table with the largest amount of data, that is, the index strategy is dynamically selected according to the query mode, which can greatly improve the speed and accuracy of data retrieval. In addition, if the retrieval duration of the constructed dynamic index exceeds the set threshold, the two tables are reconstructed with an index based on deep learning for a second attempt to further shorten the search duration. This method is particularly suitable for processing large-scale data sets and is particularly important for the large amount of complex data that is often processed in the field of ship design.
[0036] Second, this method significantly reduces data redundancy by optimizing the data storage structure and improves the overall performance and storage efficiency of the database. By classifying the data and adopting an intermediate table structure, the disk I / O operations during the query process are reduced, thereby accelerating the data access speed and reducing the system load.
[0037] Finally, this method enhances the manageability and scalability of data. By constructing appropriate indexes and clear data classification, the maintenance and expansion of the system become simpler and more intuitive. This is particularly important for a ship design database that needs to continuously update and maintain a large amount of dynamic data.
[0038] In summary, this application not only improves the efficiency and accuracy of ship database retrieval, but also improves the way data is organized, providing significant technical advantages for ship design and other related fields. Brief Description of the Drawings
[0039] Figure 1 is a schematic flowchart of the intelligent retrieval method for a ship database provided by this application.
[0040] Figure 2 is a schematic diagram of the target ship data in JSON format provided by this application. Detailed Description of the Preferred Embodiments
[0041] The following further describes the specific embodiments of the present invention with reference to the accompanying drawings.
[0042] This application proposes an intelligent retrieval method for a ship database, specifically designed to solve the problems of low data retrieval efficiency and insufficient accuracy in a large-scale ship database. By introducing an efficient data structure and index mechanism, this method not only significantly improves the retrieval speed, but also enhances the flexibility and accuracy of data management. The following is a detailed description of the invention content, and the process is shown in Figure 1 as shown, and specifically includes the following steps:
[0043] In the first step, the data in the ship database is classified and preprocessed to obtain a ship basic information table and a ship department parameter table.
[0044] In this embodiment, first, the data in the ship database is accurately classified, including the basic information of the ship, design parameters, historical design data, test data, etc. The key to this step is to use the differences in the attributes and uses of ship data to classify the data, so as to optimize the subsequent indexing and retrieval processes. For example, separating static design data from dynamic test data to adapt to different retrieval requirements and optimize the storage structure.
[0045] Secondly, the classified data is cleaned and preprocessed, including filling in the missing values in each category of data and screening out abnormal data values, so as to form a ship basic information table, a ship department parameter table, a form value table, and a test information table. Among them, the ship basic information table usually stores information such as the main dimensions, working conditions, and owners of each ship. The ship department parameter table usually stores the maximum dimensions (length, width, and height) and ship type dimensions (length between perpendiculars, beam, depth, draft) of each ship. This embodiment provides two methods for filling in missing values. One can use KNN filling, which fills in based on data of similar ship types. For example, missing values are filled by the similar characteristics of the nearest neighbor ships. The other can use the multiple imputation method (MICE), which uses regression prediction to fill in missing values. For example, missing resistance data is predicted based on the main dimensions of the ship. Among them, the constructed ship basic information table and ship department parameter table are linked to the form value table and the test information table, and the data in the linked tables corresponds based on the ship number and the propeller model number.
[0046] In the second step, a dynamic index is constructed for the ship basic information table and the ship department parameter table with the largest amount of data in the previous step.
[0047] In this embodiment, the index strategy of the ship basic information table and the ship department parameter table is dynamically selected according to the query mode. The index strategies include but are not limited to index strategies such as B+ tree index, B tree index, skip list, and LSM-Tree. For example, for the ship basic information table, if the data in the table needs to be frequently accessed, the B+ tree index is selected; if the data in the table is to be queried within a range, the skip list is selected; if batch writing optimization to the table is required, the LSM-Tree is selected. For the ship department parameter table, if the data in the table needs to be frequently accessed, the B tree index is selected; if the data in the table is to be queried within a range, the skip list is selected; if batch writing optimization to the table is required, the LSM-Tree is selected.
[0048] In the third step, the corresponding parent ship and its basic information are screened from the ship basic information table based on the retrieval conditions to form a temporary intermediate table Tempo.
[0049] In this embodiment, common retrieval conditions include draft, beam, length between perpendiculars, etc. The temporary intermediate table Tempo is used to store the key information of the parent ship and inherits the index and external links of the ship basic information table. Constructing Tempo can further narrow the retrieval range and support efficient data retrieval and rapid acquisition of information.
[0050] In the fourth step, a combined retrieval is performed on the temporary intermediate table Tempo and the ship department parameter table to obtain the complete data of the target ship.
[0051] In this embodiment, by matching the target ship number, target propeller model number, and main operating conditions (such as speed, draft, etc.) in the Tempo and the ship department parameter table, and using (target ship number, target propeller model number, speed, draft) as the primary key, the matched data is merged to obtain the complete data of the target ship. Among them, the form values and test information of the target ship can be obtained through the external linked form value table and test information table.
[0052] Step 5: Process the complete data of the target ship to support complex data analysis and ship design decisions, so as to obtain a target ship data file in a fixed format.
[0053] In this embodiment, first, clean and preprocess the complete data of the target ship, including: (1) Completing missing values, for example, using linear interpolation to fill in continuous variables such as speed and resistance data. (2) Ensuring consistent data types, for example, parsing numerical data, such as converting "10,500 DWT" to 10500. Secondly, convert the processed target ship data into a fixed format and standardize the data structure. For example, uniformly use the standard camel naming method for data, such as "Ship_Name" → "shipName", "Ship Type" → "shipType". This embodiment considers converting the data after combined retrieval and processing into a format that is easy to manage and analyze, such as JSON. Figure 2 A data output format in JSON format is given. The static data in JSON format constructs a static file of all data of a single ship through the corresponding relationship of key values. The selection of this format provides higher flexibility and scalability, facilitating cross-platform use of data and future technology upgrades. The data file after format conversion can be stored in the local system for designers to perform subsequent data analysis or directly apply it to design software.
[0054] Optionally, extend the time series data of the specified data in the target ship data file based on the navigation log. This method considers that in the actual ship data of the ship at sea, some data needs to be supported by time series data for subsequent management. For example:
[0055]
[0056]
[0057] Step 6: Delete the temporary intermediate table Tempo to free up the memory space of the local system.
[0058] Through the above technical solutions, the present invention not only realizes the efficient management and rapid retrieval of a large amount of data in the ship database, but also improves the flexibility of data processing and the scalability of the system. These technologies innovatively provide a new data processing tool for the ship design and manufacturing industries, greatly enhancing the ship design efficiency and data utilization rate.
[0059] Further, after the execution of the fourth step, the method further includes: if the combined retrieval duration exceeds a set threshold (such as 30 seconds), then construct a deep learning-based index for the ship basic information table and the ship department parameter table, and re-execute the third and fourth steps for a second attempt.
[0060] In this embodiment, the deep learning-based index can be implemented using a hierarchical model. The first layer uses a convolutional neural network to extract relevant features of the input data and input them to the second layer, and the second layer uses an MLP to predict the specific location of the target data. The method first makes a first attempt based on the basic dynamic index. If the effect of the basic index cannot meet the expected retrieval effect, then further uses the deep learning-based index to optimize the retrieval efficiency, improving the intelligence level of ship design.
[0061] Further, in the third step, the construction speed of the temporary intermediate table Tempo can be reduced by optimizing the retrieval conditions logically, and the waste of physical storage space can also be reduced, and the data access speed can be improved. The logical optimizations include: (1) Based on the hierarchical relationship between ship data (such as shipowner → ship → test data), adjust the elements in the retrieval conditions according to the master-slave relationship to avoid repeated queries. For example: SELECT*FROM Ship_Experiment WHERE shipId IN(
[0062] SELECT shipId FROM Ship_Info WHERE ownerId='A001'
[0063] ); / / First retrieve the shipowner to which the mother ship belongs to determine the target ship, and then perform a secondary retrieval through test data / working conditions
[0064] (2) Combine the ship navigation environment to reduce redundant queries. For example, the ship's speed data is highly correlated with the navigation environment, and the search range can be narrowed according to the navigation environment conditions during query. (3) Combine ship attribute constraints to narrow the search range. Many retrieval requirements can be directly inferred from the physical characteristics of the ship without full-scale search. For example, when the input ship length in the retrieval element is greater than 150, it is equivalent to covering the retrieval element with a ship carrying tonnage greater than 210,000 at the same time.
[0065] In this embodiment, two indexing strategies are particularly worth introducing. Among them, the B-tree index is a self-balancing tree that can maintain the balance of data among multiple levels of indexes. Each node of the B-tree can contain multiple keys (and values) and pointers to its child nodes.
[0066] The number of keys k contained in each node satisfies:
[0067] t - 1 ≤ k ≤ 2t – 1 (1)
[0068] The number of keys c contained in each leaf node satisfies:
[0069] t ≤ c ≤ 2t (2)
[0070] Where t is the minimum degree of the B-tree. At the same time, all leaf nodes are at the same depth.
[0071] For data tables that require frequent point queries, such as the ship department parameter table, the B-tree index is used. The B-tree index has high efficiency in processing a large number of update operations and can maintain data balance, making it suitable for dynamic query environments.
[0072] The B+-tree index is a variant of the B-tree index. All values and data pointers are stored in the leaf nodes, and the internal nodes only store key values without storing data. This index structure reduces the number of disk accesses, thereby further improving the retrieval efficiency, which makes the B+-tree widely used in database indexing.
[0073] For the B+-tree, the number of keys contained in each internal node satisfies:
[0074] t - 1 ≤ k ≤ 2t - 1 (3)
[0075] Constructing a B+-tree index on the ship basic information table is particularly suitable for fast range queries of large amounts of data. The characteristic of the B+-tree is that all value nodes exist in the leaf nodes, and the child nodes are connected by pointers, which makes sequential access more efficient, provides great convenience for full table scans and range queries, enables designers to find existing ships similar to the new ship design faster, and greatly improves the data processing ability and user experience in the ship design process. In addition, the non-leaf nodes of the B+-tree only store key value information, further improving the search speed of the index.
[0076] Based on the above constructed index structure, the system can quickly locate specific data nodes from the B+-tree or B-tree index according to the user's query requirements, reduce the number of data accesses, and thus improve the retrieval efficiency. This innovative technical method not only improves the intelligent level of ship design but also has broad application prospects, bringing a new solution for ship design.
[0077] The above are only the preferred embodiments of the present application, and the present invention is not limited to the above embodiments. It can be understood that other improvements and changes directly derived or associated by those skilled in the art without departing from the spirit and concept of the present invention shall be considered to be included within the protection scope of the present invention.
Claims
1. An intelligent retrieval method for a ship database, characterized in that The method includes: Classify and preprocess the data in the ship database to obtain a ship basic information table and a ship department parameter table; Build a dynamic index for the ship basic information table and the ship department parameter table; Based on the retrieval conditions, screen the corresponding parent ships and their basic information from the ship basic information table to form a temporary intermediate table; Perform a joint retrieval on the temporary intermediate table and the ship department parameter table to obtain the complete data of the target ship; Process the complete data of the target ship to obtain a target ship data file in a fixed format; Delete the temporary intermediate table.
2. The intelligent retrieval method for a ship database according to claim 1, wherein Building a dynamic index for the ship basic information table and the ship department parameter table includes: Dynamically select the index strategies for the ship basic information table and the ship department parameter table according to the query mode. The index strategies include B+ tree index, B tree index, skip list, and LSM-Tree.
3. The intelligent retrieval method for a ship database according to claim 2, wherein, Dynamically selecting the index strategy for the ship basic information table according to the query mode includes: If the data in the table is frequently accessed, select the B+ tree index; if the data in the table is queried within a range, select the skip list; if data is written to the table in batches, select the LSM-Tree; Dynamically selecting the index strategy for the ship department parameter table according to the query mode includes: If the data in the table is frequently accessed, select the B tree index; if the data in the table is queried within a range, select the skip list; if data is written to the table in batches, select the LSM-Tree.
4. The intelligent retrieval method for a ship database according to any one of claims 1 to 3, characterized in that, The method further includes: If the joint retrieval duration exceeds the set threshold, build an index based on deep learning for the ship basic information table and the ship department parameter table, and re-perform the joint retrieval on the temporary intermediate table and the ship department parameter table.
5. The intelligent retrieval method for a ship database according to claim 4, wherein The index based on deep learning adopts a hierarchical model. The first layer uses a convolutional neural network to extract relevant features of the input data and input them to the second layer, and the second layer uses an MLP to predict the specific location of the target data.
6. The intelligent retrieval method for a ship database according to claim 1, wherein The method further includes optimizing the retrieval condition logic, including: Based on the hierarchical relationship between ship data, adjust the elements in the retrieval conditions according to the master-slave relationship, and narrow the search range by combining the ship navigation environment and ship attribute constraints.
7. The intelligent retrieval method for a ship database according to claim 1, characterized in that Performing a joint retrieval on the temporary intermediate table and the ship department parameter table to obtain the complete data of the target ship includes: Match the target ship number, target propeller model number, and main operating conditions in the temporary intermediate table and the ship department parameter table, and use them as the primary key to merge the matched data to obtain the complete data of the target ship.
8. The intelligent retrieval method for a ship database according to claim 1, characterized in that Processing the complete data of the target ship includes: Complete the missing values in the complete data of the target ship and ensure that the data types are consistent; Convert the processed target ship data into a fixed format and standardize the data structure.
9. The intelligent retrieval method for a ship database according to claim 8, wherein Processing the complete data of the target ship further includes: Expand the time series data of the specified data in the target ship data file based on the navigation log.
10. The intelligent retrieval method for a ship database according to claim 1, wherein The classifying and preprocessing the data in the ship database to obtain a ship basic information table and a ship department parameter table includes: Utilize the differences in the attributes and uses of ship data to classify the data in the ship database, including the basic information, design parameters, historical design data, and test data of the ship; For the classified data, complete the missing values and identify abnormal data values, thereby forming a ship basic information table, a ship department parameter table, a offsets table, and a test information table; Among them, the ship basic information table and the ship department parameter table are linked to the offsets table and the test information table, and the data in the linked tables corresponds based on the ship number and the propeller model number.