Distributed map slice file storage method and device, equipment and medium

Abstract

The invention provides a distributed map slice file storage method and device, equipment and a medium, and relates to the technical field of image processing, and the method comprises the steps: obtaining the coordinate information of at least one to-be-processed slice file; for each to-be-processed slice file, determining a storage node corresponding to the to-be-processed slice file according to the coordinate information, and sending the to-be-processed slice file to the storage node; inputting the to-be-processed slice file into a picture recognition model of the storage node to obtain a recognition result and index information; when the identification result is repeated, storing the index information; and when the identification result is that the to-be-processed slice file is not repeated, storing the to-be-processed slice file and the index information. According to the technical scheme, the storage efficiency of the to-be-processed slice file can be improved, and redundant storage is reduced.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a distributed map slice file storage method, device, equipment and medium. Background Art

[0002] Map slicing is the process of dividing the entire map area into a series of small, regular image blocks (i.e., slices) according to certain rules (such as zoom level, geographic coordinate range, etc.). These slice files are usually used in applications such as online map services, navigation systems, and spatial data analysis.

[0003] Traditional map slice storage methods often use a centralized storage architecture. However, with the rapid increase in the number of slices, this centralized storage method faces problems such as high storage costs, large data access delays, and high single-point failure risks. Especially in large-scale, high-concurrency application scenarios, centralized storage architectures often fail to meet the needs of fast response and efficient processing. In addition, during the storage and management of map slice files, due to updates and changes in map data, new slice files may have duplicate content with stored slice files. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a distributed map slice file storage method, device, equipment and medium. By distributing the map slice files on multiple physical nodes, the parallel processing and load balancing of the map slice files are realized, thereby improving the storage efficiency. At the same time, by repetitive identification and establishment of index information, the redundant storage of map slice files is reduced, the utilization efficiency of storage space is optimized, and the storage cost is reduced.

[0005] In a first aspect, an embodiment of the present invention provides a distributed map slice file storage method, comprising: Obtaining coordinate information of at least one slice file to be processed; For each slice file to be processed, determine the storage node corresponding to the slice file to be processed according to the coordinate information, and send the slice file to be processed to the storage node; Input the slice file to be processed into the image recognition model of the storage node to obtain the recognition result and index information; When the identification result is a duplicate, the index information is stored; When the recognition result is non-duplicate, the slice file and index information to be processed are stored.

[0006] In a preferred embodiment of the present invention, for each slice file to be processed, determining the storage node corresponding to the slice file to be processed according to the coordinate information, and sending the slice file to be processed to the storage node, includes: For each slice file to be processed, determine the grid number of the slice file to be processed according to the correspondence between the coordinate information and the grid number; According to the grid number, query the storage node corresponding to the grid number, and send the slice file to be processed to the corresponding storage node.

[0007] In a preferred embodiment of the present invention, the above-mentioned inputting the slice file to be processed into the image recognition model of the storage node to obtain the recognition result and the corresponding index information includes: Obtaining a feature matrix to be processed of a slice file to be processed and a feature matrix of at least one map slice file associated with a storage node; Based on the arrangement order of the feature matrix, the feature matrix to be processed is compared with the feature matrix to obtain a comparison result; When the comparison result is a duplicate, determining that the recognition result is a duplicate; When the comparison result is non-repetitive, return to execute the arrangement order based on the feature matrix, compare the feature matrix to be processed with the feature matrix, and obtain the comparison result until all feature matrices are traversed and the recognition result is determined to be non-repetitive; According to the comparison results, the index information of the slice file to be processed is generated.

[0008] In a preferred embodiment of the present invention, the feature matrix to be processed is compared with the feature matrix to obtain a comparison result, including: Sequentially obtain the first data and the second data located at the same position in the feature matrix to be processed and the feature matrix; the first data is the data in the feature matrix to be processed, and the second data is the data in the feature matrix; When the difference between the first data and the second data is greater than or equal to a threshold, determining that the comparison result is non-repetitive; When the difference between the first data and the second data is less than a threshold value, return to execute the step of sequentially acquiring the first data and the second data located at the same position in the feature matrix to be processed and the feature matrix, until the first data at all positions in the feature matrix to be processed are traversed and it is determined that the comparison result is repeated.

[0009] In a preferred embodiment of the present invention, the index information of the slice file to be processed is generated according to the comparison result, including: When the comparison result is duplicate, the index information of the map slice file associated with the feature matrix is ​​obtained and used as the index information of the slice file to be processed; When the comparison result is no duplication, a storage location is allocated for the slice file to be processed; The coordinate information, grid number and storage location pointer are used as index information of the slice file to be processed.

[0010] In a preferred embodiment of the present invention, the above-mentioned storage index information includes: Add the index information of the slice file to be processed and the name of the slice file to be processed to the index file corresponding to the storage node.

[0011] In a preferred embodiment of the present invention, the method further comprises: Get the read instruction of the map slice file; According to the read instruction, query the index information of the map slice file; According to the pointer of the storage location in the index information, the map slice file is read from the storage location.

[0012] In a second aspect, an embodiment of the present invention further provides a distributed map slice file storage device, comprising: A coordinate information acquisition module, used to acquire coordinate information of at least one slice file to be processed; A storage node determination module is used to determine the storage node corresponding to each slice file to be processed according to the coordinate information, and send the slice file to be processed to the storage node; The recognition module is used to input the slice file to be processed into the image recognition model of the storage node to obtain the recognition result and index information; A first storage module, used for storing index information when the recognition result is a duplicate; The second storage module is used to store the slice file and index information to be processed when the recognition result is non-duplicate.

[0013] In a third aspect, an embodiment of the present invention further provides an electronic device, including a processor and a memory, wherein the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement the distributed map slice file storage method of the first aspect mentioned above.

[0014] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the distributed map slice file storage method of the first aspect mentioned above.

[0015] The embodiments of the present invention bring the following beneficial effects: The embodiment of the present invention provides a distributed map slice file storage method, which stores the slice files to be processed through a distributed system. In multiple concurrent scenarios, multiple slice files to be processed can be synchronously stored and processed in different storage nodes, thereby improving storage efficiency. At the same time, during the storage process, repeatability is identified through an image recognition model. When the slice files to be processed are repeated, only index information can be saved to facilitate reading of the slice files to be processed. At the same time, redundant storage of slice files to be processed is reduced, the use efficiency of storage space is optimized, and storage costs are reduced.

[0016] Other features and advantages of the present invention will be set forth in the following description, or some features and advantages may be inferred or unambiguously determined from the description, or may be learned by implementing the above-mentioned technology of the present invention.

[0017] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 A flowchart of a distributed map slice file storage method provided by an embodiment of the present invention; Figure 2 A flowchart of another distributed map slice file storage method provided by an embodiment of the present invention; Figure 3 A flowchart of another distributed map slice file storage method provided by an embodiment of the present invention; Figure 4 A schematic diagram of the structure of a distributed map slice file storage device provided by an embodiment of the present invention; Figure 5 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] Map slicing is the process of dividing the entire map area into a series of small, regular image blocks (i.e., slices) according to certain rules (such as zoom level, geographic coordinate range, etc.). These slice files are usually used in applications such as online map services, navigation systems, and spatial data analysis.

[0022] Traditional map slice storage methods often use a centralized storage architecture, that is, all slice files are stored on one or a few central servers. However, with the rapid increase in the number of slices, this centralized storage method faces problems such as high storage costs, large data access delays, and high single point failure risks. Especially in large-scale, high-concurrency application scenarios, centralized storage architectures often cannot meet the needs of fast response and efficient processing.

[0023] To overcome these limitations, distributed storage technology has been introduced into the management of map slice files. Distributed storage achieves parallel processing and load balancing of data by distributing and storing data on multiple physical nodes, thereby improving the scalability, availability, and performance of the storage system. However, distributed storage also brings new challenges, such as how to effectively determine the storage location of each slice file (i.e., the selection of storage nodes), and how to efficiently retrieve and manage these distributedly stored slice files.

[0024] In addition, in the process of storing and managing map slice files, the duplication of slice files needs to be considered. Due to the update and change of map data, new slice files may have duplicate content with stored slice files. If these duplicate slice files are not effectively identified and deduplicated, a lot of storage space will be wasted and the complexity of data retrieval will be increased.

[0025] Therefore, there is an urgent need for a distributed map slice file storage method that can efficiently and accurately determine the storage location of slice files and effectively identify and deduplicate duplicate slice files. This method can not only improve the overall performance and scalability of the storage system, but also optimize the use efficiency of storage space, reduce storage costs, and provide more stable and efficient data support for map services.

[0026] Based on this, a distributed map slice file storage method provided by an embodiment of the present invention can realize parallel processing and load balancing of map slice files by dispersing and storing map slice files on multiple physical nodes through distributed storage, thereby improving storage efficiency. At the same time, by repeatedly identifying and establishing index information, redundant storage of map slice files can be reduced, the utilization efficiency of storage space can be optimized, and storage costs can be reduced.

[0027] To facilitate understanding of this embodiment, a distributed map slice file storage method disclosed in an embodiment of the present invention is first introduced in detail.

[0028] Example 1 The embodiment of the present invention provides a distributed map slice file storage method. Figure 1 A flow chart of a distributed map slice file storage method provided in an embodiment of the present invention. The method is applied to a distributed storage system, which includes multiple storage nodes, and each storage node is used to store map slice files in a preset geographical area. Exemplarily, a Grid grid can be constructed across the country to form a grid of approximately 10 km × 10 km with a longitude of 0.09° and a latitude of 0.087°, forming a two-dimensional grid matrix. Each object in the two-dimensional grid matrix corresponds to a grid, and the longitude and latitude of the grid can be stored in the two-dimensional grid matrix. A storage node can store map slice files for multiple grids. The size of the map slice file is the same as the size of the grid. The technical solution of the embodiment of the present invention can be used to solve the problem that most of the files in the rural road map slice data of previous years are consistent, causing huge redundancy. Figure 1 As shown, the distributed map slice file storage method may include the following steps: Step S101, obtaining coordinate information of at least one slice file to be processed.

[0029] The slice file to be processed refers to a map slice file that needs to be stored in a distributed storage system. The coordinate information is used to describe the geographical area corresponding to the slice file to be processed. In an embodiment of the present invention, the coordinate information may be the longitude and latitude of the geographical area corresponding to the slice file to be processed.

[0030] Specifically, when receiving at least one slice file to be processed, the distributed storage system obtains coordinate information from the slice file to be processed.

[0031] Step S102: for each slice file to be processed, determine the storage node corresponding to the slice file to be processed according to the coordinate information, and send the slice file to be processed to the storage node.

[0032] Specifically, for each slice file to be processed, according to the coordinate information of the slice file to be processed, the grid corresponding to the coordinate information is queried in the two-dimensional grid matrix, and then according to the correspondence between the grid and the storage node, the storage node corresponding to the grid is queried, and the distributed storage system serves as the storage node corresponding to the slice file to be processed. The slice file to be processed is sent to the corresponding storage node, and the corresponding storage node stores the slice file to be processed.

[0033] Step S103, input the slice file to be processed into the image recognition model of the storage node to obtain the recognition result and index information.

[0034] The image recognition model is used to identify whether the slice file to be processed is repeated with the map slice file already stored in the storage node, and generate index information for the slice file to be processed. The image recognition model can be a convolutional neural network model. The index information is used to describe the storage location of the slice file to be processed. The recognition result is used to describe whether the slice file to be processed is repeated with the map slice file already stored in the storage node, including whether it is repeated or not.

[0035] Specifically, the storage node inputs the slice file to be processed into the image recognition model, and the image recognition model compares the slice file to be processed with at least one map slice file already stored in the storage node in sequence, and obtains the recognition result and index information output by the image recognition model. Exemplarily, the map slice files already stored in the storage node are file 1, file 2, and file 3. The image recognition model compares the slice file to be processed with file 1, file 2, and file 3 in sequence. If the slice file to be processed is the same as any file in file 1, file 2, and file 3, the recognition result is duplicate, and the index information of the file in file 1, file 2, and file 3 that is the same as the slice file to be processed is used as the index information of the slice file to be processed. If the slice file to be processed is not the same as all the files in file 1, file 2, and file 3, the recognition result is non-duplicate, and the storage node allocates a storage location for the slice file to be processed, and uses it as the index information of the slice file to be processed.

[0036] Step S104: When the identification result is a duplicate, store the index information.

[0037] The recognition result is duplicate, indicating that the map slice file already stored in the storage node is the same as the slice file to be processed. At this time, the storage node only stores index information, which can be the index information of the file in the map slice file already stored in the storage node that is the same as the slice file to be processed.

[0038] Specifically, when the recognition result is a duplicate, the index information of the slice file to be processed and the name of the slice file to be processed are added to the index file corresponding to the storage node.

[0039] The index file is used to store index information. Different storage nodes can correspond to different index files or the same index file. For example, the map slice files stored in the storage node are file 1, file 2 and file 3. The slice file to be processed is the same as file 2. At this time, the index information of file 2 is used as the index information of the slice file to be processed and stored in the index file, that is, the name of the slice file to be processed and the index information are stored in the index file.

[0040] Step S105, when the recognition result is non-duplicate, the slice file to be processed and the index information are stored.

[0041] When the recognition result is no duplication, it indicates that at least one map slice file stored in the storage node is completely different from the slice file to be processed. At this time, the storage node allocates a storage location for the slice file to be processed, thereby generating index information. The storage node stores the slice file to be processed in the allocated storage location, and adds the index information of the slice file to be processed and the name of the slice file to be processed to the index file corresponding to the storage node.

[0042] In another possible implementation, the storage node may not have a map slice file stored. In this case, the image recognition model directly determines the recognition result as non-duplicate, and generates index information according to the storage location assigned by the storage node to the slice file to be processed. The storage node stores the slice file to be processed in the assigned storage location, and adds the index information of the slice file to be processed and the name of the slice file to be processed to the index file corresponding to the storage node.

[0043] The embodiment of the present invention provides a distributed map slice file storage method, which stores the slice files to be processed through a distributed system. In multiple concurrent scenarios, multiple slice files to be processed can be synchronously stored and processed in different storage nodes, thereby improving storage efficiency. At the same time, during the storage process, repeatability is identified through an image recognition model. When the slice files to be processed are repeated, only index information can be saved to facilitate reading of the slice files to be processed. At the same time, redundant storage of slice files to be processed is reduced, the use efficiency of storage space is optimized, and storage costs are reduced.

[0044] Example 2 An embodiment of the present invention also provides another distributed map slice file storage method; this method is implemented on the basis of the method of the above embodiment; this method focuses on describing, for each slice file to be processed, determining the storage node corresponding to the slice file to be processed according to the coordinate information, sending the slice file to be processed to the storage node, inputting the slice file to be processed into the image recognition model of the storage node, and obtaining the specific implementation method of the recognition result and index information.

[0045] Figure 2 A flowchart of another distributed map slice file storage method provided by an embodiment of the present invention, such as Figure 2 As shown, the distributed map slice file storage method may include the following steps: Step S201, obtaining coordinate information of at least one slice file to be processed.

[0046] Step S202: for each slice file to be processed, determine the grid number of the slice file to be processed according to the correspondence between the coordinate information and the grid number.

[0047] A grid is constructed across the country, forming a 10 km × 10 km grid with longitude 0.09° and latitude 0.087°, forming a two-dimensional grid matrix. The two-dimensional grid matrix is ​​as follows: , in, The minimum longitude value and the maximum longitude value of the m-th row and m-th column grid are stored at.

[0048] The coordinate information of the slice file to be processed may include the minimum longitude value (χ min , γ min ) and the maximum longitude value (χ max , γ max ). The minimum longitude value and the maximum longitude value can be used to determine the grid of about 10 km × 10 km corresponding to the slice file to be processed. In a two-dimensional grid matrix, each object corresponds to a grid, and the correspondence between the coordinate information and the grid number can be established through the minimum longitude value and the maximum longitude value of the grid. For example, the coordinate information of the slice file to be processed is compared with By comparing the data stored at Is it the grid number of the slice file to be processed?

[0049] Specifically, the coordinate information can be compared with the minimum longitude value and the maximum longitude value stored at the grid number in the order of grid numbers. If they are the same, the grid number is determined as the grid number of the slice file to be processed. If they are not the same, the coordinate information is compared with the minimum longitude value and the maximum longitude value stored at the next grid number until all grid numbers in the two-dimensional grid matrix are traversed.

[0050] Step S203, according to the grid number, query the storage node corresponding to the grid number, and send the slice file to be processed to the corresponding storage node.

[0051] Specifically, in the distributed storage system, the grid number corresponding to each storage node is stored. By querying the storage node corresponding to the grid number, the slice file to be processed is sent to the corresponding storage node. For example, the grid number of the slice file to be processed is , by querying, The corresponding storage node is node 1, and the distributed storage system sends the slice file to be processed to node 1.

[0052] Step S204, obtaining a feature matrix of the slice file to be processed and a feature matrix of at least one map slice file associated with the storage node.

[0053] The feature matrix to be processed is used to describe the image features of the slice file to be processed. The at least one map slice file associated with the storage node refers to the map slice file actually stored in the storage node. It is understandable that when the storage node stores multiple map slice files, for duplicate map slice files, only one copy can be stored in the memory as the actually stored map slice file. The feature matrix is ​​used to describe the image features of the map slice file. It is understandable that the slice file to be processed can be a picture file.

[0054] Specifically, after the storage node inputs the slice file to be processed into the image recognition model, the image recognition model can pixelate the slice file to be processed, and then perform feature extraction to form a set of 256×256 two-dimensional RGB color feature matrices. It can be understood that the size of the slice file to be processed can be 256 pixels × 256 pixels. For each pixel in the slice file to be processed, the RGB value of the pixel is obtained to form a two-dimensional RGB color feature matrix, that is, the feature matrix to be processed. For at least one map slice file associated with the storage node, the feature matrix of the map slice file can be stored in the image recognition model, that is, the map slice file is pixelated, and then its feature matrix is ​​extracted to form a 256×256 two-dimensional RGB color feature matrix. Among them, the two-dimensional RGB color feature matrix can be expressed by the following formula: .

[0055] In the two-dimensional RGB color feature matrix, each object corresponds to the RGB value of the pixel at the same position. Corresponding to the RGB value of the pixel in the first row and first column, The RGB value of the pixel corresponding to the mth row and 1st column.

[0056] Step S205 : Based on the arrangement order of the feature matrix, the feature matrix to be processed is compared with the feature matrix to obtain a comparison result.

[0057] The arrangement order of the feature matrix can be set according to actual conditions. For example, the arrangement order of the feature matrix can be the order in which the map slice files corresponding to the feature matrix are stored in the storage node.

[0058] Specifically, based on the arrangement order of the feature matrix, the feature matrix to be processed is compared with the feature matrix to obtain a comparison result. If the comparison result is repeated, it means that the feature matrix is ​​the same as the feature matrix to be processed, and if the comparison result is not repeated, it means that the feature matrix is ​​different from the feature matrix to be processed.

[0059] Specifically, step A 1 -Step A 3 Determine the comparison results.

[0060] Step A 1 , sequentially obtain the first data and the second data located at the same position in the feature matrix to be processed and the feature matrix.

[0061] The first data is the data in the feature matrix to be processed, and the second data is the data in the feature matrix.

[0062] Step A 2 When the difference between the first data and the second data is greater than or equal to a threshold, the comparison result is determined to be non-repetitive.

[0063] Step A 3 , when the difference between the first data and the second data is less than the threshold, return to execute the step of sequentially acquiring the first data and the second data located at the same position in the feature matrix to be processed and the feature matrix, until the first data at all positions in the feature matrix to be processed are traversed, and it is determined that the comparison result is repeated.

[0064] Exemplarily, the feature matrix to be processed is , the feature matrix is , it is understandable that a jk and b jk is the RGB value of the pixel, so the first data is a jk (r, g, b), the second data is b jk (r, g, b), where r is the brightness of red, g is the brightness of green, and b is the brightness of blue. , Calculate a jk and b jk The difference is calculated and compared with the threshold. The threshold can be set according to the actual situation. An exemplary threshold is θ(r, g, b) = (5, 5, 5).

[0065] By exiting the loop when the comparison result is non-repetitive, there is no need to traverse the first data at all positions in the feature matrix to be processed, thereby improving the efficiency of determining the comparison result.

[0066] Step S206: when the comparison result is a duplicate, determine that the recognition result is a duplicate.

[0067] Specifically, when the comparison result is a duplicate, the recognition result is determined to be a duplicate, and step S208 is executed.

[0068] Step S207, when the comparison result is non-repetition, return to execute the arrangement order based on the feature matrix, compare the feature matrix to be processed with the feature matrix, and obtain the comparison result, until all feature matrices are traversed and the recognition result is determined to be non-repetition.

[0069] Specifically, when the comparison result is non-repetitive, the process returns to step S205 until all feature matrices are traversed and it is determined that the recognition result is non-repetitive.

[0070] Exemplarily, the map slice files associated with the storage node are file 1, file 2 and file 3, the feature matrix of file 1 is matrix 1, the feature matrix of file 2 is matrix 2, the feature matrix of file 3 is matrix 3, and the arrangement order of the feature matrices is matrix 1, matrix 2 and matrix 3. Based on the arrangement order of the feature matrices, the feature matrix to be processed is compared with matrix 1 to obtain a comparison result. If the comparison result is repeated, it is determined that the recognition result is repeated, and step 208 is executed. If the comparison result is not repeated, the feature matrix to be processed is compared with matrix 2 to obtain a comparison result. If the comparison result is repeated, it is determined that the recognition result is repeated, and step 208 is executed. If the comparison result is not repeated, the feature matrix to be processed is compared with matrix 3 to obtain a comparison result. If the comparison result is repeated, it is determined that the recognition result is repeated. If the comparison result is not repeated, all feature matrices have been traversed at this time, and it is determined that the recognition result is not repeated.

[0071] Step S208: Generate index information of the slice file to be processed according to the comparison result.

[0072] Specifically, step B 1 -Step B 3 Determine the index information.

[0073] Step B 1 ,When the comparison result is repeated, the index information of the map slice file associated with the feature matrix is ​​obtained and used as the index information of the slice file to be processed.

[0074] The map slice file associated with the feature matrix refers to the map slice file of the feature matrix obtained after pixelation and feature extraction. When the comparison result is repeated, the index information of the map slice file associated with the feature matrix can be read from the index file and used as the index information of the slice file to be processed.

[0075] Step B 2 When the comparison result is no duplication, a storage location is allocated for the slice file to be processed.

[0076] When the comparison result is non-duplicate, the storage node allocates a storage location for the slice file to be processed, and the storage location is used to store the slice file to be processed.

[0077] Step B 3 , the coordinate information, grid number and storage location pointer are used as index information of the slice file to be processed.

[0078] The pointer of the storage location is used to locate the storage location. The coordinate information, grid number and pointer of the storage location of the slice file to be processed are used as index information of the slice file to be processed.

[0079] The coordinate information, the grid number and the pointer of the storage location are used together as index information, which facilitates the subsequent reading of the slice file to be processed, thereby increasing the flexibility and accuracy of reading the slice file to be processed.

[0080] Step S209: When the identification result is a duplicate, store the index information.

[0081] Step S210: When the recognition result is non-duplicate, the slice file to be processed and the index information are stored.

[0082] The distributed map slice file storage method provided by the embodiment of the present invention can quickly determine the grid to which the slice file belongs based on the coordinate information of the slice file by dividing the map area into multiple grids and assigning a unique grid number to each grid. This grid-based data positioning method is more intuitive and efficient than the traditional method of directly calculating the storage location based on coordinates, which greatly reduces the computational complexity and time overhead. The correspondence between the grid division and the storage node can be flexibly configured according to the actual situation of the distributed storage system to achieve load balancing. In the process of comparing the feature matrix to be processed with the feature matrix, if the comparison result is repeated, the comparison process can be directly exited to determine the index information, which can improve the efficiency of the comparison process, thereby improving the efficiency of the storage node in storing the slice file to be processed.

[0083] Example 3 The embodiment of the present invention also provides another distributed map slice file storage method; this method is implemented on the basis of the method of the above embodiment; this method focuses on describing the specific implementation method of map slice file reading.

[0084] Figure 3 A flowchart of another distributed map slice file storage method provided by an embodiment of the present invention, such as Figure 3 As shown, the distributed map slice file storage method may include the following steps: Step S301, obtaining coordinate information of at least one slice file to be processed.

[0085] Step S302: for each slice file to be processed, determine the storage node corresponding to the slice file to be processed according to the coordinate information, and send the slice file to be processed to the storage node.

[0086] Step S303: input the slice file to be processed into the image recognition model of the storage node to obtain the recognition result and index information.

[0087] Step S304: When the identification result is a duplicate, store the index information.

[0088] Step S305: When the recognition result is non-duplicate, the slice file to be processed and the index information are stored.

[0089] Step S306, obtaining a read instruction for the map slice file.

[0090] It is understandable that after the slice file to be processed is stored in the storage node, it can be used as a map slice file. The read instruction is used to read the map slice file stored in the storage node.

[0091] Step S307: query the index information of the map slice file according to the read instruction.

[0092] The file name and / or coordinate information of the map slice file may be included in the read instruction. After receiving the read instruction, the distributed storage system may query the index information identical to the file name and / or coordinate information from the index file according to the file name and / or coordinate information in the read instruction, and use the index information as the index information of the map slice file.

[0093] Step S308: Read the map slice file from the storage location according to the pointer of the storage location in the index information.

[0094] The storage location is located according to the pointer of the storage location in the index information, and the map slice file is read from the storage location.

[0095] The distributed map slice file storage method provided by the embodiment of the present invention can quickly locate the storage location of the map slice file by querying the index information according to the read instruction. The pointer of the storage location in the index information provides a direct access path, avoiding the tedious process of traversing the entire storage system to find the file. Through the storage location pointer in the index information, the system can accurately locate the storage location of the map slice file, avoiding the problem of reading errors or data loss caused by inaccurate positioning, thereby significantly improving the reading efficiency and accuracy.

[0096] Example 4 Corresponding to the above method embodiment, the embodiment of the present invention provides a distributed map slice file storage device, Figure 4 A schematic diagram of a distributed map slice file storage device provided by an embodiment of the present invention is shown in FIG. Figure 4 As shown, the distributed map slice file storage device may include: A coordinate information acquisition module 401 is used to acquire coordinate information of at least one slice file to be processed; The storage node determination module 402 is used to determine the storage node corresponding to each slice file to be processed according to the coordinate information, and send the slice file to be processed to the storage node; The recognition module 403 is used to input the slice file to be processed into the image recognition model of the storage node to obtain the recognition result and index information; A first storage module 404 is used to store index information when the recognition result is a duplicate; The second storage module 405 is used to store the slice file to be processed and the index information when the recognition result is non-duplicate.

[0097] The embodiment of the present invention provides a distributed map slice file storage device, which stores the slice files to be processed through a distributed system. In multiple concurrent scenarios, multiple slice files to be processed can be synchronously stored and processed in different storage nodes, thereby improving storage efficiency. At the same time, during the storage process, repeatability is identified through an image recognition model. When the slice files to be processed are repeated, only index information can be saved to facilitate reading of the slice files to be processed. At the same time, redundant storage of slice files to be processed is reduced, the use efficiency of storage space is optimized, and storage costs are reduced.

[0098] In some embodiments, the storage node determination module 402 is further configured to: For each slice file to be processed, determine the grid number of the slice file to be processed according to the correspondence between the coordinate information and the grid number; According to the grid number, query the storage node corresponding to the grid number, and send the slice file to be processed to the corresponding storage node.

[0099] In some embodiments, the identification module 403 is further configured to: Obtaining a feature matrix to be processed of a slice file to be processed and a feature matrix of at least one map slice file associated with a storage node; Based on the arrangement order of the feature matrix, the feature matrix to be processed is compared with the feature matrix to obtain a comparison result; When the comparison result is a duplicate, determining that the recognition result is a duplicate; When the comparison result is non-repetitive, return to execute the arrangement order based on the feature matrix, compare the feature matrix to be processed with the feature matrix, and obtain the comparison result until all feature matrices are traversed and the recognition result is determined to be non-repetitive; According to the comparison results, the index information of the slice file to be processed is generated.

[0100] In some embodiments, the feature matrix to be processed is compared with the feature matrix to obtain a comparison result, which is also used for: Sequentially obtain the first data and the second data located at the same position in the feature matrix to be processed and the feature matrix; the first data is the data in the feature matrix to be processed, and the second data is the data in the feature matrix; When the difference between the first data and the second data is greater than or equal to a threshold, determining that the comparison result is non-repetitive; When the difference between the first data and the second data is less than a threshold value, return to execute the step of sequentially acquiring the first data and the second data located at the same position in the feature matrix to be processed and the feature matrix, until the first data at all positions in the feature matrix to be processed are traversed and it is determined that the comparison result is repeated.

[0101] In some embodiments, based on the comparison results, index information of the slice file to be processed is generated, which is also used for: When the comparison result is duplicate, the index information of the map slice file associated with the feature matrix is ​​obtained and used as the index information of the slice file to be processed; When the comparison result is no duplication, a storage location is allocated for the slice file to be processed; The coordinate information, grid number and storage location pointer are used as index information of the slice file to be processed.

[0102] In some embodiments, the index information is stored and further used for: Add the index information of the slice file to be processed and the name of the slice file to be processed to the index file corresponding to the storage node.

[0103] In some embodiments, the apparatus further comprises: The instruction acquisition module is used to obtain the reading instructions of the map slice file; A query module is used to query the index information of the map slice file according to the read instruction; The reading module is used to read the map slice file from the storage location according to the pointer of the storage location in the index information.

[0104] The device provided in the embodiment of the present invention has the same implementation principle and technical effects as those of the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding contents in the aforementioned method embodiment.

[0105] Example 5 The embodiment of the present invention also provides an electronic device for running the above-mentioned distributed map slice file storage method; see Figure 5 A structural schematic diagram of an electronic device is shown, which includes a memory 500 and a processor 501, wherein the memory 500 is used to store one or more computer instructions, and the one or more computer instructions are executed by the processor 501 to implement the above-mentioned distributed map slice file storage method.

[0106] Further, Figure 5 The electronic device shown further includes a bus 502 and a communication interface 503 , and the processor 501 , the communication interface 503 and the memory 500 are connected via the bus 502 .

[0107] The memory 500 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk storage. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 503 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used. The bus 502 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0108] The processor 501 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 501. The above processor 501 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present invention can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module may be located in a storage medium mature in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 500, and the processor 501 reads the information in the memory 500 and completes the steps of the method of the above embodiment in combination with its hardware.

[0109] An embodiment of the present invention also provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the above-mentioned distributed map slice file storage method. The specific implementation can be found in the method embodiment, which will not be repeated here.

[0110] The computer program product for performing a distributed map slice file storage method provided in an embodiment of the present invention includes a computer-readable storage medium storing a non-volatile program code executable by a processor. The instructions included in the program code can be used to execute the method described in the previous method embodiment. The specific implementation can be found in the method embodiment, which will not be repeated here.

[0111] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0112] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0113] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0114] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0115] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that can be executed by a processor. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., various media that can store program codes.

[0116] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-mentioned embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A distributed map slice file storage method, characterized in that: include: Obtaining coordinate information of at least one slice file to be processed; For each of the slice files to be processed, determine the storage node corresponding to the slice file to be processed according to the coordinate information, and send the slice file to be processed to the storage node; Input the slice file to be processed into the image recognition model of the storage node to obtain the recognition result and index information; When the recognition result is a duplicate, storing the index information; When the recognition result is non-duplicate, the slice file to be processed and the index information are stored.

2. The method according to claim 1, characterized in that The step of determining, for each of the slice files to be processed, a storage node corresponding to the slice file to be processed according to the coordinate information, and sending the slice file to be processed to the storage node includes: For each of the slice files to be processed, determining the grid number of the slice file to be processed according to the correspondence between the coordinate information and the grid number; According to the grid number, the storage node corresponding to the grid number is queried, and the slice file to be processed is sent to the corresponding storage node.

3. The method according to claim 2, characterized in that The step of inputting the slice file to be processed into the image recognition model of the storage node to obtain the recognition result and the corresponding index information includes: Obtaining a feature matrix to be processed of the slice file to be processed and a feature matrix of at least one map slice file associated with the storage node; Based on the arrangement order of the feature matrix, the feature matrix to be processed is compared with the feature matrix to obtain a comparison result; When the comparison result is a duplicate, determining that the recognition result is a duplicate; When the comparison result is non-repetitive, returning to execute based on the arrangement order of the feature matrix, comparing the feature matrix to be processed with the feature matrix to obtain a comparison result, until all feature matrices are traversed and it is determined that the recognition result is non-repetitive; According to the comparison result, index information of the slice file to be processed is generated.

4. The method according to claim 3, characterized in that The step of comparing the feature matrix to be processed with the feature matrix to obtain a comparison result includes: Sequentially acquiring the feature matrix to be processed and first data and second data located at the same position in the feature matrix; the first data is the data in the feature matrix to be processed, and the second data is the data in the feature matrix; When the difference between the first data and the second data is greater than or equal to a threshold, determining that the comparison result is non-duplicate; When the difference between the first data and the second data is less than a threshold value, return to execute the step of sequentially acquiring the first data and the second data located at the same position in the feature matrix to be processed and the feature matrix, until the first data at all positions in the feature matrix to be processed are traversed and it is determined that the comparison result is repeated.

5. The method according to claim 3, characterized in that: Generating index information of the slice file to be processed according to the comparison result includes: When the comparison result is repeated, the index information of the map slice file associated with the feature matrix is ​​obtained and used as the index information of the slice file to be processed; When the comparison result is non-duplicate, a storage location is allocated for the slice file to be processed; The coordinate information, the grid number and the pointer to the storage location are used as index information of the slice file to be processed.

6. The method according to claim 1, characterized in that The storing of the index information comprises: The index information of the slice file to be processed and the name of the slice file to be processed are added to the index file corresponding to the storage node.

7. The method according to claim 1, characterized in that Also includes: Get the read instruction of the map slice file; According to the read instruction, query the index information of the map slice file; The map slice file is read from the storage location according to the pointer of the storage location in the index information.

8. A distributed map slice file storage device, characterized in that: include: A coordinate information acquisition module, used to acquire coordinate information of at least one slice file to be processed; A storage node determination module, for determining, for each of the slice files to be processed, a storage node corresponding to the slice file to be processed according to the coordinate information, and sending the slice file to be processed to the storage node; A recognition module, used for inputting the slice file to be processed into the image recognition model of the storage node to obtain recognition results and index information; A first storage module, used for storing the index information when the recognition result is a duplicate; The second storage module is used to store the slice file to be processed and the index information when the recognition result is non-duplicate.

9. An electronic device, characterized in that: It includes a processor and a memory, the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement the distributed map slice file storage method described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are called and executed by the processor, the computer-executable instructions prompt the processor to implement the distributed map slice file storage method described in any one of claims 1 to 7.

Images