Monitoring method, device, electronic device and storage medium for natural resource database

By extracting characteristic data from natural resource databases to filter target categories and creating parallel monitoring thread tasks, the problem of low data processing efficiency in the existing technology is solved, and the parallel processing and efficiency improvement of multiple data is achieved, meeting the needs of monitoring and update.

CN118152630BActive Publication Date: 2024-11-15广州市基础地理信息中心(广州市规划和自然资源自动化中心) +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410273127.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-03-11
Publication Date
2024-11-15
Estimated Expiration
2044-03-11

AI Technical Summary

Technical Problem

The existing natural resource database monitoring methods are inefficient when processing multiple data, making it difficult to meet monitoring needs. Especially when there are many types and types of data, the processing takes a long time and it is difficult to achieve timely updates.

Method used

By extracting the characteristic data of the pending data, filtering the target category, and creating multiple monitoring thread tasks that can be operated in parallel based on the number of data uploads, using the similarity calculation model to detect and update data, and using memory pool technology to optimize data storage and deal with.

Benefits of technology

It realizes parallel processing of multiple data, broadens the scope of database update and monitoring, shortens data processing time, improves processing efficiency, and meets existing monitoring and update needs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118152630B_ABST
    Figure CN118152630B_ABST
Patent Text Reader

Abstract

The present invention discloses a monitoring method, device, electronic device and storage medium for a natural resource database, the method comprising: when a monitoring user uploads data to be processed to the natural resource database, extracting feature data from the data to be processed; after using the feature data to screen the corresponding target category from the natural resource database, creating multiple monitoring thread tasks that can be operated in parallel according to the upload quantity of the data to be processed; calling each monitoring thread task to perform data change detection using the data to be processed and the historical data recorded by the target category, obtaining the detection result, and performing data update processing using the data to be processed according to the detection result. The present invention can process multiple data in parallel at the same time, which can not only broaden the update and monitoring scope of the database, but also shorten the processing time of the data and improve the processing efficiency of the data to meet the existing monitoring and update requirements.
Need to check novelty before this filing date? Find Prior Art

Description

Background Art

[0002] With the development of economy, the pollution of natural resources has also increased. Different pollution sources threaten the development of the current ecological environment to a certain extent. Therefore, it is necessary to monitor and manage natural resources at all times. Since natural resources involve a wide range and content, in order to coordinate and organize the information of various natural resources, a natural resource database can be set up to store various information in the natural resource database.

[0003] The natural resource database stores a large amount of data and involves a large number of types. In order to monitor and update the data in a timely manner, the currently commonly used monitoring method is to compare the currently entered data with the previously stored data one by one when the new data is received. If the value changes and it is determined that the data has changed, the currently entered data can replace the previously stored data to achieve data update.

[0004] However, the above monitoring method has the following technical problems: due to the large number of data types and categories involved, monitoring data changes one by one not only has a small monitoring scope and low processing efficiency, but once multiple data are received at the same time, the processing takes a long time and it is difficult to meet the existing monitoring needs. Summary of the invention

[0005] The present invention provides a natural resource database monitoring method, device, electronic device and storage medium, and the method can solve one or more of the above-mentioned technical problems.

[0006] A first aspect of an embodiment of the present invention provides a natural resource database monitoring method, the method comprising:

[0007] When the monitoring user uploads the data to be processed to the natural resource database, extracting feature data from the data to be processed;

[0008] After selecting the corresponding target category from the natural resource database using the characteristic data, a plurality of monitoring thread tasks capable of operating in parallel are created according to the uploaded quantity of the data to be processed;

[0009] Each of the monitoring thread tasks is called separately to use the data to be processed and the historical data recorded in the target category to perform data change detection to obtain a detection result, and the data to be processed is used for data update processing according to the detection result.

[0010] In a possible implementation manner of the first aspect, using the feature data to select one of the multiple data categories included in the natural resource database as a target category includes:

[0011] After converting the feature data into feature text, the feature text is segmented using a preset segmentation model to obtain a plurality of feature segmentations;

[0012] Aggregate and classify the multiple feature segmented words through an aggregation classification algorithm to obtain a number of segmented word categories;

[0013] Inputting the characteristic segmentation corresponding to each segmentation category into a preset similarity calculation model to calculate the similarity between each segmentation category and each data category of the natural resource database, and obtaining a plurality of category similarities;

[0014] According to the numerical value of the category similarity, the corresponding data category is selected as the target category.

[0015] In a possible implementation manner of the first aspect, the preset similarity calculation model includes: M network modules and a Softmax module connected in sequence;

[0016] Each network module includes a residual network unit and a score matching layer arranged in sequence, the residual network unit includes a linear layer, a residual connection layer and a normalization layer arranged in sequence, and the score matching layer includes a first connection layer, a first activation layer, a second connection layer, a second activation layer, a first correction layer and a third activation layer connected in sequence.

[0017] In a possible implementation manner of the first aspect, the operation of calculating the category similarity includes:

[0018] Performing matrix processing on the feature segmentations corresponding to the segmentation categories to obtain a segmentation matrix, and inputting the segmentation matrix into the linear layer to calculate an attention score;

[0019] Using the residual connection layer to propagate the attention score in forward and reverse directions to obtain a direction matrix, and using the normalization layer to integrate the direction matrix to obtain an integrated matrix;

[0020] Using the first connection layer, the first activation layer, the second connection layer, the second activation layer, the first correction layer and the third activation layer to perform dimension raising and lowering processing in sequence to obtain a processing matrix;

[0021] The Softmax function of the Softmax module is used to perform similarity calculation on the processing matrix to obtain category similarity.

[0022] In a possible implementation manner of the first aspect, after the step of performing data update processing using the to-be-processed data according to the detection result, the method further includes:

[0023] Searching for a feature image of the data to be processed according to the target category;

[0024] After adjusting the chromaticity value or image contour of the feature image to form an adjusted image, the adjusted image and the data to be processed are used to construct a knowledge graph for user viewing.

[0025] In a possible implementation manner of the first aspect, after the step of performing data update processing using the to-be-processed data according to the detection result, the method further includes:

[0026] Obtain the timestamp of the data to be processed uploaded by the user, extract several numbers from the timestamp according to the preset rules and concatenate them to generate an encrypted value;

[0027] The encrypted value is used to encrypt and save the data to be processed.

[0028] In a possible implementation manner of the first aspect, the creating a plurality of monitoring thread tasks that can operate in parallel according to the upload quantity of the to-be-processed data includes:

[0029] After creating the main thread, the calculation time of the data to be processed is calculated, and the number of threads that the built-in processor can be divided into is determined according to the calculation time;

[0030] If the number of threads is greater than or equal to the number of uploaded data to be processed, the memory pool technology is used to divide the built-in processor memory into several storage partitions according to the number of uploaded data to be processed and store the data to be processed, and then a monitoring thread task responsible for executing the operation is created according to the number of uploaded data to be processed.

[0031] A second aspect of an embodiment of the present invention provides a monitoring device for a natural resource database, the device comprising:

[0032] A data extraction module, used for extracting characteristic data from the data to be processed when the monitoring user uploads the data to be processed to the natural resource database;

[0033] Creating a task module, which is used to create a plurality of monitoring thread tasks that can be operated in parallel according to the uploaded quantity of the data to be processed after filtering the corresponding target category from the natural resource database using the feature data;

[0034] The monitoring update module is used to call each of the monitoring thread tasks to use the data to be processed and the historical data recorded in the target category to perform data change detection, obtain detection results, and use the data to be processed to perform data update processing according to the detection results.

[0035] In a possible implementation manner of the second aspect, using the feature data to select one of the multiple data categories included in the natural resource database as a target category includes:

[0036] After converting the feature data into feature text, the feature text is segmented using a preset segmentation model to obtain a plurality of feature segmentations;

[0037] Aggregate and classify the multiple feature segmented words through an aggregation classification algorithm to obtain a number of segmented word categories;

[0038] Inputting the characteristic segmentation corresponding to each segmentation category into a preset similarity calculation model to calculate the similarity between each segmentation category and each data category of the natural resource database, and obtaining a plurality of category similarities;

[0039] According to the numerical value of the category similarity, the corresponding data category is selected as the target category.

[0040] In a possible implementation manner of the second aspect, the preset similarity calculation model includes: M network modules and a Softmax module connected in sequence;

[0041] Each network module includes a residual network unit and a score matching layer arranged in sequence, the residual network unit includes a linear layer, a residual connection layer and a normalization layer arranged in sequence, and the score matching layer includes a first connection layer, a first activation layer, a second connection layer, a second activation layer, a first correction layer and a third activation layer connected in sequence.

[0042] In a possible implementation manner of the second aspect, the operation of calculating the category similarity includes:

[0043] Performing matrix processing on the feature segmentations corresponding to the segmentation categories to obtain a segmentation matrix, and inputting the segmentation matrix into the linear layer to calculate an attention score;

[0044] Using the residual connection layer to propagate the attention score in forward and reverse directions to obtain a direction matrix, and using the normalization layer to integrate the direction matrix to obtain an integrated matrix;

[0045] Using the first connection layer, the first activation layer, the second connection layer, the second activation layer, the first correction layer and the third activation layer to perform dimension raising and lowering processing in sequence to obtain a processing matrix;

[0046] The Softmax function of the Softmax module is used to perform similarity calculation on the processing matrix to obtain category similarity.

[0047] In a possible implementation manner of the second aspect, the apparatus further includes:

[0048] An image search module is used to search for a characteristic image of the data to be processed according to the target category;

[0049] A knowledge graph module is constructed to adjust the chromaticity value or image contour of the feature image to form an adjusted image, and then the adjusted image and the data to be processed are used to construct a knowledge graph for user viewing.

[0050] In a possible implementation manner of the second aspect, the apparatus further includes:

[0051] A timestamp acquisition module is used to obtain the timestamp of the data to be processed uploaded by the user, extract several numbers from the timestamp according to a preset rule, and concatenate them to generate an encrypted value;

[0052] The encryption module is used to encrypt and save the data to be processed using the encryption value.

[0053] In a possible implementation manner of the second aspect, the creating a plurality of monitoring thread tasks that can operate in parallel according to the upload quantity of the to-be-processed data includes:

[0054] After creating the main thread, the calculation time of the data to be processed is calculated, and the number of threads that the built-in processor can be divided into is determined according to the calculation time;

[0055] If the number of threads is greater than or equal to the number of uploaded data to be processed, the memory pool technology is used to divide the built-in processor memory into several storage partitions according to the number of uploaded data to be processed and store the data to be processed, and then a monitoring thread task responsible for executing the operation is created according to the number of uploaded data to be processed.

[0056] Compared with the prior art, the monitoring method, device, electronic device and storage medium of a natural resource database provided by an embodiment of the present invention have the following beneficial effects: when a monitoring user uploads data to be processed to the natural resource database, the present invention can determine the category of the data in the natural resource database according to the characteristics of the data to be processed and create multiple monitoring thread tasks that can be operated in parallel according to the number of data to be processed, and process multiple tasks in parallel to detect and process multiple data to be processed and data of their corresponding categories, so as to update the data according to the detection results; by processing multiple data in parallel at the same time, not only can the update and monitoring scope of the database be broadened, but also the processing time of the data can be shortened and the processing efficiency of the data can be improved to meet the existing monitoring and update needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 It is a flow chart of a natural resource database monitoring method provided by one embodiment of the present invention;

[0058] Figure 2 is a structural diagram of a similarity calculation model provided by an embodiment of the present invention;

[0059] Figure 3 It is a structural diagram of a monitoring device for a natural resource database provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0060] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] In order to solve the above problems, a natural resource database monitoring method provided in an embodiment of the present application will be introduced and explained in detail through the following specific embodiments.

[0062] Reference Figure 1 , showing a flow chart of a natural resource database monitoring method provided by an embodiment of the present invention.

[0063] The method is applicable to a management system of a natural resource database, and the management system may be an intelligent device or a computer device.

[0064] Wherein, as an example, the monitoring method of the natural resource database may include:

[0065] S11. When a monitoring user uploads data to be processed to a natural resource database, feature data is extracted from the data to be processed.

[0066] In one embodiment, the user may interact with the management system, or the management system may interact with the user's smart terminal, and the user may upload data through his smart terminal, which may be relevant data collected by the user about natural resources.

[0067] For example, if a user conducts exploration and measurement of abandoned farmland, the uploaded data to be processed may include: the measured area of ​​the farmland, location coordinates, property attributes, farmland owner information, measurement time, etc. For another example, if a user conducts measurement of an outdoor fish pond, the uploaded data to be processed may include: the measured area of ​​the fish pond, location coordinates of the fish pond, property attributes of the fish pond, annual fish production of the fish pond, photographed images of the fish pond, measurement time of the fish pond, etc.

[0068] Since the data to be processed contains a lot of data and the data formats are also different, in order to facilitate its classification and storage of each data type, feature data can be extracted from the data to be processed, so that the category of the data can be determined according to its characteristics, and then subsequent updates and other processing can be performed according to its category.

[0069] S12: After selecting one data category as a target category from multiple data categories included in the natural resource database using the feature data, create multiple monitoring thread tasks that can operate in parallel according to the uploaded number of the data to be processed.

[0070] In one embodiment, the natural resource database may be provided with multiple data categories, each of which may store data of one energy category, wherein the data categories may be preset by the user, and the number may also be adjusted according to actual needs.

[0071] For example, suppose the natural resource database stores data on five fish ponds in xx village in xx place. Each fish pond can correspond to 10 data items, for a total of 5*10=50 data items. To be precise with each data item, each data item can be used as a data category, and 50 data categories can be set. To manage each piece of fish pond data, the number of fish ponds can be used as a data category, and 5 data categories can be set. Specific operations can be adjusted according to actual needs.

[0072] In one implementation, it can be determined which of the multiple data categories contained in the natural resources database the data to be processed input by the user belongs to. After determining the target category to which the data to be processed belongs, the data to be processed can be subsequently compared with the data of that category to determine whether data updating or replacement is required.

[0073] In one embodiment, the data to be processed uploaded by the user may include multiple items, which may involve multiple different categories. If each piece of data is subsequently detected and processed one by one, not only will the processing take a long time, but the efficiency will also be low. In order to improve efficiency, multiple monitoring thread tasks that can be operated in parallel can be created in the system according to the number of uploaded data to be processed, and then each monitoring thread task detects and processes one piece of data to be processed, thereby realizing parallel processing of multiple data. By processing multiple pieces of data to be processed simultaneously, the processing time can be effectively shortened to improve processing efficiency.

[0074] In order to accurately determine the data category of the data to be processed, in an optional embodiment, the step of using the feature data to select one of the multiple data categories included in the natural resource database as a target category may include the following sub-steps:

[0075] S121: After converting the feature data into feature text, a preset word segmentation model is used to perform word segmentation processing on the feature text to obtain a plurality of feature word segments.

[0076] S122: Aggregate and classify the multiple feature segmentations using an aggregation classification algorithm to obtain a number of segmentation categories.

[0077] S123, inputting the characteristic segmentation corresponding to each segmentation category into a preset similarity calculation model to calculate the similarity between each segmentation category and each data category of the natural resource database, and obtaining a plurality of category similarities.

[0078] Specifically, the feature data may be converted into text using a preset text converter or text conversion model to obtain feature text, and then the feature text may be segmented using a preset word segmentation model to obtain a plurality of feature segmented words.

[0079] The multiple characteristic segmentations obtained after segmentation may have the same part of speech or different parts of speech. In order to integrate the various characteristic segmentations, the multiple characteristic segmentations are aggregated and classified by an aggregation classification algorithm to obtain several segmentation categories, each of which may contain one or more characteristic segmentations. In one implementation, the aggregation classification algorithm can be adjusted according to the actual needs of the user, and can be aggregated and classified according to the part of speech or the semantics of the word.

[0080] Next, the characteristic segmented words corresponding to each segmented word category may be input into a preset similarity calculation model, and the preset similarity calculation model may be used to calculate the similarity between each segmented word category and each data category in the natural resource database.

[0081] For example, there are 3 segmentation categories and the natural resource database has 5 data categories. The similarity between each segmentation category and the 5 data categories can be calculated, so that the 5 category similarities corresponding to each segmentation category can be input.

[0082] Reference Figure 2 , showing a structural diagram of a similarity calculation model provided by an embodiment of the present invention.

[0083] In one embodiment, the preset similarity calculation model includes: M network modules and a Softmax module connected in sequence;

[0084] Each network module includes a residual network unit and a score matching layer which are arranged in sequence.

[0085] The residual network unit includes a linear layer, a residual connection layer and a normalization layer arranged in sequence.

[0086] The scoring matching layer includes a first connection layer, a first activation layer, a second connection layer, a second activation layer, a first correction layer and a third activation layer which are connected in sequence.

[0087] As an example, the calculation operation of the category similarity may include the following sub-steps:

[0088] S1231. Perform matrix processing on the feature segmentations corresponding to the segmentation category to obtain a segmentation matrix, and input the segmentation matrix into the linear layer to calculate an attention score.

[0089] S1232. Use the residual connection layer to propagate the attention scores in forward and reverse directions to obtain a direction matrix, and use the normalization layer to integrate the direction matrix to obtain an integrated matrix.

[0090] S1233. Use the first connection layer, the first activation layer, the second connection layer, the second activation layer, the first correction layer and the third activation layer to perform dimensionality increase and decrease processing in sequence to obtain a processing matrix.

[0091] S1234. Utilize the Softmax function of the Softmax module to perform similarity calculation on the processing matrix to obtain category similarity.

[0092] Specifically, the segmentation category including one or more characteristic segmentations may be first subjected to matrix processing, so that each segmentation may be converted into a matrix, thereby obtaining a segmentation matrix.

[0093] If the word segmentation category contains a feature word segmentation, the feature word segmentation can be directly converted into a matrix. Optionally, the feature word segmentation can be mapped into a matrix, and the commonly used one-hot encoding and bag-of-words model can be used.

[0094] If the word segmentation category contains multiple characteristic word segmentations, the word frequency of each characteristic word segmentation in the characteristic text can be calculated separately, and an n*m word segmentation matrix can be constructed based on its word frequency value, where n is the word frequency value and m is the number of characteristic word segmentations.

[0095] Next, the word segmentation matrix can be input into the first network module, and the linear layer of the residual network unit of the first network module can convert the word segmentation matrix into a linear matrix of a corresponding format, and then calculate the attention score of the linear matrix. In one embodiment, the attention score can be shown as follows:

[0096] .

[0097] In the above formula, Q represents a linear matrix, T represents transpose; d k is a constant whose function is to reduce Q T The value of can push the subsequent Softmax function to the area with moderate gradient, and A is a constant.

[0098] Then the attention scores in the above steps can be input into the residual connection layer, which performs forward and reverse propagation to obtain the direction matrix. Then the direction matrix is ​​input into the normalization layer, which integrates it to obtain the integrated matrix.

[0099] In one embodiment, the formulas of the linear matrix, the direction matrix and the integration matrix are as follows:

[0100] ;

[0101] Among them, Q is the linear matrix, K is the direction matrix, and V is the integration matrix. For input data, is the weight of the linear layer, is the weight of the residual connection layer, is the weight of the normalization layer.

[0102] After the integration is completed, the integration matrix can be input into the score matching layer of the first network module, and the first connection layer of the score matching layer performs dimensionality increase processing on the integration matrix to obtain the increased dimensionality matrix; then the increased dimensionality matrix is ​​input into the first activation layer for sparse processing to obtain the first activation matrix; then the first activation matrix is ​​input into the second connection layer of the score matching layer for dimensionality reduction processing to obtain the reduced dimensionality matrix; then the reduced dimensionality matrix is ​​input into the second activation layer of the score matching layer for sparse processing to obtain the second activation matrix; then the second activation matrix is ​​input into the first correction layer of the score matching layer for correction to obtain the correction matrix; finally, the correction matrix is ​​input into the third activation layer of the score matching layer for sparse processing to obtain the third activation matrix.

[0103] The first forward and reverse propagation operation of the residual connection layer can not only retain the previous information during forward propagation, but also solve the gradient vanishing and gradient explosion during backward propagation. The first matrix integration operation (LayerNormalization) can integrate the matrix, making the model easier to converge.

[0104] Then, the third activation matrix can be input into the residual network unit of the next network module and processed by the linear layer of the residual network unit of the next network module, and so on, and finally the processing matrix is ​​output.

[0105] Finally, the processed matrix can be input into the Softmax module, and the Softmax function of the Softmax module is used for calculation, and the category similarity is output.

[0106] In one embodiment, the Softmax function can be expressed as follows:

[0107] .

[0108] Among them, A i is the processing matrix, C is the category similarity,

[0109] S124: Filter the corresponding data category as the target category according to the numerical value of the category similarity.

[0110] In one embodiment, after calculating the several category similarities corresponding to each segmentation category, the several category similarities corresponding to each segmentation category may be summed up, and then the segmentation category with the largest value may be selected from the sum.

[0111] Next, the category similarity with the largest value is selected from several category similarities corresponding to the word segmentation category with the largest value, and the data category corresponding to the category similarity with the largest value is taken as the target category.

[0112] In an optional embodiment, creating multiple monitoring thread tasks that can be operated in parallel according to the upload quantity of the to-be-processed data may include the following sub-steps:

[0113] S125. After creating the main thread, calculate the calculation time of the data to be processed, and determine the number of threads that the built-in processor can be divided into according to the calculation time.

[0114] S126. If the number of threads is greater than or equal to the number of uploaded data to be processed, a memory pool technology is used to divide the built-in processor memory into several storage partitions according to the number of uploaded data to be processed and store the data to be processed, and then a monitoring thread task responsible for executing the operation is created according to the number of uploaded data to be processed.

[0115] Specifically, a main thread may be created first, and the task unit computing time of the CPU or GPU may be calculated to obtain the computing duration. Then, the number of threads that can be divided may be determined according to the computing duration.

[0116] Then the number of uploaded data to be processed and the number of threads can be calculated; if the number of threads is greater than or equal to the number of uploaded data to be processed, it means that parallel processing can be performed at the same time, and thread tasks responsible for executing operations can be created according to the number of uploaded data to be processed. If the number of threads is less than the number of uploaded data to be processed, thread tasks responsible for executing operations are created according to the number of threads.

[0117] In order to facilitate the on-site extraction of data to be processed, memory pool technology can be used to divide specific partitions in the memory according to the number of thread tasks, and a certain scheduling strategy can be used to store the data to be processed in the memory pool, so that the CPU can obtain the data to be processed from the data storage area in the memory and perform subsequent operations.

[0118] Among them, the specific storage operation can first divide n memory areas in the memory pool. Each area stores a data to be processed. N is equal to the number of uploaded data to be processed. Then initialize n channels between the memory pools, each channel corresponds to a memory area, and when calling the data to be processed, the memory area can be accessed through the channel and the corresponding data to be processed can be extracted. In addition, in the memory pool, each channel interface will set a mark Pi, and each thread task will set a counter. When the data to be processed is added to the memory pool, the counter number will increase by 1 and wait for the handler to obtain it. Among them, Pi is marked as overwriteable, and when the data to be processed in the memory pool is deleted, the counter is set to 0. Before reading data from the memory, a hash table and information summary can be used to identify the object of each data to be processed, which specifically includes the following definitions: The information summary content of the thread task includes name, type, size and MD5 checksum. After splitting the large block of data, the block information of each small block of data is generated, and other information of the small block of data is written into the information summary.

[0119] When data is subsequently read from the memory, pipeline technology can be used to overlap the reading of data, preprocessing, and data processing in time, and let the CPU use multi-core multi-threading to execute multiple thread tasks simultaneously, and transmit the results back to the CPU for result aggregation.

[0120] S13, respectively calling each of the monitoring thread tasks to perform data change detection using the data to be processed and the historical data recorded in the target category to obtain a detection result, and performing data update processing using the data to be processed according to the detection result.

[0121] In one embodiment, after determining the target category of the data to be processed, previously recorded historical data may be extracted from the target category, and then the data to be processed may be compared with the historical data, and the comparison method may be adjusted according to the type or format of the data.

[0122] For example, if the data is a numerical value, the numerical value can be compared; if the data is an image, the image can be compared; if the data is text information, the text can be compared.

[0123] The test results obtained can be the same or different. If the test results are the same, the recording time of the previously recorded historical data can be updated to facilitate subsequent inquiries by users. If the test results are different, the data to be processed can be stored, or the data to be processed can be used to replace the historical data.

[0124] After recording or updating the data, in order to ensure data security, in one embodiment, the natural resource database monitoring method may further include the following steps:

[0125] S21. Obtain the timestamp of the data to be processed uploaded by the user, extract several numbers from the timestamp according to a preset rule, and concatenate them to generate an encrypted value.

[0126] S22: using the encrypted value to encrypt and save the data to be processed.

[0127] Specifically, an encryption key may be generated based on the time of data upload, the amount of information or text as features, and each stored or updated data to be processed may be encrypted using the encryption key, and then each encrypted data may be stored in the database.

[0128] Since each data to be processed has a corresponding timestamp when it is uploaded, in order to allow users to understand the time when the data was uploaded and the characteristics contained in the data, the specific encryption method includes the following process:

[0129] Get the timestamp when the user uploads the data to be processed, extract several numbers from the timestamp according to the preset rules and generate an encrypted value. (For example, the upload time is 202212021806, the rule is the first, third, fifth, sixth, eighth and tenth numbers, which are 2, 2, 1, 2, 2, 8 respectively, and add or multiply these numbers to get the encrypted value).

[0130] The encrypted value may be used to encrypt the data to be processed and then saved. Optionally, the encrypted value may be converted into a value in a preset format (eg, octal), and the converted value may be used as a key to encrypt the data.

[0131] Since the natural resource database stores a lot of data, in order to facilitate user viewing, in one embodiment, the natural resource database monitoring method may further include the following steps:

[0132] S31. Searching for a feature image related to the data to be processed according to the target category.

[0133] S32. After adjusting the chromaticity value or image contour of the feature image to form an adjusted image, the adjusted image and the data to be processed are used to construct a knowledge graph for user viewing.

[0134] Optionally, a knowledge graph can be created and each piece of data can be annotated in the knowledge graph to facilitate users or managers to view and call the data.

[0135] In order to add corresponding images to the knowledge graph for users to view, you can search for feature images of the data to be processed according to the target category. For example, if the object of the data to be processed is a fish pond, you can search for images of fish ponds; if the object of the data to be processed is a wasteland, you can search for images of wasteland.

[0136] Next, the chromaticity value or image contour of the feature image may be adjusted to form an adjusted image, and then the adjusted image and the data to be processed may be used to construct a knowledge graph for user viewing.

[0137] If the data changes, the adjustment method can adjust the chromaticity value or contour size of the image according to the size of the change.

[0138] For example, if the image to be adjusted is a fish pond, and the data to be processed is the area of ​​the fish pond, if the area changes, the horizontal contour of the fish pond can be adjusted, specifically, the horizontal contour can be widened or narrowed. For another example, if the image to be adjusted is a fish pond, and the data to be processed is the depth of the fish pond, if the depth changes, the chromaticity value of the fish pond can be adjusted, specifically, when the depth increases, the chromaticity value can be made darker, and vice versa. The specific operation method can be adjusted according to actual needs.

[0139] In this embodiment, an embodiment of the present invention provides a monitoring method for a natural resource database, and its beneficial effects are: when a monitoring user uploads data to be processed to the natural resource database, the present invention can determine the category of data in the natural resource database according to the characteristics of the data to be processed and create multiple monitoring thread tasks that can be operated in parallel according to the number of data to be processed, and process multiple tasks in parallel to detect and process multiple data to be processed and data of their corresponding categories, so as to update the data according to the detection results; by processing multiple data in parallel at the same time, not only can the update and monitoring scope of the database be broadened, but also the processing time of the data can be shortened, and the processing efficiency of the data can be improved to meet the existing monitoring and update needs.

[0140] The embodiment of the present invention also provides a monitoring device for a natural resource database, see Figure 3 , showing a schematic structural diagram of a natural resource database monitoring device provided by an embodiment of the present invention.

[0141] Wherein, as an example, the monitoring device of the natural resource database may include:

[0142] The data extraction module 301 is used to extract feature data from the data to be processed when the monitoring user uploads the data to be processed to the natural resource database;

[0143] A task creation module 302 is used to create a plurality of monitoring thread tasks that can be operated in parallel according to the upload quantity of the data to be processed after filtering the corresponding target category from the natural resource database using the feature data;

[0144] The monitoring update module 303 is used to call each of the monitoring thread tasks to perform data change detection using the data to be processed and the historical data recorded in the target category, obtain detection results, and perform data update processing using the data to be processed according to the detection results.

[0145] Optionally, the step of using the feature data to select one of the multiple data categories included in the natural resource database as a target category includes:

[0146] After converting the feature data into feature text, the feature text is segmented using a preset segmentation model to obtain a plurality of feature segmentations;

[0147] Aggregate and classify the multiple feature segmented words through an aggregation classification algorithm to obtain a number of segmented word categories;

[0148] Inputting the characteristic segmentation corresponding to each segmentation category into a preset similarity calculation model to calculate the similarity between each segmentation category and each data category of the natural resource database, and obtaining a plurality of category similarities;

[0149] According to the numerical value of the category similarity, the corresponding data category is selected as the target category.

[0150] Optionally, the preset similarity calculation model includes: M network modules and a Softmax module connected in sequence;

[0151] Each network module includes a residual network unit and a score matching layer arranged in sequence, the residual network unit includes a linear layer, a residual connection layer and a normalization layer arranged in sequence, and the score matching layer includes a first connection layer, a first activation layer, a second connection layer, a second activation layer, a first correction layer and a third activation layer connected in sequence.

[0152] Optionally, the calculation operation of the category similarity includes:

[0153] Performing matrix processing on the feature segmentations corresponding to the segmentation categories to obtain a segmentation matrix, and inputting the segmentation matrix into the linear layer to calculate an attention score;

[0154] Using the residual connection layer to propagate the attention score in forward and reverse directions to obtain a direction matrix, and using the normalization layer to integrate the direction matrix to obtain an integrated matrix;

[0155] Using the first connection layer, the first activation layer, the second connection layer, the second activation layer, the first correction layer and the third activation layer to perform dimension raising and lowering processing in sequence to obtain a processing matrix;

[0156] The Softmax function of the Softmax module is used to perform similarity calculation on the processing matrix to obtain category similarity.

[0157] Optionally, the device further comprises:

[0158] An image search module is used to search for a characteristic image of the data to be processed according to the target category;

[0159] A knowledge graph module is constructed to adjust the chromaticity value or image contour of the feature image to form an adjusted image, and then the adjusted image and the data to be processed are used to construct a knowledge graph for user viewing.

[0160] Optionally, the device further comprises:

[0161] A timestamp acquisition module is used to obtain the timestamp of the data to be processed uploaded by the user, extract several numbers from the timestamp according to a preset rule, and concatenate them to generate an encrypted value;

[0162] The encryption module is used to encrypt and save the data to be processed using the encryption value.

[0163] Optionally, the creating a plurality of monitoring thread tasks that can be operated in parallel according to the upload quantity of the data to be processed includes:

[0164] After creating the main thread, the calculation time of the data to be processed is calculated, and the number of threads that the built-in processor can be divided into is determined according to the calculation time;

[0165] If the number of threads is greater than or equal to the number of uploaded data to be processed, the memory pool technology is used to divide the built-in processor memory into several storage partitions according to the number of uploaded data to be processed and store the data to be processed, and then a monitoring thread task responsible for executing the operation is created according to the number of uploaded data to be processed.

[0166] Those skilled in the art can clearly understand that, for the sake of convenience and brevity of description, the specific working process of the device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0167] Furthermore, an embodiment of the present application also provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for monitoring the natural resource database as described in the above embodiment is implemented.

[0168] Furthermore, an embodiment of the present application also provides a computer-readable storage medium, which stores a computer-executable program, and the computer-executable program is used to enable a computer to execute the natural resource database monitoring method as described in the above embodiment.

[0169] Those skilled in the art will appreciate that the embodiments of the present application may also provide computer program products. Therefore, the present application may adopt the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the present application may adopt the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program codes.

[0170] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), apparatuses, and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0171] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0172] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0173] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A natural resource database monitoring method, characterized in that: The method comprises: When the monitoring user uploads the data to be processed to the natural resource database, extracting feature data from the data to be processed; After selecting the corresponding target category from the natural resource database using the characteristic data, a plurality of monitoring thread tasks capable of operating in parallel are created according to the uploaded quantity of the data to be processed; Calling each of the monitoring thread tasks to perform data change detection using the data to be processed and the historical data recorded in the target category to obtain a detection result, and performing data update processing using the data to be processed according to the detection result; Using the characteristic data, one of the multiple data categories included in the natural resource database is selected as a target category, including: After converting the feature data into feature text, the feature text is segmented using a preset segmentation model to obtain a plurality of feature segmentations; Aggregate and classify the multiple feature segmented words through an aggregation classification algorithm to obtain a number of segmented word categories; Inputting the characteristic segmentation corresponding to each segmentation category into a preset similarity calculation model to calculate the similarity between each segmentation category and each data category of the natural resource database, and obtaining a plurality of category similarities; According to the numerical value of the category similarity, the corresponding data category is selected as the target category; The preset similarity calculation model includes: M network modules and a Softmax module connected in sequence; Each network module includes a residual network unit and a score matching layer arranged in sequence, the residual network unit includes a linear layer, a residual connection layer and a normalization layer arranged in sequence, and the score matching layer includes a first connection layer, a first activation layer, a second connection layer, a second activation layer, a first correction layer and a third activation layer connected in sequence; The calculation operation of the category similarity includes: Performing matrix processing on the feature segmentations corresponding to the segmentation categories to obtain a segmentation matrix, and inputting the segmentation matrix into the linear layer to calculate an attention score; Using the residual connection layer to propagate the attention score in forward and reverse directions to obtain a direction matrix, and using the normalization layer to integrate the direction matrix to obtain an integrated matrix; Using the first connection layer, the first activation layer, the second connection layer, the second activation layer, the first correction layer and the third activation layer to perform dimension raising and lowering processing in sequence to obtain a processing matrix; Using the Softmax function of the Softmax module to perform similarity calculation on the processing matrix to obtain category similarity; The attention score is as follows: ; In the above formula, Q represents a linear matrix, T represents transpose; d k is a constant; A is a constant; The formulas of the linear matrix, the direction matrix and the integration matrix are respectively shown as follows: ; Among them, Q is the linear matrix, K is the direction matrix, and V is the integration matrix. For input data, is the weight of the linear layer, is the weight of the residual connection layer, is the weight of the normalization layer; Among them, after the integration is completed, the integration matrix is ​​input into the score matching layer of the first network module, and the first connection layer of the score matching layer performs dimension-upgrading processing on the integration matrix to obtain the dimension-upgrading matrix; the dimension-upgrading matrix is ​​input into the first activation layer for sparse processing to obtain the first activation matrix; the first activation matrix is ​​input into the second connection layer of the score matching layer for dimension-reduction processing to obtain the dimension-reduced matrix; the dimension-reduced matrix is ​​input into the second activation layer of the score matching layer for sparse processing to obtain the second activation matrix; the second activation matrix is ​​input into the first correction layer of the score matching layer for correction to obtain the correction matrix; the correction matrix is ​​input into the third activation layer of the score matching layer for sparse processing to obtain the third activation matrix; The third activation matrix is ​​input to the residual network unit of the next network module, processed by the linear layer of the residual network unit of the next network module, and output to obtain a processing matrix; The step of creating a plurality of monitoring thread tasks that can be operated in parallel according to the upload quantity of the data to be processed includes: After creating the main thread, the calculation time of the data to be processed is calculated, and the number of threads that the built-in processor can be divided into is determined according to the calculation time; If the number of threads is greater than or equal to the number of uploaded data to be processed, the memory pool technology is used to divide the built-in processor memory into several storage partitions according to the number of uploaded data to be processed and store the data to be processed, and then a monitoring thread task responsible for executing the operation is created according to the number of uploaded data to be processed.

2. The natural resource database monitoring method according to claim 1, characterized in that: After the step of performing data update processing using the data to be processed according to the detection result, the method further includes: Searching for a feature image of the data to be processed according to the target category; After adjusting the chromaticity value or image contour of the feature image to form an adjusted image, the adjusted image and the data to be processed are used to construct a knowledge graph for user viewing.

3. The natural resource database monitoring method according to claim 1, characterized in that: After the step of performing data update processing using the data to be processed according to the detection result, the method further includes: Obtain the timestamp of the data to be processed uploaded by the user, extract several numbers from the timestamp according to the preset rules and concatenate them to generate an encrypted value; The encrypted value is used to encrypt and save the data to be processed.

4. A monitoring device for a natural resource database, characterized in that: The device comprises: A data extraction module, used for extracting characteristic data from the data to be processed when the monitoring user uploads the data to be processed to the natural resource database; Creating a task module, which is used to create a plurality of monitoring thread tasks that can be operated in parallel according to the uploaded quantity of the data to be processed after filtering the corresponding target category from the natural resource database using the feature data; A monitoring update module, used to call each of the monitoring thread tasks to perform data change detection using the data to be processed and the historical data recorded in the target category, obtain a detection result, and perform data update processing using the data to be processed according to the detection result; The step of using the characteristic data to select one of the multiple data categories included in the natural resource database as a target category includes: After converting the feature data into feature text, the feature text is segmented using a preset segmentation model to obtain a plurality of feature segmentations; Aggregate and classify the multiple feature segmented words through an aggregation classification algorithm to obtain a number of segmented word categories; Inputting the characteristic segmentation corresponding to each segmentation category into a preset similarity calculation model to calculate the similarity between each segmentation category and each data category of the natural resource database, and obtaining a plurality of category similarities; According to the numerical value of the category similarity, the corresponding data category is selected as the target category; The preset similarity calculation model includes: M network modules and a Softmax module connected in sequence; Each network module includes a residual network unit and a score matching layer arranged in sequence, the residual network unit includes a linear layer, a residual connection layer and a normalization layer arranged in sequence, and the score matching layer includes a first connection layer, a first activation layer, a second connection layer, a second activation layer, a first correction layer and a third activation layer connected in sequence; The calculation operation of the category similarity includes: Performing matrix processing on the feature segmentations corresponding to the segmentation categories to obtain a segmentation matrix, and inputting the segmentation matrix into the linear layer to calculate an attention score; Using the residual connection layer to propagate the attention score in forward and reverse directions to obtain a direction matrix, and using the normalization layer to integrate the direction matrix to obtain an integrated matrix; Using the first connection layer, the first activation layer, the second connection layer, the second activation layer, the first correction layer and the third activation layer to perform dimension raising and lowering processing in sequence to obtain a processing matrix; Using the Softmax function of the Softmax module to perform similarity calculation on the processing matrix to obtain category similarity; The attention score is as follows: ; In the above formula, Q represents a linear matrix, T represents transpose; d k is a constant; A is a constant; The formulas of the linear matrix, the direction matrix and the integration matrix are respectively shown as follows: ; Among them, Q is the linear matrix, K is the direction matrix, and V is the integration matrix. For input data, is the weight of the linear layer, is the weight of the residual connection layer, is the weight of the normalization layer; Among them, after the integration is completed, the integration matrix is ​​input into the score matching layer of the first network module, and the first connection layer of the score matching layer performs dimension-upgrading processing on the integration matrix to obtain the dimension-upgrading matrix; the dimension-upgrading matrix is ​​input into the first activation layer for sparse processing to obtain the first activation matrix; the first activation matrix is ​​input into the second connection layer of the score matching layer for dimension-reduction processing to obtain the dimension-reduced matrix; the dimension-reduced matrix is ​​input into the second activation layer of the score matching layer for sparse processing to obtain the second activation matrix; the second activation matrix is ​​input into the first correction layer of the score matching layer for correction to obtain the correction matrix; the correction matrix is ​​input into the third activation layer of the score matching layer for sparse processing to obtain the third activation matrix; The third activation matrix is ​​input to the residual network unit of the next network module, processed by the linear layer of the residual network unit of the next network module, and output to obtain a processing matrix; The step of creating a plurality of monitoring thread tasks that can be operated in parallel according to the upload quantity of the data to be processed includes: After creating the main thread, the calculation time of the data to be processed is calculated, and the number of threads that the built-in processor can be divided into is determined according to the calculation time; If the number of threads is greater than or equal to the number of uploaded data to be processed, the memory pool technology is used to divide the built-in processor memory into several storage partitions according to the number of uploaded data to be processed and store the data to be processed, and then a monitoring thread task responsible for executing the operation is created according to the number of uploaded data to be processed.

5. An electronic device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the natural resource database monitoring method as described in any one of claims 1 to 3 when executing the computer program.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer-executable program, and the computer-executable program is used to enable a computer to execute the natural resource database monitoring method as described in any one of claims 1-3.