A method and system for transmitting shipping manifest data

Abstract

The present invention relates to the technical field of shipping manifest data transmission, and in particular to a method and system for shipping manifest data transmission, including: obtaining shipping manifest data of a shipping company for a period of time; classifying information data in all shipping manifest data to obtain a number of project clusters and a number of project cluster sets; updating information data to obtain updated information data; obtaining repetition weights and stability of updated information data in project clusters; obtaining a number of to-be-compressed project clusters and a number of non-to-be-compressed project clusters according to the repetition weights and stability of updated information data; encoding the updated information data in the to-be-compressed project clusters and the to-be-compressed project clusters to obtain encoded data; transmitting the encoded data and the updated information data in all non-to-be-compressed project clusters. The present invention improves the speed of shipping manifest data transmission.

Description

Technical Field

[0001] The present invention relates to the technical field of shipping manifest data transmission, and in particular to a shipping manifest data transmission method and system. Background Art

[0002] With the rapid development of information technology, the shipping industry has attached more and more importance to data transmission and information management. As an important part of this, shipping manifest data transmission has also received more and more attention.

[0003] When the existing shipping manifest data is transmitted, since the shipping manifest data contains a variety of information data, different information data have different repetition situations, and different information data correspond to different stability situations, if all the information data are directly transmitted without considering the repetition and stability of the information data, the speed of the shipping manifest data transmission will be slow. Summary of the invention

[0004] In order to solve the above problems, the present invention provides a method and system for transmitting shipping manifest data.

[0005] A method and system for transmitting shipping manifest data of the present invention adopts the following technical solutions:

[0006] An embodiment of the present invention provides a method for transmitting shipping manifest data, the method comprising the following steps:

[0007] Obtain a number of shipping manifest data from a shipping company over a preset period of time;

[0008] Classify the information data in all shipping manifest data according to the project name to obtain several project clusters; obtain several project cluster sets according to the synchronous changes of the information data in different project clusters under the preset sorting rules and the number of different information data in the project clusters; update the information data according to the position differences of the corresponding characters of different information data in the same project cluster and the proportion of repeated characters in the information data to obtain updated information data;

[0009] According to the proportion of repeated occurrences of updated information data in the project cluster, the repetition weight of each updated information data in each project cluster is obtained; according to the number of changes in the updated information data in the project cluster within the project cluster set, the frequency and fluctuation of the updated information data in all shipping manifest data, the stability of each updated information data in each project cluster is obtained;

[0010] According to the number of different update information data in the project cluster, the repetition weight and stability of the update information data, several project clusters to be compressed and several project clusters not to be compressed are obtained; the update information data in the project clusters to be compressed and the project clusters to be compressed are encoded to obtain encoded data; the encoded data and the update information data in all the project clusters not to be compressed are transmitted.

[0011] Furthermore, the steps of obtaining a plurality of project cluster sets according to the synchronous changes of information data in different project clusters under the preset sorting rules and the quantity of different information data in the project clusters include the following:

[0012] For any two project clusters, all information data in the first project cluster are sorted from most to least according to the number of characters, and the number of times the information data in the first project cluster changes and the number of times the information data in the second project cluster changes are obtained, and the number of different information data in the first project cluster and the number of different information data in the second project cluster are obtained, and the ratio of the number of times the information data in the first project cluster changes to the number of times the information data in the second project cluster changes is recorded as a first ratio, and the absolute value of the difference between the first ratio and 1 is used as an asynchronous change index of the first project cluster and the second project cluster, and the product of the number of different information data in the first project cluster and the number of different information data in the second project cluster is recorded as a first product, and the asynchronous change index and the first product are fused to obtain the correlation between the first project cluster and the second project cluster, the asynchronous change index and the first product are inversely proportional to the correlation, and the correlation is a normalized value; according to the size of the correlation, a number of project cluster sets are obtained.

[0013] Furthermore, the steps of obtaining a plurality of project cluster sets according to the magnitude of the correlation degree include the following:

[0014] The correlation between every two project clusters is obtained, two project clusters whose correlation is greater than or equal to a preset first threshold are divided into a project cluster set, and the correlation between all project clusters is judged in turn to obtain several project cluster sets.

[0015] Furthermore, the information data is updated according to the position difference of the characters corresponding to different information data in the same project cluster and the proportion of repeated characters in the information data to obtain the updated information data, which includes the following specific steps:

[0016] For two information data of any project cluster, the number of occurrences of all characters in the first information data in the second information data is obtained, recorded as the first number, and the ratio of the first number to the total number of characters in the first information data is used as the ratio of repeated occurrences of characters in the first information data; the characters in the first information data and the second information data are numbered consecutively starting from 1 in order, the absolute value of the difference between the number value of each character in the first information data and the number value of the corresponding same character in the second information data is recorded as a number value difference, and the mean of all number value differences corresponding to the first information data and the second information data is used as the position difference of the characters in the first information data and the second information data; the ratio of the first number to the total number of characters in the first information data and the mean of the number value differences are merged to obtain the correlation between the first information data and the second information data, the ratio of the first number to the total number of characters in the first information data is in direct proportion to the correlation, the mean of the number value differences is inversely proportional to the correlation, and the correlation is a normalized value; the information data is updated according to the size of the correlation to obtain updated information data.

[0017] Furthermore, the updating of the information data according to the magnitude of the correlation to obtain the updated information data includes the following specific steps:

[0018] A second threshold is preset, and two information data corresponding to a correlation greater than the second threshold are regarded as a pair of similar information data. A pair of similar information data are both updated to the information data with the largest number of characters, and vice versa, to their original information data, thereby obtaining a number of updated information data.

[0019] Furthermore, the repetition weight of each update information data in each project cluster is obtained according to the proportion of repeated occurrences of update information data in the project cluster, and the specific steps include the following:

[0020] For any updated information data in any project cluster, obtain the number of repeated occurrences of the updated information data in the project cluster, obtain the number of updated information data in the project cluster, and record the ratio of the number of repeated occurrences of the updated information data in the project cluster to the number of updated information data in the project cluster as the first proportion of repeated occurrences of the updated information data; obtain the maximum number of repeated occurrences of the updated information data in the project cluster, and record the ratio of the number of repeated occurrences of the updated information data in the project cluster to the maximum number of repeated occurrences of the updated information data in the project cluster as the second proportion of repeated occurrences of the updated information data; merge the first proportion and the second proportion to obtain the repetition weight of the updated information data in the project cluster, and the first proportion and the second proportion are directly proportional to the repetition weight.

[0021] Furthermore, the method of obtaining the stability of each updated information data in each item cluster according to the number of times the updated information data in the item clusters in the item cluster set changes, the frequency and fluctuation of the updated information data in all shipping manifest data, includes the following specific steps:

[0022] For any updated information data in any project cluster; obtain the stability factor of the updated information data in the project cluster, and the specific method for obtaining the stability factor is: record the project cluster set to which the project cluster belongs as the first project cluster set; obtain all the updated information data corresponding to the updated information data in other project clusters in the first project cluster set as the associated updated information data set corresponding to the updated information data in the project cluster, recorded as the first associated updated information data set; obtain the associated updated information data set corresponding to each updated information data in the project cluster, and record the associated updated information data sets that are the same as the first associated updated information data set as the second associated updated information data set; obtain the updated information data corresponding to all the second associated updated information data sets and obtain the number of different updated information data, recorded as the number of times the updated information data in the project cluster changes, and use the value after the number of times the updated information data in the project cluster changes through the inverse proportional function mapping as the stability factor of the updated information data in the project cluster;

[0023] Obtain the time interval from the first appearance to the last appearance of the update information data in all shipping manifest data; obtain the variance of the time intervals of all appearances of the update information data in all shipping manifest data; obtain the mean of the time intervals of all appearances of the update information data in all shipping manifest data; merge the stability factor, the time interval from the first appearance to the last appearance, the variance of the time interval, and the mean of the time interval to obtain the stability of the update information data in the project cluster; the stability factor and the time interval from the first appearance to the last appearance are in direct proportion to the stability, and the variance of the time interval and the mean of the time interval are in inverse proportion to the stability.

[0024] Furthermore, the method of obtaining a plurality of project clusters to be compressed and a plurality of project clusters not to be compressed according to the number of different update information data in the project cluster, the repetition weight and the stability of the update information data includes the following specific steps:

[0025] For any project cluster; obtain the number of different update information data in the project cluster; multiply the repetition weight and stability of each update information data in the project cluster to obtain the first product of each update information data in the project cluster; obtain the cumulative value of the first products of all update information data in the project cluster; merge the number of different update information data in the project cluster and the cumulative value of the first product to obtain the compression necessity of the project cluster, the number of different update information data in the project cluster is inversely proportional to the compression necessity, the cumulative value of the first product is directly proportional to the compression necessity, and the compression necessity is a normalized value; preset a third threshold value, and record the project clusters corresponding to the compression necessity greater than the third threshold value as project clusters to be compressed, and record the project clusters corresponding to the compression necessity less than or equal to the third threshold value as project clusters not to be compressed.

[0026] Furthermore, the encoding of the to-be-compressed item cluster and the update information data in the to-be-compressed item cluster to obtain the encoded data includes the following specific steps:

[0027] All the clusters of items to be compressed are sorted from large to small according to the corresponding compression necessity, and the encoding is started from the first cluster of items to be compressed. The corresponding code of the first cluster of items to be compressed is A1, and the corresponding code of the second cluster of items to be compressed is A2. The encoding is continued in sequence until all the clusters of items to be compressed have corresponding codes; starting from the first cluster of items to be compressed, all the update information data in the first cluster of items to be compressed are sorted from large to small according to the number of occurrences, and the encoding is started from the first update information data. The first update information data is coded as B11, and the second update information data is coded as B2. The code is B12, and it is encoded in sequence until all the updated information data in the first item cluster to be compressed have corresponding codes; according to the encoding method of the updated information data in the first item cluster to be compressed, all the updated information data in the second item cluster to be compressed are encoded, the first updated information data in the second item cluster to be compressed is encoded as B21, the second updated information data in the second item cluster to be compressed is encoded as B22, until all the updated information data in the second item cluster to be compressed have corresponding codes, and the updated information data in all the item clusters to be compressed are encoded to obtain encoded data.

[0028] The present invention also proposes a shipping manifest data transmission system, comprising a memory and a processor, wherein the processor executes a computer program stored in the memory to implement the steps of the aforementioned method.

[0029] The beneficial effect of the technical solution of the present invention is as follows: after obtaining a number of shipping manifest data of a shipping company for a preset period of time, the present invention obtains a number of project clusters and a number of project cluster sets by classification, updates the information data by analyzing the position difference of the corresponding characters of different information data in the same project cluster and the proportion of repeated characters in the information data, obtains updated information data, makes the repetition rate of similar information data higher during subsequent transmission, improves the transmission speed, obtains the repetition weight of each updated information data in each project cluster by analyzing the proportion of repeated updated information data in the project cluster, and obtains the repetition weight of each updated information data in each project cluster by analyzing the proportion of repeated updated information data in the project cluster set. The number of changes, the frequency and fluctuation of the updated information data in all shipping manifest data are used to obtain the stability of each updated information data in each item cluster, and then combined with the number, repetition weight and stability of different updated information data in the item cluster, several item clusters to be compressed and several item clusters not to be compressed are obtained. The repetition rate of the updated information data in the item clusters to be compressed is high and relatively stable. Finally, the updated information data in the item clusters to be compressed and the item clusters to be compressed are encoded to obtain the encoded data; the encoded data and the updated information data in all item clusters not to be compressed are transmitted, which improves the speed of shipping manifest data transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0031] Figure 1 A flowchart of a method for transmitting shipping manifest data provided by an embodiment of the present invention;

[0032] Figure 2 A flowchart of transmitting coded data and update information data in all non-to-be-compressed item clusters provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0033] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of a method and system for transmitting shipping manifest data according to the present invention, its specific implementation, structure, features and effects, in conjunction with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0034] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0035] The following describes in detail a method and system for transmitting shipping manifest data provided by the present invention in conjunction with the accompanying drawings.

[0036] See also Figure 1 and Figure 2 , which shows a flowchart of a method for transmitting shipping manifest data according to an embodiment of the present invention and a flowchart of transmitting coded data and updated information data in all non-compressed item clusters, the method comprising the following steps:

[0037] Step S001, obtaining a number of shipping manifest data of a shipping company within a preset period of time.

[0038] It should be noted that this embodiment mainly analyzes the ocean manifest data transmission. Before starting the analysis, it is necessary to first obtain some ocean manifest data of the shipping company.

[0039] Specifically, the shipping manifest data of the shipping company in a preset period of time is obtained. It should be noted that obtaining the shipping manifest data of the shipping company in a preset period of time is an existing method, which will not be described in detail in this embodiment; the shipping manifest data includes the shipping manifests filled out by the shipping company in the past year and the relevant information provided by the corresponding shipper, so as to obtain various information on the shipping manifest.

[0040] At this point, some shipping manifest data of the shipping company over a preset period of time has been obtained.

[0041] Step S002: classify the information data in all the shipping manifest data according to the project name to obtain a number of project clusters; obtain a number of project cluster sets according to the synchronous changes of the information data in different project clusters under the preset sorting rules and the number of different information data in the project clusters; update the information data according to the position difference of the corresponding characters of different information data in the same project cluster and the proportion of repeated characters in the information data to obtain updated information data.

[0042] It should be noted that for the transmission of ocean manifest data, the numerous manifest information must first be preliminarily sorted and classified to improve the repeatability of the data for better subsequent transmission.

[0043] Specifically, the information data in all the shipping manifest data are classified according to the item names to obtain several item clusters. It should be noted that the item names of the shipping manifest data include various other item names such as the consignor's name and the ship's name. For example, if the item name is the consignor's name, the information data about the consignor's name in all the shipping manifest data are counted to obtain an item cluster. Similarly, several item clusters can be obtained.

[0044] It should be noted that the above preliminary sorting and classification of the information data in all shipping manifest data according to the project name is due to the different correlations between different project clusters. If they are not distinguished, the subsequent transmission effect will be poor. Therefore, it is necessary to analyze the correlation between project clusters. Since project clusters with strong correlations will change with the changes in information data within a project cluster, and the smaller the number of corresponding different information data in another project cluster, the stronger the correlation is. Therefore, by analyzing the number of changes in information data in another project cluster as the information data in one project cluster changes and the number of corresponding different information data in another project cluster, the correlation between project clusters is obtained, and then according to the size of the correlation, a number of project cluster sets are screened.

[0045] Specifically, according to the synchronous changes of information data in different project clusters under the preset sorting rules and the quantity of different information data in the project clusters, a plurality of project cluster sets are obtained, including the following steps:

[0046] Firstly, according to the synchronous changes of information data in different project clusters under the preset sorting rules and the number of different information data in the project clusters, the correlation between the project clusters is obtained.

[0047] Secondly, according to the size of the correlation, several sets of project clusters are obtained.

[0048] Preferably, the correlation between project clusters is obtained according to the synchronous changes of information data in different project clusters under the preset sorting rules and the quantity of different information data in the project clusters, as follows:

[0049] For any two project clusters, all information data in the first project cluster are sorted from most to least according to the number of characters, and the number of times the information data in the first project cluster changes and the number of times the information data in the second project cluster changes are obtained, and the number of different information data in the first project cluster and the number of different information data in the second project cluster are obtained. The ratio of the number of times the information data in the first project cluster changes to the number of times the information data in the second project cluster changes is recorded as a first ratio, and the absolute value of the difference between the first ratio and 1 is used as an asynchronous change index of the first project cluster and the second project cluster. The product of the number of different information data in the first project cluster and the number of different information data in the second project cluster is recorded as a first product. The asynchronous change index and the first product are fused to obtain the correlation between the first project cluster and the second project cluster. The asynchronous change index and the first product are inversely proportional to the correlation, and the correlation is a normalized value.

[0050] As a specific example, the specific method for obtaining the correlation degree is as follows:

[0051]

[0052] Where n γ is the number of times the information data in the γth project cluster changes after all the information data in the γth project cluster are sorted from most to least by the number of characters; it should be noted that the original order of all the information data in the project cluster is arranged in the order of acquisition. After sorting from most to least by the number of characters, the information data will change, for example, the information data with more characters will be placed in front, that is, a change will occur; n γ,β is the number of times the information data in the βth project cluster changes after all the information data in the γth project cluster are sorted from most to least in terms of the number of characters; || is the absolute value; k γ is the number of different information data in the γth project cluster; k β is the number of different information data in the βth project cluster; exp[] is an exponential function with a natural constant as the base. This embodiment adopts the exp[-U] model to present the inverse proportional relationship and normalization processing. U is the input of the model. Other inverse proportional functions and normalization functions can be set in specific implementation; G γ,β is the correlation between the γth item cluster and the βth item cluster.

[0053] It should be noted that The smaller it is, the more the number of changes in the information data in another project cluster is the same as the change in the information data in one project cluster. γ,β The larger the k γ ×kβ The smaller the value, the smaller the number of corresponding different information data in another project cluster is, and the stronger the correlation is. γ,β The bigger.

[0054] Furthermore, according to the size of the correlation, several project cluster sets are obtained, as follows:

[0055] The correlation between every two project clusters is obtained, and two project clusters whose correlation is greater than or equal to a preset first threshold are divided into a project cluster set. The correlation between all project clusters is judged in turn to obtain several project cluster sets. This embodiment is described with the first threshold being 0.6.

[0056] It should be noted that the purpose of obtaining the project cluster set is to facilitate the subsequent analysis of the stability factor of the information data in the project cluster, so as to better transmit the information data and improve the transmission efficiency.

[0057] It should be noted that, since different information data in the project cluster also have correlations, if the correlations between different information data are not distinguished and directly transmitted, the transmission efficiency will be low. Therefore, it is necessary to analyze the correlations between different information data in the same project cluster; since the correlations between different information data are mainly related to the repeated appearance of characters in the information data and the position differences of corresponding characters, the more repeated characters in the information data, the more correlated the different information data are. If the position differences of corresponding characters are small at the same time, it also means that the different information data are more correlated. Therefore, by analyzing the repeated appearance of characters in different information data and the position differences of corresponding characters, the correlations between different information data are obtained, and then the information data is replaced and updated according to the size of the correlation.

[0058] Specifically, according to the position difference of the characters corresponding to different information data in the same project cluster and the proportion of repeated characters in the information data, the information data is updated to obtain the updated information data, including the steps of:

[0059] Firstly, according to the position difference of the corresponding characters of different information data in the same project cluster and the proportion of repeated characters in the information data, the correlation between the information data in the same project cluster is obtained.

[0060] Secondly, the information data is updated according to the size of the correlation to obtain updated information data.

[0061] Preferably, the correlation between the information data in the same project cluster is obtained according to the position difference of the corresponding characters of different information data in the same project cluster and the proportion of repeated characters in the information data, as follows:

[0062] For two information data of any project cluster, the number of occurrences of all characters in the first information data in the second information data is obtained, recorded as the first number, and the ratio of the first number to the total number of characters in the first information data is used as the proportion of repeated occurrences of characters in the first information data; the characters in the first information data and the second information data are numbered consecutively starting from 1 in order, and the absolute value of the difference between the number value of each character in the first information data and the number value of the corresponding same character in the second information data is recorded as a number value difference, and the mean of all number value differences corresponding to the first information data and the second information data is used as the position difference of the characters in the first information data and the second information data; the ratio of the first number to the total number of characters in the first information data and the mean of the number value differences are merged to obtain the correlation between the first information data and the second information data, the ratio of the first number to the total number of characters in the first information data is directly proportional to the correlation, the mean of the number value differences is inversely proportional to the correlation, and the correlation is a normalized value.

[0063] As a specific example, the specific method for obtaining the association is as follows:

[0064] For the bth information data and cth information data of any project cluster.

[0065]

[0066] In the formula, l b,c is the number of characters in the bth information data that appear in the cth information data; N b is the number of characters in the bth information data; sx b,c The specific method for obtaining is as follows: the characters in the b-th information data are numbered continuously starting from 1 in sequence, and the characters in the c-th information data are numbered continuously starting from 1 in sequence, and the number value of each character in the b-th information data corresponding to the same character in the c-th information data is obtained, and the absolute value of the difference between the number value of each character in the b-th information data and the number value of the same character in the c-th information data is recorded as a number value difference, and the average of all number value differences corresponding to the b-th information data and the c-th information data is recorded as sx b,c ; It should be noted that if there is no corresponding identical character in the cth information data, the number value of the character in the bth information data corresponding to the identical character in the cth information data is 0; exp() is an exponential function with a natural constant as the base; tanh[] is a hyperbolic tangent function used for normalization; S b,c is the correlation between the bth information data and the cth information data.

[0067] It should be noted that The larger the value, the more characters in the bth information data appear in the cth information data. b,c The bigger; sx b,c The smaller it is, the relative order of the characters in the bth information data and the characters in the cth information data remains unchanged. b,c The bigger.

[0068] Furthermore, the information data is updated according to the magnitude of the correlation to obtain updated information data, as follows:

[0069] A second threshold is preset. This embodiment is described with the second threshold being 0.5. Two information data corresponding to a correlation greater than the second threshold are regarded as a pair of similar information data. A pair of similar information data are both updated to the information data with the largest number of characters, and vice versa, they are updated to their original information data, thereby obtaining a number of updated information data.

[0070] It should be noted that the updated information data is more unified, which facilitates better subsequent transmission.

[0071] At this point, the update information data is obtained.

[0072] Step S003: obtain the repetition weight of each updated information data in each project cluster according to the proportion of repeated occurrences of updated information data in the project cluster; obtain the stability of each updated information data in each project cluster according to the number of times the updated information data in the project clusters within the project cluster set changes, the frequency and fluctuation of the updated information data in all shipping manifest data.

[0073] It should be noted that the above-mentioned update information data is updated. Since the repetition situations of each update information data in the project cluster are different, if the repetition situations of the update information data are not distinguished and directly transmitted, it will cause several identical update information data to be transmitted multiple times, resulting in low transmission efficiency. Therefore, it is necessary to analyze the repetition situation of each update information data in each project cluster. Since the repetition situation of the update information data is related to the proportion of the update information data repeated in the project cluster, the larger the proportion, the more repetitions. Therefore, by analyzing the proportion of the update information data repeated in the project cluster, the repetition weight of each update information data in each project cluster is obtained.

[0074] Preferably, the repetition weight of each update information data in each project cluster is obtained according to the proportion of repeated occurrences of update information data in the project cluster, which is specifically as follows:

[0075] For any updated information data in any project cluster, obtain the number of repeated occurrences of the updated information data in the project cluster, obtain the number of updated information data in the project cluster, and record the ratio of the number of repeated occurrences of the updated information data in the project cluster to the number of updated information data in the project cluster as the first proportion of repeated occurrences of the updated information data; obtain the maximum number of repeated occurrences of the updated information data in the project cluster, and record the ratio of the number of repeated occurrences of the updated information data in the project cluster to the maximum number of repeated occurrences of the updated information data in the project cluster as the second proportion of repeated occurrences of the updated information data; merge the first proportion and the second proportion to obtain the repetition weight of the updated information data in the project cluster, and the first proportion and the second proportion are directly proportional to the repetition weight.

[0076] As a specific example, the specific method for obtaining the repetition weight is as follows:

[0077]

[0078] In the formula, g α,i n1 is the number of repetitions of the i-th update information data in the α-th project cluster; α is the number of updated information data in the αth project cluster; g α,max is the maximum number of repetitions of the updated information data in the αth item cluster; Q α,i is the repetition weight of the i-th updated information data in the α-th item cluster.

[0079] It should be noted that The larger the value is, the more updated information data in the a-th project cluster accounts for. α,i The bigger; The larger the value, the larger the proportion of updated information data in the a-th project cluster is. α,i The bigger.

[0080] It should be noted that the above analysis analyzes the repetition weight of each updated information data in each project cluster. In order to facilitate better subsequent processing and transmission, it is also necessary to analyze the stability of each updated information data in each project cluster to reduce the impact of the instability of the updated information data in the project cluster on the subsequent transmission; since the stability of the updated information data in the project cluster is related to the number of times the updated information data changes and the frequency and fluctuation of the updated information data in all shipping manifest data, the fewer the number of times the updated information data changes, the more stable the updated information data is, the more frequently the updated information data appears in all shipping manifest data, the more stable it is, and the smaller the fluctuation of the updated information data in all shipping manifest data is, it also means that the updated information data is more stable. The factor obtains the stability of the updated information data by analyzing the number of times the updated information data in the project cluster changes and the frequency and fluctuation of the updated information data in all shipping manifest data.

[0081] Preferably, the stability of each updated information data in each item cluster is obtained according to the number of times the updated information data in the item cluster within the item cluster set changes, the frequency and fluctuation of the updated information data in all shipping manifest data, as follows:

[0082] For any updated information data in any project cluster; obtain the stability factor of the updated information data in the project cluster, and the specific method for obtaining the stability factor is: record the project cluster set to which the project cluster belongs as the first project cluster set; obtain all the updated information data corresponding to the updated information data in other project clusters in the first project cluster set, as the associated updated information data set corresponding to the updated information data in the project cluster, recorded as the first associated updated information data set; it should be noted that all the updated information data corresponding to the updated information data in other project clusters in the first project cluster set are obtained, wherein all the updated information data are associated with the updated information data. The information data belongs to the same shipping manifest data, and the correspondence here describes the same shipping manifest data; obtain the associated update information data set corresponding to each update information data in the project cluster, and record the associated update information data set that is the same as the first associated update information data set as the second associated update information data set; obtain the update information data corresponding to all the second associated update information data sets and obtain the different numbers of update information data, which are recorded as the number of times the update information data in the project cluster has changed, and the value after the number of times the update information data in the project cluster has changed is mapped by an inverse proportional function as the stability factor of the update information data in the project cluster.

[0083] Obtain the time interval from the first appearance to the last appearance of the update information data in all shipping manifest data; obtain the variance of the time intervals of all appearances of the update information data in all shipping manifest data; obtain the mean of the time intervals of all appearances of the update information data in all shipping manifest data; merge the stability factor, the time interval from the first appearance to the last appearance, the variance of the time interval, and the mean of the time interval to obtain the stability of the update information data in the project cluster; the stability factor and the time interval from the first appearance to the last appearance are in direct proportion to the stability, and the variance of the time interval and the mean of the time interval are in inverse proportion to the stability.

[0084] As a specific example, the specific method for obtaining the stability is as follows:

[0085]

[0086] In the formula, X a,i is the stability factor of the i-th updated information data in the a-th project cluster; the specific method for obtaining the stability factor is as follows: α,i =exp(-x α,i ), exp() is an exponential function with a natural constant as the base; x α,i The specific method of obtaining is as follows:

[0087] The project cluster set to which the αth project cluster belongs is recorded as the first project cluster set;

[0088] Obtain the i-th update information data in the α-th project cluster, and all corresponding update information data in other project clusters in the first project cluster set as the associated update information data set corresponding to the i-th update information data in the α-th project cluster; it should be noted that there is a corresponding relationship between the update information data in different project clusters;

[0089] Record the associated update information data set corresponding to the i-th update information data as the first associated update information data set;

[0090] Obtain the associated update information data set corresponding to each update information data in the αth project cluster, and record the associated update information data set that is the same as the first associated update information data set as the second associated update information data set;

[0091] Get all the update information data corresponding to the second associated update information data set and get the number of different update information data, recorded as x α,i It should be noted that all the update information data corresponding to the second associated update information data set are update information data in the αth project cluster;

[0092] t i is the time interval from the first appearance to the last appearance of the i-th updated information data in all shipping manifest data; σ i is the variance of the time intervals between all occurrences of the ith update information data in all shipping manifest data; μ i is the mean of the time intervals between all occurrences of the ith update information data in all shipping manifest data; W α,i is the stability of the i-th updated information data in the α-th item cluster.

[0093] It should be noted that X α,i The larger the value is, the less the number of changes of the i-th update information data is when the corresponding other update information data are the same. α,i The larger the i The larger the value is, the longer the duration of the i-th updated information data in all shipping manifest data is. α,i The larger the i ×μ i The smaller it is, the smaller the time interval between the i-th updated information data in all shipping manifest data is, the smaller the fluctuation is and the shorter the interval is. That is, the more frequent the shipping business is, the higher the W α,i The bigger.

[0094] At this point, the stability of each update information data in each project cluster is obtained.

[0095] Step S004: According to the number of different update information data in the project cluster, the repetition weight and the stability of the update information data, a number of project clusters to be compressed and a number of project clusters not to be compressed are obtained; the update information data in the project clusters to be compressed and the project clusters to be compressed are encoded to obtain encoded data; the encoded data and the update information data in all the project clusters not to be compressed are transmitted.

[0096] It should be noted that the above steps analyze the repetition weight and stability of each update information data in each project cluster respectively, and then analyze the necessity of compression of each project cluster, so that the project cluster with greater compression necessity can be compressed in the future, better transmitted, and improved transmission efficiency; since the necessity of compression is related to the number of different update information data in the project cluster, the repetition weight and stability of the update information data, the fewer the number of different update information data in the project cluster, the more repetitions the update information data in the project cluster has, the more it should be compressed, and the greater the necessity of compression; at the same time, if the repetition weight and stability of the update information data in the project cluster are greater, it means that when compressed, it should be compressed more, and the necessity of compression is also greater; therefore, by analyzing the number of different update information data in the project cluster, the repetition weight and stability of the update information data, the necessity of compression of each project cluster is obtained, and then a number of project clusters to be compressed and a number of project clusters not to be compressed are screened out.

[0097] Specifically, according to the number of different update information data in the project cluster, the repetition weight and stability of the update information data, a number of project clusters to be compressed and a number of project clusters not to be compressed are obtained, including the following steps:

[0098] Firstly, according to the number of different update information data in the project cluster, the repetition weight and the stability of the update information data, the compression necessity of each project cluster is obtained.

[0099] Secondly, according to the necessity of compression, a number of clusters of items to be compressed and a number of clusters of items not to be compressed are obtained.

[0100] Preferably, the necessity of compression of each project cluster is obtained according to the number of different update information data in the project cluster, the repetition weight and the stability of the update information data, as follows:

[0101] For any project cluster; obtain the number of different update information data in the project cluster; multiply the repetition weight and stability of each update information data in the project cluster to obtain the first product of each update information data in the project cluster; obtain the cumulative value of the first products of all update information data in the project cluster; merge the number of different update information data in the project cluster and the cumulative value of the first product to obtain the compression necessity of the project cluster, the number of different update information data in the project cluster is inversely proportional to the compression necessity, the cumulative value of the first product is directly proportional to the compression necessity, and the compression necessity is a normalized value.

[0102] As a specific example, the specific method for obtaining the compression necessity is as follows:

[0103]

[0104] In the formula, m a is the number of different update information data in the a-th project cluster; n1 α is the number of updated information data in the a-th project cluster; Q α,j W is the repetition weight of the jth updated information data in the αth item cluster; α,j is the stability of the jth updated information data in the αth item cluster; tanh[] is the hyperbolic tangent function used for normalization; Z α is the necessity of compression for the αth item cluster.

[0105] It should be noted that m a The smaller it is, the fewer the number of different update information data in the project cluster is, and the more duplicate update information data in the project cluster is. α The bigger the Q α,j ×W α,j The larger the value, the higher the repetition weight and stability of the jth updated information data in the project cluster. α The bigger.

[0106] Furthermore, according to the necessity of compression, several clusters of items to be compressed and several clusters of items not to be compressed are obtained, as follows:

[0107] A third threshold is preset. In this embodiment, the third threshold is described as 0.5. Project clusters corresponding to compression necessities greater than the third threshold are recorded as project clusters to be compressed, and project clusters corresponding to compression necessities less than or equal to the third threshold are recorded as project clusters not to be compressed.

[0108] It should be noted that the above-mentioned obtained item clusters to be compressed and item clusters not to be compressed. The necessity of compression corresponding to the item clusters to be compressed is relatively large, and the updated information data within the clusters have many repetitions. Therefore, the updated information data in the item clusters to be compressed and the item clusters to be compressed are encoded and compressed, and the encoded data is obtained and transmitted with the updated information data in all the item clusters not to be compressed.

[0109] Specifically, the to-be-compressed item cluster and the update information data in the to-be-compressed item cluster are encoded to obtain encoded data; specifically, as follows:

[0110] All the clusters of items to be compressed are sorted from large to small according to the corresponding compression necessity, and the encoding is started from the first cluster of items to be compressed. The corresponding code of the first cluster of items to be compressed is A1, and the corresponding code of the second cluster of items to be compressed is A2. The encoding is continued in sequence until all the clusters of items to be compressed have corresponding codes; starting from the first cluster of items to be compressed, all the update information data in the first cluster of items to be compressed are sorted from large to small according to the number of occurrences, and the encoding is started from the first update information data. The first update information data is encoded as B11, and the second update information data is encoded as B12 , and encode in sequence until all the updated information data in the first item cluster to be compressed have corresponding codes; according to the encoding method of the updated information data in the first item cluster to be compressed, all the updated information data in the second item cluster to be compressed are encoded, the first updated information data in the second item cluster to be compressed is encoded as B21, the second updated information data in the second item cluster to be compressed is encoded as B22, until all the updated information data in the second item cluster to be compressed have corresponding codes, encode the updated information data in all item clusters to be compressed, obtain the encoded data, and obtain the encoding table through statistics.

[0111] It should be noted that the purpose of obtaining the coding table is to facilitate subsequent decoding. Each update information data in the coding table corresponds to a code. For example, the code corresponding to the first update information data of the first item cluster to be compressed is A1B11.

[0112] Furthermore, the encoded data and the update information data in all non-to-be-compressed item clusters are transmitted.

[0113] It should be noted that transmitting data is an existing method, which will not be described in detail in this embodiment; the transmitted data can be decoded and restored through the coding table later.

[0114] Through the above steps, a method for transmitting shipping manifest data is completed.

[0115] Another embodiment of the present invention provides a system for transmitting shipping manifest data, the system comprising a memory and a processor, and when the processor executes a computer program stored in the memory, the processor performs the following operations:

[0116] Obtain a number of shipping manifest data of a shipping company within a preset period of time; classify the information data in all shipping manifest data according to the project name to obtain a number of project clusters; obtain a number of project cluster sets according to the synchronous changes of the information data in different project clusters under the preset sorting rules and the number of different information data in the project clusters; update the information data according to the position difference of the corresponding characters of different information data in the same project cluster and the proportion of repeated characters in the information data to obtain updated information data; obtain the number of each updated information data in each project cluster according to the proportion of repeated characters in the updated information data in the project cluster. According to the repetition weight of the updated information data; according to the number of times the updated information data in the item clusters within the item cluster set changes, the frequency and fluctuation of the updated information data in all shipping manifest data, the stability of each updated information data in each item cluster is obtained; according to the number of different updated information data in the item clusters, the repetition weight and the stability of the updated information data, a number of item clusters to be compressed and a number of item clusters not to be compressed are obtained; the updated information data in the item clusters to be compressed and the item clusters to be compressed are encoded to obtain encoded data; the encoded data and the updated information data in all item clusters not to be compressed are transmitted.

[0117] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for transmitting shipping manifest data, characterized in that: The method comprises the following steps: Obtain a number of shipping manifest data from a shipping company over a preset period of time; Classify the information data in all shipping manifest data according to the project name to obtain several project clusters; obtain several project cluster sets according to the synchronous changes of the information data in different project clusters under the preset sorting rules and the number of different information data in the project clusters; update the information data according to the position differences of the corresponding characters of different information data in the same project cluster and the proportion of repeated characters in the information data to obtain updated information data; According to the proportion of repeated occurrences of updated information data in the project cluster, the repetition weight of each updated information data in each project cluster is obtained; according to the number of changes in the updated information data in the project cluster within the project cluster set, the frequency and fluctuation of the updated information data in all shipping manifest data, the stability of each updated information data in each project cluster is obtained; According to the number of different update information data in the project cluster, the repetition weight and the stability of the update information data, a number of project clusters to be compressed and a number of project clusters not to be compressed are obtained; the update information data in the project clusters to be compressed and the project clusters to be compressed are encoded to obtain encoded data; the encoded data and the update information data in all the project clusters not to be compressed are transmitted; The specific steps of obtaining the stability of each updated information data in each item cluster according to the number of times the updated information data in the item cluster set changes, the frequency and fluctuation of the updated information data in all shipping manifest data, include the following: For any updated information data in any project cluster; obtain the stability factor of the updated information data in the project cluster, and the specific method for obtaining the stability factor is: record the project cluster set to which the project cluster belongs as the first project cluster set; obtain all the updated information data corresponding to the updated information data in other project clusters in the first project cluster set as the associated updated information data set corresponding to the updated information data in the project cluster, recorded as the first associated updated information data set; obtain the associated updated information data set corresponding to each updated information data in the project cluster, and record the associated updated information data sets that are the same as the first associated updated information data set as the second associated updated information data set; obtain the updated information data corresponding to all the second associated updated information data sets and obtain the number of different updated information data, recorded as the number of times the updated information data in the project cluster changes, and use the value after the number of times the updated information data in the project cluster changes through the inverse proportional function mapping as the stability factor of the updated information data in the project cluster; Obtain the time interval from the first appearance to the last appearance of the update information data in all shipping manifest data; obtain the variance of the time intervals of all appearances of the update information data in all shipping manifest data; obtain the mean of the time intervals of all appearances of the update information data in all shipping manifest data; merge the stability factor, the time interval from the first appearance to the last appearance, the variance of the time interval, and the mean of the time interval to obtain the stability of the update information data in the project cluster; the stability factor and the time interval from the first appearance to the last appearance are in direct proportion to the stability, and the variance of the time interval and the mean of the time interval are in inverse proportion to the stability.

2. A method for transmitting shipping manifest data according to claim 1, characterized in that: The specific steps of obtaining a plurality of project cluster sets according to the synchronous changes of information data in different project clusters under the preset sorting rules and the quantity of different information data in the project clusters are as follows: For any two project clusters, all information data in the first project cluster are sorted from most to least according to the number of characters, and the number of times the information data in the first project cluster changes and the number of times the information data in the second project cluster changes are obtained, and the number of different information data in the first project cluster and the number of different information data in the second project cluster are obtained, and the ratio of the number of times the information data in the first project cluster changes to the number of times the information data in the second project cluster changes is recorded as a first ratio, and the absolute value of the difference between the first ratio and 1 is used as an asynchronous change index of the first project cluster and the second project cluster, and the product of the number of different information data in the first project cluster and the number of different information data in the second project cluster is recorded as a first product, and the asynchronous change index and the first product are fused to obtain the correlation between the first project cluster and the second project cluster, the asynchronous change index and the first product are inversely proportional to the correlation, and the correlation is a normalized value; according to the size of the correlation, a number of project cluster sets are obtained.

3. A method for transmitting shipping manifest data according to claim 2, characterized in that: The specific steps of obtaining a plurality of project cluster sets according to the magnitude of the correlation are as follows: The correlation between every two project clusters is obtained, two project clusters whose correlation is greater than or equal to a preset first threshold are divided into a project cluster set, and the correlation between all project clusters is judged in turn to obtain several project cluster sets.

4. A method for transmitting shipping manifest data according to claim 1, characterized in that: The specific steps of updating the information data according to the position difference of the characters corresponding to different information data in the same project cluster and the proportion of repeated characters in the information data to obtain the updated information data are as follows: For two information data of any project cluster, obtain the number of all characters in the first information data that appear in the second information data, record it as the first number, and take the ratio of the first number to the total number of characters in the first information data as the ratio of repeated occurrences of characters in the first information data; The characters in the first information data and the second information data are numbered consecutively starting from 1 in order, the absolute value of the difference between the number value of each character in the first information data and the number value of the corresponding same character in the second information data is recorded as a number value difference, and the average of all number value differences corresponding to the first information data and the second information data is used as the position difference of the characters of the first information data and the second information data; the ratio of the first number to the total number of characters in the first information data and the average of the number value differences are merged to obtain the correlation between the first information data and the second information data, the ratio of the first number to the total number of characters in the first information data is in direct proportion to the correlation, the average of the number value differences is inversely proportional to the correlation, and the correlation is a normalized value; the information data is updated according to the size of the correlation to obtain updated information data.

5. A method for transmitting shipping manifest data according to claim 4, characterized in that: The updating of information data according to the magnitude of the correlation to obtain updated information data includes the following specific steps: A second threshold is preset, and two information data corresponding to a correlation greater than the second threshold are regarded as a pair of similar information data. A pair of similar information data are both updated to the information data with the largest number of characters, and vice versa, to their original information data, thereby obtaining a number of updated information data.

6. A method for transmitting shipping manifest data according to claim 1, characterized in that: The method of obtaining the repetition weight of each update information data in each project cluster according to the repetition ratio of the update information data in the project cluster includes the following specific steps: For any update information data in any project cluster, obtain the number of repetitions of the update information data in the project cluster, obtain the number of update information data in the project cluster, and record the ratio of the number of repetitions of the update information data in the project cluster to the number of update information data in the project cluster as the first proportion of repetitions of the update information data; obtain the maximum number of repetitions of the update information data in the project cluster, and record the ratio of the number of repetitions of the update information data in the project cluster to the maximum number of repetitions of the update information data in the project cluster as the second proportion of repetitions of the update information data; The first proportion and the second proportion are integrated to obtain a repetition weight of the update information data in the project cluster, and the first proportion and the second proportion are in direct proportion to the repetition weight.

7. A method for transmitting shipping manifest data according to claim 1, characterized in that: The specific steps of obtaining a plurality of project clusters to be compressed and a plurality of project clusters not to be compressed according to the number of different update information data in the project cluster, the repetition weight and the stability of the update information data are as follows: For any project cluster; obtain the number of different update information data in the project cluster; multiply the repetition weight and stability of each update information data in the project cluster to obtain the first product of each update information data in the project cluster; obtain the cumulative value of the first products of all update information data in the project cluster; merge the number of different update information data in the project cluster and the cumulative value of the first product to obtain the compression necessity of the project cluster, the number of different update information data in the project cluster is inversely proportional to the compression necessity, the cumulative value of the first product is directly proportional to the compression necessity, and the compression necessity is a normalized value; A third threshold is preset, and the project clusters corresponding to the compression necessity greater than the third threshold are recorded as the project clusters to be compressed, and the project clusters corresponding to the compression necessity less than or equal to the third threshold are recorded as the project clusters not to be compressed.

8. A method for transmitting shipping manifest data according to claim 7, characterized in that: The encoding of the to-be-compressed item cluster and the update information data in the to-be-compressed item cluster to obtain the encoded data includes the following specific steps: Sort all the project clusters to be compressed from large to small according to the corresponding compression necessity, and start coding from the first project cluster to be compressed. The code corresponding to the first project cluster to be compressed is A1, and the code corresponding to the second project cluster to be compressed is A2. Code in sequence until all the project clusters to be compressed have corresponding codes; Starting from the first item cluster to be compressed, all update information data in the first item cluster to be compressed are sorted from large to small according to the number of occurrences, and encoding is started from the first update information data. The first update information data is encoded as B11, and the second update information data is encoded as B12, and so on, until all update information data in the first item cluster to be compressed have corresponding codes; according to the encoding method of the update information data in the first item cluster to be compressed, all update information data in the second item cluster to be compressed are encoded. The first update information data in the second item cluster to be compressed is encoded as B21, and the second update information data in the second item cluster to be compressed is encoded as B22, until all update information data in the second item cluster to be compressed have corresponding codes, and the update information data in all item clusters to be compressed are encoded to obtain encoded data.

9. A system for transmitting marine manifest data, the system comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the computer program is executed by a processor, the steps of a method for transmitting shipping manifest data as described in any one of claims 1 to 8 are implemented.

Images