Distributed storage method and system for production data of functional non-dairy creamer
By constructing histograms and two-dimensional matrices, analyzing the spatial distribution characteristics of the production data of lipid planting, and determining the scanning compression path, the problem of low compression efficiency in the existing technology is solved and more efficient data compression is achieved.
Patent Information
- Application Number
- CN202411523624.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-30
- Publication Date
- 2024-11-26
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
When the prior art uses run encoding to compress functional fat-plant production data, it is subject to slight fluctuations caused by stirring operations, the compression efficiency is low, the repetition rate of adjacent data is small, and there is a lot of room for improvement.
By obtaining the production data of lecithin, constructing a histogram, calculating the degree of suspected target seed points of the peak point, converting it into a two-dimensional matrix, analyzing the spatial distribution characteristics of the data, determining the most suitable scanning compression path, and improving compression efficiency.
Improves compression efficiency, increases the possibility of adjacent data duplication, and improves the overall compression effect of data.
Smart Images

Figure CN119025053A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of electronic data compression processing, and in particular to a distributed storage method and system for functional non-dairy creamer production data. Background Art
[0002] Functional non-dairy creamer is a processed vegetable fat product, usually used in the food industry, which can improve the internal structure of food, increase flavor and fat, and make the taste delicate. In the process of non-dairy creamer preparation, the generated monitoring data needs to be distributed and stored. Storing this data can help enterprises and factories control the quality of non-dairy creamer, trace problems, optimize production processes, and meet regulatory requirements. Therefore, each distributed storage server needs to meet the storage requirements of a large amount of data. A compression algorithm needs to be used to compress the distributed storage of non-dairy creamer production data.
[0003] The production process of non-dairy creamer is gentle and changes slowly, and the similarity of production monitoring data is high. The existing technology can usually use the run-length encoding algorithm to compress the collected non-dairy creamer production data. Run-length encoding compresses data based on the repetition rate of adjacent identical data. However, due to the stirring operation during the preparation of non-dairy creamer, the production monitoring data will cause slight fluctuations, resulting in a large degree of overall data repetition but a small repetition rate of adjacent data. When using run-length encoding for compression, the compression efficiency is low, and there is a large room for improvement in run-length encoding data compression. Summary of the invention
[0004] In order to solve the above problems, the present invention provides a distributed storage method and system for functional non-dairy creamer production data, and the technical solutions adopted are as follows: In a first aspect, an embodiment of the present invention provides a distributed storage method for functional non-dairy creamer production data, comprising the following steps: Obtaining non-dairy creamer production data and preprocessing it to obtain sequence data to be compressed; constructing a histogram using the sequence data to be compressed; The peak points are obtained by using the histogram, and the degree of the suspected target seed point of the peak point is calculated according to the peak values adjacent to the peak point; the suspected target seed point is obtained by using the degree of the suspected target seed point; A two-dimensional matrix is constructed using the sequence data to be compressed; the floating degree of the suspected target seed point is obtained using the data value in the neighborhood of the suspected target seed point in the two-dimensional matrix; The suspected target seed points are screened according to the floating degree to obtain the target seed points; the region is grown based on the target seed points to obtain the growth region; Calculate the horizontal distribution degree and the vertical distribution degree of the two-dimensional matrix according to each growth area and the minimum circumscribed rectangle of each growth area; A scanning compression path is determined based on the horizontal distribution degree and the vertical distribution degree of the two-dimensional matrix; the two-dimensional matrix is compressed based on the scanning compression path, and the compressed data is distributedly stored.
[0005] Preferably, constructing a histogram using the sequence data to be compressed includes: The sequence data to be compressed is mapped to a histogram, wherein the horizontal axis of the histogram is the value of the data in the sequence data to be compressed, and the data values are sorted from small to large in the horizontal axis direction; the vertical axis of the histogram is the number of data corresponding to each data value in the sequence data to be compressed.
[0006] Preferably, obtaining each peak point using a histogram includes: Connect the highest points of the histogram with a smooth curve to obtain a coordinate curve; derive the coordinate curve to obtain a derivative curve, and use the derivative curve to obtain all peak points.
[0007] Preferably, the calculation formula for the degree of the suspected target seed point of the peak point is: , Among them, R represents the degree of suspected target seed point of a peak point; represents the peak value of the peak point; N represents the number of data in the sequence data to be compressed; Indicates the vertical axis value corresponding to the i-th horizontal axis value to the right of the peak point. Indicates the vertical axis value corresponding to the j-th horizontal axis value on the left side of the peak point; Indicates the number of horizontal axis values between the right side of the peak point and the trough. Indicates the number of horizontal axis values between the left side of the peak point and the trough; Indicates the distance between the i-th horizontal axis value on the right and the peak point. It represents the distance from the jth horizontal axis value to the left of the peak point to the peak point; exp{} represents an exponential function with a natural constant as the base.
[0008] Preferably, obtaining the suspected target seed point by using the suspected target seed point degree includes: The suspected target seed point degrees of all peak points are grouped into a sequence and linearly normalized to obtain a suspected sequence; half of the data with high suspected target seed point degrees in the suspected sequence are obtained, and the corresponding peak points are the suspected peak points; the data in all the sequence data to be compressed corresponding to the suspected peak points are the suspected target seed points.
[0009] Preferably, the calculation formula for the floating degree of the suspected target seed point is: , Among them, D represents the floating degree of a suspected target seed point; M represents the number of data in the 8-neighborhood of a suspected target seed point; Indicates the number of data with the same value as the suspected target seed point in the neighborhood of the suspected target seed point; Represents the data value of the suspected target seed point, represents the mth data value in the neighborhood of the suspected target seed point; exp{} represents an exponential function with a natural constant as the base.
[0010] Preferably, the calculation formulas for the horizontal distribution degree and the vertical distribution degree of the two-dimensional matrix are respectively: , , in, and They represent the horizontal distribution degree and the vertical distribution degree of the two-dimensional matrix respectively; and represents the horizontal and vertical lengths of the minimum circumscribed rectangle corresponding to the qth growth region; represents the number of data points in the qth growing region; represents the number of data points in the minimum bounding rectangle corresponding to the qth growth region; represents the number of data points that are not growth points in the eight-neighborhood of the bth data point in the qth growth region; represents the hyperbolic tangent function; Q represents the number of growth regions in the two-dimensional matrix; exp{} represents an exponential function with a natural constant as the base.
[0011] Preferably, determining the scanning compression path based on the horizontal distribution degree and the vertical distribution degree of the two-dimensional matrix includes: When the horizontal distribution degree is greater than or equal to the vertical distribution degree, a scanning compression path of horizontal scanning is adopted; when the horizontal distribution degree is less than the vertical distribution degree, a scanning compression path of longitudinal scanning is adopted.
[0012] Preferably, compressing the two-dimensional matrix based on the scanning compression path and distributing the compressed data includes: When the scanning compression path is horizontal scanning, the upper left corner of the two-dimensional matrix is taken as the starting point, and the two-dimensional matrix is traversed by S-shaped horizontal scanning to obtain a compression sequence; when the scanning compression path is vertical scanning, the upper left corner of the two-dimensional matrix is taken as the starting point, and the two-dimensional matrix is traversed by S-shaped vertical scanning to obtain a compression sequence.
[0013] In a second aspect, an embodiment of the present invention provides a distributed storage system for functional non-dairy creamer production data, comprising a memory and a processor, wherein the processor executes a computer program stored in the memory to implement the above-mentioned distributed storage method for functional non-dairy creamer production data.
[0014] The present invention has at least the following beneficial effects: The present invention constructs a histogram according to the sequence data to be compressed, and then obtains each peak point, and calculates the degree of the suspected target seed point of the peak point according to the peak value adjacent to the peak point; the suspected target seed point is obtained by using the degree of the suspected target seed point, and the sequence data to be compressed is further converted into a two-dimensional matrix, and the suspected target seed point is optimized according to the data space distribution characteristics in the two-dimensional matrix to obtain a target seed point with better compression effect, thereby improving the accuracy of subsequent analysis; Further analysis is performed based on the growth area of the target seed point to obtain the most suitable scanning compression path, and scanning is performed to obtain the compression sequence, thereby increasing the degree of adjacent repetition in the compression path and improving the compression efficiency of run-length encoding for non-dairy creamer production data; At the same time, the present invention also constructs the original sequence data to be compressed into a two-dimensional matrix, and converts the one-dimensional non-dairy creamer production data into two-dimensional data. Compared with the prior art that directly compresses the original data, the data only exists adjacent to the left and right. The data in the two-dimensional matrix has adjacent data points in more directions, which increases the possibility of adjacent data duplication, thereby improving the compression efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0016] Figure 1 The present invention is a method flow chart of a distributed storage method for functional non-dairy creamer production data. DETAILED DESCRIPTION
[0017] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the embodiments of the present invention.
[0018] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.
[0019] The specific scenarios targeted by the present invention are: When the run-length encoding algorithm is used to compress and store the monitoring data collected during the production process of non-dairy creamer, due to the high precision requirements in the production and preparation process, the monitoring data is prone to slight fluctuations during the mixing and preparation process, resulting in a decrease in the repetition rate of adjacent data and a decrease in the run-length encoding compression efficiency.
[0020] This embodiment provides a distributed storage method for functional non-dairy creamer production data, which is described in detail as follows: like Figure 1 As shown, a distributed storage method for functional non-dairy creamer production data comprises the following steps: Step S1, obtaining non-dairy creamer production data and preprocessing it to obtain sequence data to be compressed; and constructing a histogram using the sequence data to be compressed.
[0021] During the production process of non-dairy creamer, various data need to be collected and stored in order to monitor the production of non-dairy creamer or improve its production process.
[0022] Furthermore, multiple sensors are installed to collect production data. The sensors used to collect data include temperature sensors, humidity sensors, Ph sensors, etc. The specific data collected needs to be determined by the implementer based on the actual situation on site.
[0023] Preferably, in the embodiment of the present invention, the accuracy of data collection is 0.1 data unit. Among them, the data collection interval is unified. In the embodiment of the present invention, the collection interval is once every 10 seconds, and the collection period is 10 minutes. When performing actual operations, the implementer can adjust the collection interval and collection period. Since the time intervals are the same, they need to be preprocessed and dimensioned. The data obtained only have corresponding values and are stored in a separate sequence, which is recorded as sequence data to be compressed. The collected sequence data to be compressed is analyzed and processed, and compressed and stored. It should be noted that the sequence data to be compressed is a kind of monitoring data collected within one cycle.
[0024] Furthermore, the sequence data to be compressed is first analyzed to adaptively obtain suspected target seed points. The sequence data to be compressed is mapped to a histogram, the horizontal axis of the histogram is the value of the data in the sequence data to be compressed, and the data is sorted from small to large in the horizontal direction according to the data value, and the vertical axis of the histogram is the number of data of this value in the sequence data to be compressed. Analyze the characteristics of the data in the histogram. If there is a large peak in the histogram, it means that the number of data points corresponding to the peak data value at this location occupies a large proportion in the overall data, which may be the peak period of the same monitoring data in the production process of non-dairy creamer, then the overall repetition degree of the monitoring data is large; if there is a peak with little difference between the front and back of the histogram, and the trough extending to both sides of the peak is shallow (corresponding to the data sequence, the floating degree is small), it means that the number of data points corresponding to the peak data value occupies a certain proportion in the overall trend, and there are floating data with little difference on the left and right, then each peak data with such characteristics is more likely to be similar to each other, and more suitable for growth.
[0025] Next, after mapping the sequence data to be compressed into a histogram, the highest points of the histogram are connected with a smooth curve to obtain a coordinate curve, which is used for subsequent trend change analysis to obtain the peak point.
[0026] Step S2, using the histogram to obtain each peak point, and calculating the suspected target seed point degree of the peak point according to the peak values adjacent to the peak point; using the suspected target seed point degree to obtain the suspected target seed point.
[0027] First, in step S1, the histogram of the sequence data to be compressed and the coordinate curve are obtained. Further, the coordinate curve is differentiated to obtain the derivative curve, and the zero point on the derivative curve is found, which corresponds to the extreme point on the curve, and then the maximum point is obtained. The maximum point corresponds to the peak, which is the point where the derivative changes from negative to positive. In this way, all the peaks in the histogram are obtained, and all the peak points are obtained.
[0028] The ratio of the peak point value to the number of all data is obtained as the proportion of the peak; the difference before and after the peak point is calculated to represent the difference before and after the peak. The process is to accumulate the number of amplitudes (vertical axis values) between the left and right sides of the peak point and the trough, and subtract the number of amplitudes corresponding to each amplitude (horizontal axis value) from the number of amplitudes of the peak point in the middle. The difference is weighted according to the distance from the peak, and the weight is the ratio of 1 to the distance from the peak point. After weighting, the data closer to the peak point has a greater impact on the final result. The weighted cumulative sum average of all amplitudes from the central peak point to the troughs on both sides is calculated to represent the floating degree of the data on both sides of the peak point. At the same time, the proportion and floating degree of the peak point are combined to obtain the suspected target seed point degree of the peak point. The calculation formula for the suspected target seed point degree of the peak point is: , Among them, R represents the degree of suspected target seed point of a peak point; Indicates the peak value of the peak point, that is, the number of amplitudes corresponding to the peak amplitude, that is, the number of data in the sequence data to be compressed corresponding to a peak point; Represents the sum of all amplitude numbers, that is, the number of data in the sequence data to be compressed; is the proportion of the peak value of the peak point in the total amount of data. The larger the value is, the higher the degree of suspected target seed point is, and the greater the overall repetitive redundancy of the amplitude in the sequence data to be compressed is; is the vertical axis value corresponding to the i-th horizontal axis value on the right side of the peak point, is the vertical axis value corresponding to the jth horizontal axis value on the left side of the peak point; is the number of horizontal axis values between the right side of the peak point and the trough. It is the number of horizontal axis values between the left side of the peak point and the trough; It represents the i-th horizontal axis value to the right of the peak point, and also represents the distance between this horizontal axis value and the peak point. Indicates the jth horizontal axis value to the left of the peak point, and also indicates the distance between this horizontal axis value and the peak point; is the difference between the peak value and the vertical axis value of the i-th horizontal axis value, is the difference between the peak value and the vertical axis value of the jth horizontal axis value; and is the weight of the difference, expressed as the inverse of the distance from the horizontal axis value. The farther the distance, the smaller the impact of the difference.
[0029] is the weighted cumulative mean to the right of the peak point. It is the weighted cumulative sum average of the left side of the peak point, and the sum is obtained by adding the values on both sides of the peak point. This value reflects the degree of floating on both sides of the peak point. The smaller the floating degree, the greater the overall redundancy of the amplitude on the left and right sides of the peak point, that is, the higher the basic similarity, and the higher the degree of suspected target seed point of the peak point. Inverse proportional quantization is used. The suspected target seed point degree is obtained by combining the characteristics of similarity of the adjacent monitoring data in the non-dairy creamer production monitoring data, and obtaining data with a high overall repetitive redundancy degree and a high adjacent similarity possibility in the sequence data to be compressed as the suspected target seed point.
[0030] Furthermore, after obtaining the degree of the suspected target seed point of each peak point, the degrees of the suspected target seed points of all peak points are combined into a sequence and linearly normalized to obtain a suspected sequence; half of the peak points with high degrees of suspected target seed points in the suspected sequence are obtained and recorded as suspected peak points; the data in all the sequence data to be compressed corresponding to the suspected peak points are suspected target seed points.
[0031] Step S3, constructing a two-dimensional matrix using the sequence data to be compressed; obtaining the floating degree of the suspected target seed point using the data value in the neighborhood of the suspected target seed point in the two-dimensional matrix.
[0032] Since the data obtained from histogram analysis does not have spatial location information, it is necessary to perform spatial analysis on the suspected target seed points.
[0033] Furthermore, the sequence data to be compressed (original data) is upgraded and converted into a two-dimensional matrix. The last data of a row in the matrix and the first data of the next row correspond to the two data before and after in the sequence. The converted matrix is required to be a square matrix as much as possible. If the length c of the sequence data to be compressed is a perfect square number (such as 4, 9, 16, etc.), the sequence can be converted into a matrix with m rows and m columns. This ensures that the matrix is a square matrix and is as close to a square as possible. If the length c of the sequence is not a perfect square number, it can be decomposed into two factors a and d (a×d=c), and then the sequence is converted into a matrix with a rows and d columns. The values of a and d can be selected so that the number of rows and columns of the matrix are close, so that the constructed matrix is as square as possible. If the length of the sequence is a prime number, it can be added by 1 and then converted. The last position in the converted matrix can be empty. Run-length encoding of such a matrix increases the selectivity of the encoding direction compared to the sequence, which is the basis for the subsequent placement of seed points for regional growth. On this basis, the encoding path with a higher degree of adjacent repeated redundancy can be analyzed. In this way, the two-dimensional matrix of the sequence data to be compressed can be obtained.
[0034] The suspected target seed points are located in the two-dimensional matrix, and the spatial feature analysis of these suspected target seed points is performed to obtain the optimal seed points that meet the spatial position. The main analysis form is the eight-neighborhood feature analysis. The eight neighborhoods of the suspected target seed point are traversed, and the number of neighbors with the same value as the suspected target seed point is accumulated. If there are many neighbors with the same value, it means that the adjacent data at this location is more likely to be the same; the variance composed of the difference between the neighborhood and the suspected target seed point is obtained. The smaller the variance, the higher the possibility of adjacent similarity of the data around the suspected target seed point, the smaller the floating property, and the more suitable for growth.
[0035] Traverse the eight neighborhoods of each suspected target seed point to obtain the number of neighborhoods with the same value as the suspected target seed point, which represents the same possibility degree; perform difference in the eight neighborhoods and then use the obtained difference set to perform variance operation to obtain the similarity possibility degree. The higher the same possibility degree and similar possibility degree, the smaller the floating of the neighborhood data is, and the more suitable it is for growth, so the floating degree of the suspected target seed point can be obtained.
[0036] The calculation formula for the floating degree of the suspected target seed point is: , Wherein, D represents the floating degree of a suspected target seed point; M represents the number of data in an 8-neighborhood of a suspected target seed point. In the embodiment of the present invention, an 8-neighborhood is used, so the value of M is 8; Indicates the number of data with the same value as the suspected target seed point in the neighborhood of the suspected target seed point; The higher the ratio, the more identical data there are in the neighborhood, and the higher the possibility of spatial identity. Represents the data value of the suspected target seed point, Represents the mth data value in the neighborhood of the suspected target seed point; is the difference between the target seed point value and the i-th neighborhood value, It represents the variance form of the difference. The smaller the value, the closer the difference is, indicating that the neighborhood data is likely to be similar, and the inverse proportional quantization is adopted. Finally, the target seed point has a better possible degree D. In this way, the floating degree of each suspected target seed point can be obtained.
[0037] The area grown from the initial seed point placement point obtained by using this floating degree is larger, the overall distribution is more orderly, the possibility of monitoring data with the same value in the growth area is greater, the degree of duplication and redundancy of the overall data in the growth area is improved, and the compression effect is more ideal.
[0038] Step S4, screening the suspected target seed points according to the floating degree to obtain the target seed points; performing region growing based on the target seed points to obtain the growing region.
[0039] First, in step S3, the floating degree of each suspected target seed point is obtained, and the floating degree of the suspected target seed point is further used for screening. Specifically, a degree threshold is set. Preferably, the degree threshold in the present invention is 0.5. The implementer can adjust the degree threshold according to actual conditions. When the floating degree of the suspected target seed point is greater than the degree threshold, the suspected target seed point is the target seed point.
[0040] Next, after obtaining the target seed point, it is necessary to use the target seed point for regional growth. The target seed point is used for regional growth. The growth rule is to grow the data values in the eight neighborhoods of the target seed point with a preset value (the data set itself has a small fluctuation) to obtain a growth area, where the preset value is 0.1. After the growth is completed, the data values of all points in the growth area are averaged to replace the values of all data points in the growth area.
[0041] Step S5, calculating the horizontal distribution degree and the vertical distribution degree of the two-dimensional matrix according to each growth region and the minimum circumscribed rectangle of each growth region.
[0042] In step S4, the growth regions grown from the target seed points are obtained, and the morphological analysis is performed on these growth regions in the two-dimensional matrix. When the longitudinal distribution of the growth region morphology is greater than the lateral distribution, more similar data will be obtained by the longitudinal scanning method, and when the lateral distribution of the growth region morphology is greater than the longitudinal distribution, more similar data will be obtained by the lateral scanning method.
[0043] The minimum bounding rectangle of the growth area after all target seed points have grown completely is obtained. In the two-dimensional matrix composed of the sequence data to be compressed, these minimum bounding rectangles only exist in two forms, one is a rectangle and the other is a square. When it presents a rectangular form, there is a horizontal distribution and a vertical distribution. Regardless of the distribution, the larger the aspect ratio, the more obvious the distribution of this form in the growth area. The contribution rate of the rectangle to its own regional distribution characteristics, that is, the weight, is obtained through the internal filling rate and internal filling discreteness of the bounding rectangle. The higher the internal filling rate and the smaller the discreteness, the more the minimum bounding rectangle can represent the shape distribution of the actual growth area.
[0044] The minimum circumscribed rectangle of each growth area is obtained, and the length of the horizontal side of the minimum circumscribed rectangle is compared with the length of the vertical side to obtain the aspect ratio of the rectangle; the internal filling rate is calculated by dividing the area of the circumscribed rectangle by the area of the actual internal growth area; the internal discreteness is expressed by the cumulative and average number of free blocks around each point in the growth area inside the rectangle. The fewer the vacancies around each point on average, the denser the overall discreteness is.
[0045] Specifically, the calculation formulas for the horizontal distribution degree and the vertical distribution degree of the two-dimensional matrix are: , , in, and They represent the horizontal distribution degree and the vertical distribution degree of the two-dimensional matrix respectively; and represents the horizontal and vertical lengths of the minimum circumscribed rectangle corresponding to the qth growth region; represents the number of data points in the qth growth region, which can also be expressed as the area of the growth region; It represents the number of data points in the minimum bounding rectangle corresponding to the qth growth region, which can also be expressed as the area of the minimum bounding rectangle; represents the number of data points that are not growth points in the eight-neighborhood of the bth data point in the qth growth region; represents a hyperbolic tangent function; Q represents the number of growth regions in a two-dimensional matrix; exp{} represents an exponential function with a natural constant as the base, and exp{} in the embodiments of the present invention all represent an exponential function with a natural constant as the base.
[0046] and represents the horizontal and vertical lengths of the minimum circumscribed rectangle corresponding to the qth growth region. The distribution of the growth region is horizontal. Larger, vertical Larger; It represents the internal filling rate of the minimum circumscribed rectangle corresponding to a growth area. The higher the filling rate, the greater the contribution of this growth area to the overall distribution. It is quantified in direct proportion. surface represents the number of data points in the eight-neighborhood of the bth data point in the qth growth region that are not growth points (that is, the points in the minimum enclosing rectangle that are not in the growth region), It is the mean of the number of data points that are not growth points in the eight neighborhoods of all data points in the qth growth region, indicating the level of vacancy around the overall midpoint. The smaller the value, the denser the data, and the greater the contribution rate to the overall data. Inverse proportional quantification is used. Finally, the degree of all individual regions is summed to obtain the horizontal distribution degree and vertical distribution degree of the two-dimensional matrix as a whole.
[0047] Thus, the overall horizontal and vertical distribution degrees of the two-dimensional matrix can be obtained, which can provide guidance for subsequent scanning compression.
[0048] Step S6, determining a scanning compression path based on the horizontal distribution degree and the vertical distribution degree of the two-dimensional matrix; compressing the two-dimensional matrix based on the scanning compression path, and distributively storing the compressed data.
[0049] Furthermore, in step S5, the overall horizontal distribution degree and vertical distribution degree of the two-dimensional matrix are obtained, which can serve as a guide for the scanning path during compression; specifically, when the horizontal distribution degree is greater than or equal to the vertical distribution degree, it means that the overall layout of the two-dimensional matrix of the data after regional growth is a horizontal layout, and then a scanning compression path of horizontal scanning is adopted; when the horizontal distribution degree is less than the vertical distribution degree, it means that the overall layout of the two-dimensional matrix of the data after regional growth is a vertical layout, and then a scanning compression path of longitudinal scanning is adopted.
[0050] Furthermore, the two-dimensional matrix is compressed based on the scanning compression path, and the compressed data is distributedly stored. Specifically, when the scanning compression path is horizontal scanning, the upper left corner of the two-dimensional matrix is used as the starting point, and the two-dimensional matrix is traversed by S-shaped horizontal scanning to obtain a compression sequence; when the scanning compression path is vertical scanning, the upper left corner of the two-dimensional matrix is used as the starting point, and the two-dimensional matrix is traversed by S-shaped vertical scanning to obtain a compression sequence. Thus, the production data of the functional non-dairy creamer is obtained through the process of data sequence, dimension increase, two-dimensional matrix, dimension reduction, and compression sequence, and a relatively good compression data sequence is obtained.
[0051] The above method is used to obtain the compressed sequence of the collected non-dairy creamer production data, and the run-length encoding algorithm is used to compress it to obtain the compressed data, which is stored in the distributed storage server. At the same time, a compression direction mark is recorded in the compressed sequence head, with horizontal compression as 0 and vertical compression as 1. When decompressing the call data, the compression direction is read first, and then a two-dimensional matrix is constructed. It is decompressed in the reverse direction according to the compression direction to obtain the original data sequence.
[0052] On the other hand, the present embodiment provides a distributed storage system for functional non-dairy creamer production data, including a memory and a processor, wherein the processor executes a computer program stored in the memory to implement the above-mentioned distributed storage method for functional non-dairy creamer production data.
[0053] In summary, the present invention analyzes the characteristics of data distribution, adaptively obtains a better initial seed placement point for regional growth, performs regional growth on the data, and further obtains a better traversal path compression sequence. Through the traversal sequence, the adjacent repetition degree of the data is increased, the encoding effect of run-length encoding on the non-dairy creamer production data is optimized, and the compression storage efficiency is improved.
[0054] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.
Claims
1. A distributed storage method for functional non-dairy creamer production data, characterized in that: The method includes: Obtaining non-dairy creamer production data and preprocessing it to obtain sequence data to be compressed; constructing a histogram using the sequence data to be compressed; The peak points are obtained by using the histogram, and the degree of the suspected target seed point of the peak point is calculated according to the peak values adjacent to the peak point; the suspected target seed point is obtained by using the degree of the suspected target seed point; A two-dimensional matrix is constructed using the sequence data to be compressed; the floating degree of the suspected target seed point is obtained using the data value in the neighborhood of the suspected target seed point in the two-dimensional matrix; The suspected target seed points are screened according to the floating degree to obtain the target seed points; the region is grown based on the target seed points to obtain the growth region; Calculate the horizontal distribution degree and the vertical distribution degree of the two-dimensional matrix according to each growth area and the minimum circumscribed rectangle of each growth area; A scanning compression path is determined based on the horizontal distribution degree and the vertical distribution degree of the two-dimensional matrix; the two-dimensional matrix is compressed based on the scanning compression path, and the compressed data is distributedly stored.
2. A distributed storage method for functional non-dairy creamer production data as claimed in claim 1, characterized in that: The method of constructing a histogram using the sequence data to be compressed includes: The sequence data to be compressed is mapped to a histogram, wherein the horizontal axis of the histogram is the value of the data in the sequence data to be compressed, and the data values are sorted from small to large in the horizontal axis direction; the vertical axis of the histogram is the number of data corresponding to each data value in the sequence data to be compressed.
3. A distributed storage method for functional non-dairy creamer production data as claimed in claim 1, characterized in that: The method of obtaining each peak point by using a histogram includes: Connect the highest points of the histogram with a smooth curve to obtain a coordinate curve; derive the coordinate curve to obtain a derivative curve, and use the derivative curve to obtain all peak points.
4. A distributed storage method for functional non-dairy creamer production data as claimed in claim 1, characterized in that: The calculation formula of the suspected target seed point degree of the peak point is: , Among them, R represents the degree of suspected target seed point of a peak point; represents the peak value of the peak point; N represents the number of data in the sequence data to be compressed; Indicates the vertical axis value corresponding to the i-th horizontal axis value to the right of the peak point. Indicates the vertical axis value corresponding to the j-th horizontal axis value on the left side of the peak point; Indicates the number of horizontal axis values between the right side of the peak point and the trough. Indicates the number of horizontal axis values between the left side of the peak point and the trough; Indicates the distance between the i-th horizontal axis value on the right and the peak point. It represents the distance from the jth horizontal axis value to the left of the peak point to the peak point; exp{} represents an exponential function with a natural constant as the base.
5. A distributed storage method for functional non-dairy creamer production data as claimed in claim 1, characterized in that: The method of obtaining the suspected target seed point by using the suspected target seed point degree includes: The suspected target seed point degrees of all peak points are grouped into a sequence and linearly normalized to obtain a suspected sequence; half of the data with high suspected target seed point degrees in the suspected sequence are obtained, and the corresponding peak points are the suspected peak points; the data in all the sequence data to be compressed corresponding to the suspected peak points are the suspected target seed points.
6. A distributed storage method for functional non-dairy creamer production data as claimed in claim 1, characterized in that: The calculation formula for the floating degree of the suspected target seed point is: , Among them, D represents the floating degree of a suspected target seed point; M represents the number of data in the 8-neighborhood of a suspected target seed point; Indicates the number of data with the same value as the suspected target seed point in the neighborhood of the suspected target seed point; Represents the data value of the suspected target seed point, represents the mth data value in the neighborhood of the suspected target seed point; exp{} represents an exponential function with a natural constant as the base.
7. A distributed storage method for functional non-dairy creamer production data as claimed in claim 1, characterized in that: The calculation formulas for the horizontal distribution degree and the vertical distribution degree of the two-dimensional matrix are respectively: , , in, and They represent the horizontal distribution degree and the vertical distribution degree of the two-dimensional matrix respectively; and represents the horizontal and vertical lengths of the minimum circumscribed rectangle corresponding to the qth growth region; represents the number of data points in the qth growing region; represents the number of data points in the minimum bounding rectangle corresponding to the qth growth region; represents the number of data points that are not growth points in the eight-neighborhood of the bth data point in the qth growth region; represents the hyperbolic tangent function; Q represents the number of growth regions in the two-dimensional matrix; exp{} represents an exponential function with a natural constant as the base.
8. A distributed storage method for functional non-dairy creamer production data as claimed in claim 1, characterized in that: The determining of the scanning compression path based on the horizontal distribution degree and the vertical distribution degree of the two-dimensional matrix includes: When the horizontal distribution degree is greater than or equal to the vertical distribution degree, a scanning compression path of horizontal scanning is adopted; when the horizontal distribution degree is less than the vertical distribution degree, a scanning compression path of longitudinal scanning is adopted.
9. A distributed storage method for functional non-dairy creamer production data as claimed in claim 1, characterized in that: The method of compressing the two-dimensional matrix based on the scanning compression path and distributing the compressed data includes: When the scanning compression path is horizontal scanning, the upper left corner of the two-dimensional matrix is taken as the starting point, and the two-dimensional matrix is traversed by S-shaped horizontal scanning to obtain a compression sequence; when the scanning compression path is vertical scanning, the upper left corner of the two-dimensional matrix is taken as the starting point, and the two-dimensional matrix is traversed by S-shaped vertical scanning to obtain a compression sequence.
10. A distributed storage system for functional non-dairy creamer production data, comprising a memory and a processor, characterized in that: The processor executes the computer program stored in the memory to implement a distributed storage method for functional non-dairy creamer production data as described in any one of claims 1 to 9.
