An image feature encoding method, apparatus, device and medium

Abstract

The present invention provides an image feature encoding method, including: performing a first scan with a pixel point in an unscanned part of a preset scan area in the image as a scan origin according to a preset scan template, and sequentially storing the position information of all pixel points in this scan according to the scan order; sequentially obtaining the values of preset features of the pixel points at the positions according to the position information of the pixel points in this scan stored above and forming a time series; encoding the values of the elements in the time series according to the change direction between the values of any two adjacent elements in the time series and the comparison relationship between the change of the values of any two adjacent elements in the time series and a preset element change threshold; repeating the above steps until there is no unscanned part in the preset scan area. It also includes related devices, equipment and storage media. It has the functions of accurately extracting the features of any edge pixel point target and storing the pixel point positions.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision, and in particular relates to an image feature encoding method, apparatus, computer device, and computer-readable storage medium. Background Art

[0002] The background information provided in this part related to the present invention may not all be prior art, and there may be content that does not constitute prior art.

[0003] Computer vision is a discipline that studies how to enable a computer to observe and understand objects through vision like the human eye. It attempts to establish an artificial intelligence system that can obtain information from multi-dimensional data such as images.

[0004] Computer vision includes technologies such as image classification, object detection, object tracking, semantic segmentation, instance segmentation, and image reconstruction.

[0005] Image description includes first extracting features and then describing the extracted features. It is an underlying technology required by many of the above-mentioned computer vision technologies.

[0006] Algorithms for implementing the above image description include the HOG (Histogram of Oriented Gradient) algorithm, the SIFT (Scale Invariant Feature Transform) algorithm, etc.

[0007] The above algorithms for image description have advantages and problems in practical applications. For example, since the HOG algorithm operates on local grid cells of an image, it can maintain good invariance to geometric and optical deformations of the image, but it does not have scale invariance and rotation invariance, having certain limitations. The SIFT algorithm realizes finding key points on a picture and extracting neighborhood information of the key points. In this way, when extracting features, it only focuses on stable key points and the information nearby, making the features more descriptive, but it cannot accurately extract features for targets with smooth edges. Summary of the Invention

[0008] The purpose of the present invention is to propose an image feature encoding method that extracts features based on pixel points, so as to accurately extract the features of any edge pixel point target. At the same time, it has the function of storing the position information of the scanned pixel points and can completely restore the image according to the stored position information.

[0009] Based on the above purpose, the present invention proposes an image feature encoding method, including:

[0010] According to a preset scanning template, a pixel point in the unscanned part of the preset scanning area in the image is used as the scanning origin for a scan, and the position information of all the pixel points in this scan is stored in sequence according to the scanning order;

[0011] According to the position information of the pixel points in this scan stored above, the values of the preset features of the pixel points at these positions are obtained in sequence and form a time series;

[0012] According to the change direction between the values of any two adjacent elements in this time series, and the comparison relationship between the change of the values of any two adjacent elements in this time series and a preset element change threshold, the values of the elements in this time series are encoded;

[0013] Repeat the above steps until there is no unscanned part in the preset scanning area.

[0014] In one embodiment, before the encoding, feature extraction is performed on this time series;

[0015] The feature extraction includes:

[0016] Using this time series as input data, inputting the input data into a memory model to obtain multiple memory sequences for representing the degree of memory, and each memory sequence represents a different degree of memory; and

[0017] Performing a difference operation on all the memory sequences to obtain the final feature of this time series;

[0018] Wherein, the memory model includes a forgetting factor and a memory factor, the forgetting factor and the memory factor are used to represent the degree of memory, and the sum of the values of the forgetting factor and the memory factor in each memory model is equal to 1;

[0019] Wherein, the inputting the input data into the memory model to obtain multiple memory sequences for representing the degree of memory includes: inputting the input data into multiple memory models with different degrees of memory representation respectively to obtain corresponding multiple memory sequences with different degrees of memory representation, and the number of memory sequences obtained through the memory model is at least 3;

[0020] Wherein, the time series used during the encoding is the final feature of this time series.

[0021] In one embodiment, the performing a difference operation on all the memory sequences to obtain the final feature of this time series includes:

[0022] Performing a difference operation on any two adjacent ones of all the memory sequences; and

[0023] If the number of memory sequences in the obtained result after the above differential operation is greater than 1, perform the above differential operation step on all memory sequences in the obtained result until the number of memory sequences in the obtained result is 1 and then stop. At this time, the only one of the memory sequences in the obtained result is the final feature;

[0024] Wherein, the final feature is a statistical feature.

[0025] In one embodiment, the encoding of the values of the elements in the time series includes:

[0026] If the absolute value of the change value of two adjacent elements in the time series is less than or equal to the preset element change threshold, encode it as the first preset constant;

[0027] If the absolute value of the change value of two adjacent elements in the time series is greater than the preset element change threshold and the value of the former element is less than the value of the latter element, encode it as the second preset constant;

[0028] If the absolute value of the change value of two adjacent elements in the time series is greater than the preset element change threshold and the value of the former element is greater than the value of the latter element, encode it as the third preset constant.

[0029] In one embodiment, the first preset constant, the second preset constant and the third preset constant are all numerical constants;

[0030] Form a pulse signal according to the set of this encoding.

[0031] In one embodiment, the first preset constant is the numerical value 0;

[0032] The second preset constant and the third preset constant are positive and negative values respectively, and their absolute values are the same.

[0033] In one embodiment, the scanning template is based on the working principle of the human perception system, and generates a spiral scanning template within the range of ω with θ as the angular step and r as the radius step.

[0034] Based on the above object, the present invention also proposes an image feature encoding device, including:

[0035] A pixel point scanning unit, according to a preset scanning template, performs a scan with a pixel point in the unscanned part of the preset scanning area in the image as the scanning origin, and stores the position information of the pixel points of this scan in sequence according to the scanning order;

[0036] The pixel feature value time series generation unit sequentially obtains the values of preset features of the pixel at the position according to the position information of the pixel in the current scan stored above and forms a time series;

[0037] The time series element value encoding unit encodes the values of the elements in the time series according to the change direction between the values of any two adjacent elements in the time series and the comparison relationship between the change of the values of any two adjacent elements in the time series and a preset element change threshold.

[0038] In one embodiment, the device further includes:

[0039] The time series element value extraction unit performs feature extraction on the time series before the encoding, and the feature extraction includes:

[0040] Taking the time series as input data, inputting the input data into a memory model to obtain a plurality of memory sequences for representing the memory degree, and each memory sequence represents a different memory degree; and

[0041] Performing a difference operation on all the memory sequences to obtain the final feature of the time series;

[0042] Wherein, the memory model includes a forgetting factor and a memory factor, the forgetting factor and the memory factor are used to represent the memory degree, and the sum of the values of the forgetting factor and the memory factor in each memory model is equal to 1;

[0043] Wherein, the step of inputting the input data into the memory model to obtain a plurality of memory sequences for representing the memory degree includes: inputting the input data into a plurality of memory models representing different memory degrees respectively to obtain corresponding a plurality of memory sequences representing different memory degrees, and the number of memory sequences obtained through the memory model is at least 3;

[0044] Wherein, the time series used during the encoding is the final feature of the time series.

[0045] Based on the above object, the present invention also provides a computer device, including a memory and a processor, the memory stores a computer program, and it is characterized in that when the processor executes the computer program, the steps of the method described in any one of the above are implemented.

[0046] Based on the above object, the present invention also provides a computer-readable storage medium, on which a computer program is stored, and it is characterized in that when the computer program is executed by a processor, the steps of the method described in any one of the above claims are implemented.

[0047] Additional aspects and advantages of the present invention will be given in the following description, some of which will become apparent from the following description, or can be learned through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0049] Figure 1 It is a flowchart of an implementation manner of the image feature encoding method in the present invention.

[0050] Figure 2 It is a block diagram of an implementation manner of the image feature encoding device in the present invention.

[0051] Figure 3 It is a schematic diagram of a spiral template used in an implementation manner of the image feature encoding method according to the present invention.

[0052] Figure 4 It is a schematic diagram of an image to be scanned in an implementation manner of the image feature encoding method according to the present invention.

[0053] Figure 5 It is a schematic diagram of the neighborhood structure of horizontal edge feature points in an implementation manner of the image feature encoding method according to the present invention.

[0054] Figure 6 It is a schematic diagram of a pulse signal generated after feature extraction of a time series in another implementation manner of an image feature encoding method according to the present invention.

[0055] Figure 7 It is a flowchart of an implementation manner of a time series feature extraction method of the present invention.

[0056] Figure 8 It is a rectangular coordinate diagram of a time series data.

[0057] Figure 9 It is for extracting only by using the first-order difference operation without using the feature extraction method in the present invention Figure 8 the features of the time series in.

[0058] Figure 10 It is for extracting by using the extraction method described in Embodiment 1 of the present invention Figure 8 the features of the time series in.

[0059] Figure 11 The schematic diagram of differential operation in an embodiment of an extraction method of the present invention when the number of generated memory sequences is 3.

[0060] Figure 12 The schematic diagram of differential operation in an embodiment of an extraction method of the present invention when the number of generated memory sequences is 4.

[0061] The accompanying drawings are for illustrative purposes only and are not intended to be drawn to scale. In the drawings, the same reference signs are used to indicate the same elements. For the sake of brevity, not every component is labeled in each drawing. Detailed Embodiments

[0062] The present invention will be described below with reference to several examples. It should be understood that these embodiments are described to enable those of ordinary skill in the art to better understand and implement the present invention, without indicating or implying any limitation to the scope of the present invention.

[0063] As an algorithm in image description, namely the SIFT algorithm, it can be decomposed into the following four steps:

[0064] The first step: Scale-space extreme detection: Search for image positions at all scales. Identify potential interest points that are invariant to scale and rotation through the Gaussian differential function.

[0065] The second step: Keypoint localization: At each candidate position, determine the position and scale by fitting a refined model. The selection of keypoints is based on their stability.

[0066] The third step: Orientation determination: Based on the local gradient direction of the image, assign one or more orientations to each keypoint position. All subsequent operations on the image data are transformed relative to the orientation, scale, and position of the keypoints, thus providing invariance to these transformations.

[0067] The fourth step: Keypoint description: Measure the local gradient of the image within the neighborhood around each keypoint at the selected scale. These gradients are transformed into a representation that allows for comparison of large local shape deformations and illumination changes.

[0068] However, the SIFT algorithm cannot accurately extract features for targets with smooth edges and has a large amount of computation. In addition, in existing image description algorithms, when describing image features, the relative position information between image features is lost, resulting in the inability to recover the image relatively completely according to memory.

[0069] For this reason, as Figure 1As shown, an implementation of an image feature encoding method in the present invention includes steps S210, S220, S230, and S240 as follows.

[0070] Step S210: According to a preset scanning template, use a pixel point in the unscanned part of a preset scanning area in the image as the scanning origin for a single scan, and sequentially store the position information of all the pixel points in this scan according to the scanning order.

[0071] It should be noted that an image can also refer to a frame in a video.

[0072] It should be noted that the usage method of this scanning template is to place the scanning template in the set scanning area of the target image, start scanning from a pixel point (as the scanning origin) in the target image corresponding to the starting point of the scanning template, and sequentially scan the pixel points on the target image corresponding to the scanning template along the set scanning path. The structure of this scanning template can be set according to needs, and there are no relevant limitations.

[0073] It should be noted that the preset scanning area can usually be set to the entire target image, but can also be set to a part of the target image according to needs.

[0074] It should be noted that in an electronic device, the data structure for storing the position information of the pixel points can be an array. Of course, according to needs, it can also be other data structures, such as setting corresponding multiple variables and storing the position information of each pixel point in a corresponding variable.

[0075] Step S220: Sequentially obtain the values of the preset features of the pixel points at the positions according to the stored position information of the pixel points in this scan and form a time series.

[0076] It should be noted that a time series is an important type of high-dimensional data, which is a sequence of data points arranged in the order of time occurrence, and has wide applications in the fields of economic management and engineering. By using time series data mining, useful information related to time contained in the data can be obtained to achieve knowledge extraction.

[0077] Step S230: Encode the values of the elements in this time series according to the change direction between the values of any two adjacent elements in the time series and the comparison relationship between the change of the values of any two adjacent elements in the time series and a preset element change threshold.

[0078] It should be noted that the setting of the preset element change threshold in this step can filter out minor changes and retain the position information of pixel points with significant changes.

[0079] Specifically, the encoding rule in step S230 is as follows:

[0080] If the absolute value of the change value of two adjacent elements in the time series is less than or equal to the preset element change threshold, it is encoded as a first preset constant;

[0081] If the absolute value of the change value of two adjacent elements in the time series is greater than the preset element change threshold, and the value of the former element is less than the value of the latter element, it is encoded as a second preset constant;

[0082] If the absolute value of the change value of two adjacent elements in the time series is greater than the preset element change threshold, and the value of the former element is greater than the value of the latter element, it is encoded as a third preset constant.

[0083] It should be noted that the preset element change threshold can be set according to the overall situation of the absolute values of the change values of all two adjacent elements in the time series.

[0084] It should be noted that the first preset constant, the second preset constant, and the third preset constant can be numerical values, characters, or strings.

[0085] It should be noted that after being encoded by the above encoding rule, the encoding set for this scan is a set composed of the first preset constant, the second preset constant, and the third preset constant.

[0086] Step S240: Repeat the above steps until there is no unscanned part in the preset scan area.

[0087] It should be noted that in this embodiment, feature extraction is based on pixel points, so the features of any edge pixel point target can be accurately extracted. In addition, while encoding, the relative position information between these pixel points can be retained.

[0088] In addition, it should be noted that the pixel domain within the above scan template can be regarded as a cell (for example, a 7*7 cell), and every few cells form a block (for example, 2*2 cells = 1 block). Each cell is encoded using the above encoding method, that is, a description of each cell is formed. The feature encodings of all cells within a block are concatenated to obtain the structural feature encoding of the block. The blocks composed of cells are scanned and encoded in the target image with a certain step size, and finally, the structural feature encodings of all blocks within the image image are concatenated to obtain the structural feature encoding of the image. The finally obtained structural feature vector can be used for further image processing tasks.

[0089] To more fully understand the content of this encoding method, the following embodiment is specifically listed.

[0090] Embodiment 1

[0091] Step 1: Based on the principle that the human visual system obtains visible information through visual perception organs and forms cognition through encoding, and the working principle of the human perception system is determined by the relative judgment of the observed things. As Figure 3 shown, a spiral scanning template within the range of ω is generated by taking integer processing with θ as the angular step size and r as the radius step size, enabling it to simultaneously focus on the target pixel point and other pixel points within its neighborhood. Use the spiral scanning template to scan and judge the pixel points of the target image. Before scanning, let the currently scanned pixel point be located at the center of the spiral template, and sequentially store the index coordinates (i.e., position information) of the pixel points passed by the spiral line into an array (one of the storage structures).

[0092] It should be noted that this spiral scanning template is constructed based on the principle of human eye saccade to obtain information. The angular step size θ, radius step size r, and spiral range ω can all be changed accordingly according to the actual situation and are not limited.

[0093] Step 2: According to the index coordinates of the pixel points stored in the above array, read the gray values of all pixel points within the corresponding spiral scanning range (here, gray scale is one of the preset features) and store them as a time series that saves the gray values of all pixel points within the current template.

[0094] It should be noted that a pulse signal can be generated according to this time series, and the following preset element change threshold can be set according to the amplitude of this pulse signal.

[0095] Step 3: Imitate the retina nerve of the human eye, let each pixel work independently, and perform encoding by perceiving the change of gray values between pixels.

[0096] The specific encoding rule is: when the absolute value of the difference between the former and the latter of two adjacent elements in this time series is greater than the preset element change threshold, and the former of the two adjacent elements in this time series is less than the latter, it is encoded as 1 (i.e., the second preset constant, which represents the jump from low gray value to high gray value here); when the absolute value of the difference between the former and the latter of two adjacent elements in this time series is greater than the preset element change threshold, and the former of the two adjacent elements in this time series is greater than the latter, it is encoded as -1 (i.e., the third preset constant, which represents the jump from high gray value to low gray value here); when the absolute value of the difference between the former and the latter of two adjacent elements in this time series is less than or equal to the preset element change threshold, it is encoded as 0 (i.e., the first preset constant, which represents that the gray value remains unchanged or changes within an approximate range).

[0097] The encoding set obtained for this scan consists of -1, 0, and 1.

[0098] In another embodiment of an image feature encoding method in the present invention, before the step S230, it further includes the steps of Figure 7 feature extraction from this time series as shown, including:

[0099] Step S110: Using the time series as input data, inputting the input data into a memory model to obtain multiple memory sequences for characterizing the degree of memory, and each memory sequence represents a different degree of memory; and

[0100] Step S120: Performing a difference operation on all the memory sequences to obtain the final feature of this time series;

[0101] Among them, the memory model includes a forgetting factor and a memory factor, and the forgetting factor and the memory factor are used to characterize the degree of memory. The sum of the values of the forgetting factor and the memory factor in each memory model is equal to 1;

[0102] Among them, inputting the input data into the memory model to obtain multiple memory sequences for characterizing the degree of memory includes: inputting the input data into multiple memory models with different degrees of memory representation respectively to obtain corresponding multiple memory sequences with different degrees of memory representation, and the number of memory sequences obtained through the memory model is at least 3;

[0103] Among them, the time series used during encoding is the final feature of this time series.

[0104] It should be noted that the above memory model can be set to include different parameter forms according to specific needs. In an embodiment of this feature extraction method, the memory model includes a forgetting factor and a memory factor, and the forgetting factor and the memory factor are used to characterize the degree of memory.

[0105] In addition, it should be noted that the number of memory sequences generated in the above steps S110 and S120 can also be set according to specific needs, but it is at least 3 to make it possible to perform second-order and higher-order difference operations.

[0106] Specifically, performing a difference operation on all the memory sequences to obtain the final feature of this time series includes:

[0107] Performing a difference operation on any two adjacent ones among all the memory sequences; and

[0108] If the number of memory sequences in the obtained result is greater than 1 after the above differential operation, then perform the above one differential operation step on all the memory sequences in the obtained result until the number of memory sequences in the obtained result is 1 and then stop. At this time, the only one memory sequence in the obtained result is the final feature;

[0109] Among them, the final feature is a statistical feature.

[0110] It should be noted that the method for feature extraction of this time series can detect and extract the mutation position of the time series. Inspired by the above, based on the principle of obtaining information by human eye saccades, a scanning template is constructed to convert the change situation of pixel feature (such as grayscale, and other features of pixels can also be set according to needs) values into a sequence, and the features are described in an encoded manner, which can retain the relative position information between features and can accurately extract the features of edge-smooth targets.

[0111] It should be noted that for the analysis method of time series, the first step of analysis is to extract the deterministic information (such as trend information, seasonal cycle information, cyclic change information, etc.) contained in the sequence. There are many methods for extracting deterministic information. For example, constructing seasonal indices, fitting long-term trend models, moving averages, exponential smoothing and many other methods are all methods for extracting deterministic information. However, their extraction of deterministic information is not sufficient enough. There is a relatively high probability of a large loss of features after extraction.

[0112] Therefore, in another implementation manner of the present invention, in the method for feature extraction of this time series, by introducing a memory model with a standard memory degree and a differential algorithm, the loss during feature extraction is reduced to improve the efficiency of extracting deterministic information.

[0113] The above memory model can generate different memory sequences according to different memory degrees of the input time series data, and then perform a differential algorithm on the memory sequences, so as to avoid excessive loss of information when extracting feature information, be able to better conform to the data change rate of the original time series, improve the accuracy, and can avoid the influence of noise.

[0114] The above differential algorithm (also known as the difference function or difference operation) refers to the discrete operation corresponding to the difference, which is the operation using the difference. The result of the difference reflects a change between discrete quantities and is a tool for studying discrete mathematics. In social and economic activities and natural science research, we often encounter variables related to time t, and people can often only observe or record the values of these variables at discrete times t. For such variables, to study their mutual relationships, the tools of difference and difference equations are indispensable. The tools of differential and differential equations in calculus actually originate from difference and difference equations. Therefore, difference and difference equations are more primitive, objective, and vivid materials. One function of the above differential algorithm is to reduce the irregular fluctuations between data and make its fluctuation curve smoother. The deterministic information is extracted by using the autoregressive method. Moreover, the differential algorithm is easy to operate, and the resources consumed when implemented by a computer are relatively small.

[0115] In addition, by introducing multiple memory levels, each memory level represents the memory level of the current information and past information of the original time series. Then, by performing multi-level difference operations on the memory sequences generated by different memory levels, excessive loss of information can be avoided, the data change rate of the original time series can be well matched, the accuracy can be improved, and the influence of noise can be avoided.

[0116] To more fully understand the content of the above steps for feature extraction of the time series, the following two embodiments are specifically listed, namely Embodiment 2 and Embodiment 3.

[0117] Embodiment 2

[0118] Step 1: Extract the first number of the original time series list and store it in the first storage positions of L1, L2, and L3 respectively.

[0119] It should be noted that in this example, list is the input data for inputting the memory model, and the number of generated memory sequences is 3, namely L1, L2, and L3. In a computer, list, L1, L2, and L3 here can all be of array type and can be arrays with the same dimension and size.

[0120] Step 2: Set parameters f1 and f2 to adjust the fineness of the extracted edge information or feature points. According to different memory levels, set different forgetting factors fgr j and memory factors mem j For j = 1, 2, 3, fgr1 = f1 - f2, mem1 = 1 - fgr1; fgr2 = f1, mem2 = 1 - fgr2; fgr3 = f1 + f2, mem3 = 1 - fgr3.

[0121] It should be noted that in the second embodiment, the forgetting factor fgr is introduced into the memory model. j (used to represent the degree of forgetting of information) and the memory factor mem j (used to represent the degree of memory of information), and three pairs of memory models with different forgetting factors fgr j and memory factors mem j are set. That is, memory model A includes fgr1 and mem1, memory model B includes fgr2 and mem2, and memory model C includes fgr3 and mem3.

[0122] It should be noted that for each pair of forgetting factors fgr j and memory factors mem j , j = 1, 2, 3

[0123] Step 3: Traverse the length k of the time series list, and calculate the memory and forgetting degrees of the current information and past information according to the following formula to finally generate the memory sequence L j . L j [i] = mem j *L[i - 1] + fgr j *list[i], j = 1, 2, 3; i = 1, 2, 3...k.

[0124] It should be noted that each of the memory models also includes the above calculation formula: L j [i] = mem j *L[i - 1] + fgr j *list[i], j = 1, 2, 3; i = 2, 3,...k. The memory sequence L j , j = 1, 2, 3, represents L1, L2, L3 mentioned above, and is responsible for assigning values to the memory sequences L1, L2, L3 respectively in this step. Its input data is the time series list. The value assigned to L1 is the value obtained after substituting the input data list into memory model A; the value assigned to L2 is the value obtained after substituting the input data list into memory model B; the value assigned to L1 is the value obtained after substituting the input data list into memory model A; the value assigned to L3 is the value obtained after substituting the input data list into memory model C.

[0125] It can be based on the forgetting factor fgr j and the memory factor mem jDepending on the set values, the memory models A, B, and C can represent specific different memory degrees. For example, by setting the values of the above parameters f1 and f2, the memory model A here can be set to have good memory for current information and poor memory for past information, while the memory model B here can be set to have average memory for current and past information. The memory model C here can be set to have poor memory for current information and good memory for past information.

[0126] Step 4: From the formula result = (L2 - L1) - (L3 - L2) = 2 * L2 - L1 - L3, the final feature result is obtained.

[0127] It should be noted that as Figure 11 shown, for the above three memory sequences L1, L2, and L3, first, the first-order difference calculation is performed on two adjacent memory sequences L2 and L1, L3 and L2. After the above first-order difference calculation, if the number of remaining memory sequences is greater than 1, the second-order difference calculation can be performed, that is, the difference result between L2 and L1 and the difference result between L3 and L2 are further differentiated. The final obtained sequence is the final feature, that is, the deterministic information to be extracted.

[0128] As Figure 8 shown, a time series is respectively presented, which represents the monthly shampoo sales quantity from 1900 to 1903. There are a total of 36 data points, which are contributed by Makridakis, Wheelwright, and Hyndman (1998). In the figure, the abscissa represents year-month, and the ordinate represents the shampoo sales volume.

[0129] As Figure 9 shown, it is the result of using the existing first-order difference processing (without using the memory model in the present invention). In the figure, the abscissa represents year-month, and the ordinate represents the result after the first-order difference processing. Comparing this figure with Figure 8 it can be clearly seen that the degree of coincidence between the result of using the first-order difference processing (without using the memory model in the present invention) and Figure 8 the data change rate is poor, indicating that a large amount of features extracted by using this first-order difference processing (without using the memory model in the present invention) are lost.

[0130] As Figure 10 shown, it is the result obtained by using the extraction method of the second embodiment. In the figure, the abscissa represents year-month, and the ordinate represents the processing result obtained by using the extraction method of the second embodiment. Comparing this figure with Figure 8 it can be clearly seen that the degree of coincidence between the result in this figure and Figure 8 the data change rate is better than that of Figure 9Much higher than before, indicating that the feature loss extracted by the extraction method of the second embodiment is much smaller than the feature loss extracted by the existing first-order difference processing (without using the memory model in the present invention).

[0131] Embodiment Three

[0132] Step 1: Extract the first number of the original time series list and store it in the first storage position of L j respectively, where j = 1, 2,..., n.

[0133] It should be noted that in this example, list is the input data for the memory model, and the number of generated memory sequences is j. In a computer, both list and L j , where j = 1, 2,..., n can be of array type and can be arrays with the same dimension and size.

[0134] Forgetting factor fgr j and memory factor mem j

[0135] Step 2: Set different forgetting factors fgr j and memory factors mem j , where j = 1, 2,..., n. fgr j = m + f * sin(wt), 0 < fgr j < 1, mem j = 1 - fgr j , 0 < mem j < 1, where m represents the average memory level, f represents the memory fluctuation ratio coefficient, w represents the waveform fluctuation degree, and t represents the time series time.

[0136] It should be noted that the memory model in the third embodiment also introduces a forgetting factor fgr j (used to represent the degree of forgetting of information) and a memory factor mem j (used to represent the degree of memory of information), but in the third embodiment, the forms of the forgetting factor fgr j and the memory factor mem j set are different from those in the second embodiment. Different forgetting factors fgr j and memory factors mem j correspond to different memory models representing different memory levels, where j = 1, 2,..., n.

[0137] Step 3: Traverse the length k of the time series list and calculate the memory and forgetting degrees of the current information and past information according to the following formula. L j[i] = mem j *L[i - 1] + fgr j *list[i], j = 1, 2,..., n; i = 2, 3,..., k.

[0138] It should be noted that each of the above-mentioned memory models further includes the following calculation formula: L j [i] = mem j *L[i - 1] + fgr j *list[i], j = 1, 2,..., n; i = 2, 3,..., k. After substituting the time series list into the memory models with different degrees of memory representation respectively, different memory sequences L j , j = 1, 2,..., n.

[0139] The maximum value of j here can be specifically set as needed. For example, for the case where deterministic information can be extracted only through first-order difference, the maximum value of j can be set to 2 at this time. Another example is that if the extraction of deterministic information cannot be achieved through first-order difference (for example, when the time series list contains a significant change trend, although the first-order difference can make the trend of the time series list stable, it cannot well express the data change trend of the original time series list, resulting in excessive information loss, low estimation accuracy, and the introduction of some noise), in this case, the extraction of deterministic information can be achieved through higher-order difference. At this time, the maximum value of j can be set to 3 (to achieve second-order difference calculation) or larger (to achieve higher-order difference calculation).

[0140] Step Four: According to the difference principle, perform a first-order difference operation on adjacent sequences in the time series L j (j = 1, 2,..., n) obtained in Step 4. If the final feature is not obtained, perform a second-order difference on the result of the first-order difference operation, and so on, until the final feature of the time series, that is, the deterministic information, is obtained.

[0141] It should be noted that in practice, an appropriate difference method can be selected according to the different characteristics of the time series. For example Figure 12 As shown, it is the calculation principle diagram for realizing first-order, second-order, and third-order differences respectively to obtain the final feature of the time series when the maximum value of j is 4.

[0142] In addition, it should be noted that in addition to the above two embodiments (i.e., Embodiment 2 and Embodiment 3), the setting forms of the forgetting factor fgr j and the memory factor mem j in the memory model can also be set in different forms according to specific situations.

[0143] The characteristics of a time series include the statistical characteristics, fitting characteristics, and classification characteristics of the time series. And the above-mentioned statistical characteristics are the basis for analyzing the fitting characteristics and classification characteristics.

[0144] The statistical characteristics of a time series are usually divided into two categories: the time domain and the frequency domain. The characteristics in the time domain can be further divided into dimensional characteristics and dimensionless characteristics. Dimensional characteristics such as the mean, variance, root mean square, peak value, etc., and dimensionless characteristics such as the impulse factor, peak factor, waveform factor, etc.; while the characteristics in the frequency domain include the mean square frequency, root mean square frequency, frequency variance, frequency standard deviation, etc.

[0145] Since when representing signals, dimensional indicators are sensitive to signal characteristics, but they will also change due to changes in working conditions (such as load) and are extremely vulnerable to environmental interference. Using existing methods may not detect all of them. Although dimensionless characteristics can exclude these interferences, the calculation is difficult.

[0146] By using an implementation manner of the extraction method in the present invention, not only can the property of fully extracting deterministic information by using the difference method be well utilized, but also the extraction of dimensional indicators of the time series, such as the extraction of the peak value, can be completed perfectly, thus enabling perfect feature extraction.

[0147] Such as Figures 4 to 6 shown, it is a specific case of an image feature encoding method, which can identify feature points at different positions and in different directions through different pulse coding results. Specifically, it is described as follows.

[0148] Input an image with straight lines in different directions as Figure 4 shown, and use a spiral scanning template as Figure 3 shown to scan this image.

[0149] For horizontal edge feature points, their neighborhood structure in this spiral template is as Figure 5 shown, and the sequence obtained by scanning at this time is:

[0150] [255,0,0,255,255,255,255,0,0,0,0,0,0,255,255,255,255,255,255,255,255,255,0,0,0,0,0,0,0,0,255,255,255,255,255,255,255,255,255,255,255].

[0151] Such as Figure 6 shown, after feature extraction is performed on this time series, a pulse signal diagram is formed.

[0152] Select the preset element change threshold to be 1 / 2 of the maximum amplitude of the pulse signal, and perform encoding according to the encoding method in step 3 of the first embodiment above. Finally, an encoded set is output:

[0153] 10 - 1000100000 - 10000000010000000 - 10000000000 (40 bits). By changing the scanned target pixel point, different encoding results can be obtained. For example:

[0154] 100 - 10001000000 - 100000000100000000 - 10000000 (edge feature points in the 45° direction)

[0155] 0100 - 100001000000 - 1000000000100000000 - 10000 (edge feature points in the vertical direction)

[0156] The feature of this encoding method is that it can accurately extract the structural features of any feature points and simultaneously retain the relative position information between the features (known from the template), and can completely restore the image according to the memory.

[0157] Such as Figure 2 shown, is an embodiment of the image feature encoding device of the present invention, including a pixel point scanning unit 210, a pixel point feature value time series generating unit 220, and a time series element value encoding unit 230.

[0158] The pixel point scanning unit 210 performs a single scan with a pixel point in the unscanned part of the preset scanning area in the image as the scanning origin according to the preset scanning template, and sequentially stores the position information of the pixel points in this scan according to the scanning order;

[0159] The pixel point feature value time series generating unit 220 sequentially obtains the values of the preset features of the pixel points at the positions according to the stored position information of the pixel points in this scan and forms a time series;

[0160] The time series element value encoding unit 230 encodes the values of the elements in the time series according to the change direction between the values of any two adjacent elements in the time series and the comparison relationship between the change of the values of any two adjacent elements in the time series and the preset element change threshold.

[0161] Another embodiment of the image feature encoding device of the present invention further includes:

[0162] A time series element value extraction unit, before the encoding, performs feature extraction on the time series, and the feature extraction includes:

[0163] Using the time series as input data, inputting the input data into a memory model to obtain a plurality of memory sequences for characterizing the degree of memory, and the degree of memory characterized by each memory sequence is different; and

[0164] Performing a difference operation on all the memory sequences to obtain the final feature of the time series;

[0165] Wherein, the memory model includes a forgetting factor and a memory factor, the forgetting factor and the memory factor are used to characterize the degree of memory, and the sum of the value of the forgetting factor and the value of the memory factor in each memory model is equal to 1;

[0166] Wherein, the inputting the input data into a memory model to obtain a plurality of memory sequences for characterizing the degree of memory includes: inputting the input data into a plurality of memory models with different degrees of memory respectively to obtain corresponding memory sequences with different degrees of memory, and the number of memory sequences obtained by the memory model is at least 3;

[0167] Wherein, the time series used during encoding is the final feature of the time series.

[0168] The present invention also provides a computer device, including a memory and a processor, the memory stores a computer program, and is characterized in that when the processor executes the computer program, the steps of the above encoding method are implemented.

[0169] The present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above encoding method are implemented.

[0170] The present invention also provides a computer program product, including a computer program, and when the computer program is executed by a processor, the steps of the above encoding method are implemented.

[0171] Generally, the described computing system and / or device and / or equipment can use any number of computer operating systems, including but not limited to the following versions and / or variants: application programs, application link / intelligent device link middleware, Microsoft operating system, Unix operating system, Linux operating system, Mac OSX, iOS operating system, BlackBerry OS, Android operating system, and the automotive platform for infotainment provided by QNX Software Systems. Examples of computing devices include, but are not limited to, in-vehicle vehicle computers, computer workstations, servers, desktops, laptops, portable computers, or handheld computers, or some other computing systems and / or devices.

[0172] Computing devices typically include computer-executable instructions, which can be executed by one or more computing devices such as those listed above. The computer-executable instructions can be compiled or interpreted from a computer program, which is established using a variety of programming languages and / or technologies, including but not limited to a single one or in combination of JavaTM, C, C++, Visual Basic, Java Script, Perl, etc. Some of these applications can be compiled and executed on a virtual machine, such as a Java virtual machine, a Dalvik virtual machine, and so on. Typically, a processor (e.g., a microprocessor) receives instructions, for example, from a memory, a computer-readable medium, etc., and executes these instructions to perform one or more processes, including one or more of the processes described herein. Such instructions and other data can be stored and transmitted using a variety of computer-readable media.

[0173] A computer-readable storage medium (also referred to as a processor-readable medium) includes any non-transitory (e.g., tangible) medium that participates in providing data (e.g., instructions) that are readable by a computer (e.g., via a processor of the computer). Such a medium can take many forms, including but not limited to non-volatile media and volatile media. Non-volatile media can include, for example, optical discs or magnetic disks and other persistent memories. Volatile media can include, for example, dynamic random access memory (DRAM), which typically constitutes the main memory. Such instructions can be transmitted through one or more transmission media, including coaxial cables, copper wires, and optical fibers, including the wires that comprise a system bus connecting to a processor of a computer. Common forms of computer-readable media include, for example, a floppy disk, a flexible disk, a hard disk, a magnetic tape, any other magnetic medium, a CD-ROM (Compact Disc Read-Only Memory), a DVD (Digital Versatile Disc), any other optical medium, a punched card, a paper tape, any other physical medium with a pattern of holes, a RAM (Random Access Memory), a PROM (Programmable Read-Only Memory), an EPROM (Electrically Programmable Read-Only Memory), a FLASH-EEPROM (Flash Electrically Erasable Programmable Read-Only Memory), any other memory chip or cartridge, or any other computer-readable medium.

[0174] A database, a data repository, or other data storage described herein may include various types of mechanisms for storing, accessing, and retrieving various data, including hierarchical databases, sets of files in a file system, application databases in a proprietary format, relational database management systems (RDBMSs), and the like. Each such data storage is typically included within a computing device using a computer operating system such as one of those mentioned above, and is accessed over a network in any one or more of various ways. A file system can be accessed from a computer operating system and can include files stored in various formats. An RDBMS typically uses the Structured Query Language (SQL), such as the Procedural SQL (PL / SQL) language mentioned above, in addition to a language for creating, storing, editing, and executing stored procedures.

[0175] In some examples, system components may be implemented as computer-readable instructions (e.g., software) on one or more computing devices (e.g., servers, personal computers, etc.) and stored on a computer-readable medium associated therewith (such as a disk, memory, etc.). A computer program product may include such instructions stored on a computer-readable medium for performing the functions described herein.

[0176] In the claims, the word "comprising" does not exclude other elements or steps; the words "a" or "an" do not exclude a plurality. In the claims, the use of ordinal numbers such as "first", "second", etc. to modify a claim element itself does not mean that one claim element has a higher priority, order, or chronological order of action execution than another claim element, but is merely for the purpose of differentiating one claim element from another. Although certain specific technical features are separately recited in different dependent claims, this does not mean that these specific technical features cannot be combined and utilized. The various aspects of the present invention can be used alone, in combination, or in various arrangements not specifically discussed in the foregoing embodiments, and thus its application is not limited to the details and arrangements of the components described above or shown in the drawings. For example, the various aspects described in one embodiment can be combined with the various aspects described in other embodiments in any manner. The steps, functions, or features recited in multiple modules or units can be performed or satisfied by one module or one unit. The steps of the methods disclosed herein are not limited to being executed in any particular order, and it is possible to execute some or all of the steps in other orders. Any reference signs in the claims should not be construed as limiting the scope of the claims.

[0177] Although the present invention has been described by way of the accompanying drawings and embodiments, such description and illustration should be considered illustrative or exemplary rather than restrictive. Those of ordinary skill in the art should realize that various improvements, additions, and substitutions are possible without departing from the scope and spirit of the present invention disclosed by the appended claims.

Claims

1. An image feature encoding method, characterized in that, The method includes: Performing a first scan with a pixel point in an unscanned part of a preset scan area in the image as the scan origin according to a preset scan template, and sequentially storing the position information of all the pixel points in this scan according to the scan order; Sequentially obtaining the values of preset features of the pixel points at the positions according to the position information of the pixel points in this scan stored above and forming a time series; Encoding the values of the elements in the time series according to the change direction between the values of any two adjacent elements in the time series and the comparison relationship between the change of the values of any two adjacent elements in the time series and a preset element change threshold; Repeating the above steps until there is no unscanned part in the preset scan area; Wherein, before the encoding, feature extraction is performed on the time series; The feature extraction includes: Using the time series as input data, inputting the input data into a memory model to obtain multiple memory sequences for characterizing the degree of memory, and the degree of memory characterized by each memory sequence is different; and Performing a difference operation on all the memory sequences to obtain the final feature of the time series; Wherein, the memory model includes a forgetting factor and a memory factor, the forgetting factor and the memory factor are used to characterize the degree of memory, and the sum of the value of the forgetting factor and the value of the memory factor in each memory model is equal to 1; Wherein, the inputting the input data into the memory model to obtain multiple memory sequences for characterizing the degree of memory includes: inputting the input data into multiple memory models with different degrees of memory characterization respectively to obtain corresponding multiple memory sequences with different degrees of memory characterization, and the number of memory sequences obtained through the memory model is at least 3; Wherein, the time series used during the encoding is the final feature of the time series; Wherein, the encoding of the values of the elements in the time series includes: If the absolute value of the change value of two adjacent elements in the time series is less than or equal to the preset element change threshold, it is encoded as a first preset constant; If the absolute value of the change value of two adjacent elements in the time series is greater than the preset element change threshold, and the value of the former element is less than the value of the latter element, it is encoded as a second preset constant; If the absolute value of the change value of two adjacent elements in the time series is greater than the preset element change threshold, and the value of the former element is greater than the value of the latter element, it is encoded as a third preset constant.

2. The image feature encoding method according to claim 1, wherein The performing a difference operation on all the memory sequences to obtain the final feature of the time series includes: Performing a difference operation on any two adjacent ones of all the memory sequences; and If after the above difference operation, the number of memory sequences in the obtained result is greater than 1, then perform the above difference operation step on all the memory sequences in the obtained result until the number of memory sequences in the obtained result is 1 and stop, and at this time, the only one memory sequence in the obtained result is the final feature; Wherein, the final feature is a statistical feature.

3. The image feature encoding method according to any one of claims 1 to 2, characterized in that: The first preset constant, the second preset constant, and the third preset constant are all numerical constants; A pulse signal is formed according to the set of this encoding.

4. The image feature encoding method according to claim 3, characterized in that: The first preset constant is the numerical value 0; The second preset constant and the third preset constant are positive and negative values respectively, and their absolute values are the same.

5. The image feature encoding method according to any one of claims 1 to 2, characterized in that: The scanning template is based on the working principle of the human perception system, with θ as the angular step and r as the radius step, to generate a spiral scanning template within the range of ω.

6. An image feature encoding device, characterized in that The device includes: A pixel point scanning unit, which performs a single scan with a pixel point in the unscanned part of the preset scanning area in the image as the scanning origin according to the preset scanning template, and stores the position information of the pixel points in this scan in sequence according to the scanning order; A pixel point feature value time series generation unit, which sequentially obtains the values of the preset features of the pixel points at the positions according to the stored position information of the pixel points in this scan and forms a time series; A time series element value encoding unit, which encodes the values of the elements in this time series according to the change direction between the values of any two adjacent elements in this time series and the comparison relationship between the change of the values of any two adjacent elements in this time series and the preset element change threshold; A time series element value extraction unit, which performs feature extraction on this time series before the encoding, and the feature extraction includes: Using this time series as input data, inputting the input data into a memory model to obtain multiple memory sequences for representing the degree of memory, and the degree of memory represented by each memory sequence is different; and Performing a difference operation on all the memory sequences to obtain the final feature of this time series; Wherein, the memory model includes a forgetting factor and a memory factor, the forgetting factor and the memory factor are used to represent the degree of memory, and the sum of the values of the forgetting factor and the memory factor in each memory model is equal to 1; Wherein, the step of inputting the input data into the memory model to obtain multiple memory sequences for representing the degree of memory includes: inputting the input data into multiple memory models with different degrees of memory representation respectively to obtain corresponding multiple memory sequences with different degrees of memory representation, and the number of memory sequences obtained through the memory model is at least 3; Wherein, the time series used during the encoding is the final feature of this time series; Wherein, the encoding of the values of the elements in this time series includes: If the absolute value of the change value between two adjacent elements in this time series is less than or equal to the preset element change threshold, it is encoded as the first preset constant; If the absolute value of the change value between two adjacent elements in this time series is greater than the preset element change threshold, and the value of the former element is less than the value of the latter element, it is encoded as the second preset constant; If the absolute value of the change value of two adjacent elements in the time series is greater than the preset element change threshold, and the value of the former element is greater than the value of the latter element, it is encoded as the third preset constant.

7. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method described in any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the method described in any one of claims 1 to 5.

Images