Lossless hierarchical coding of image feature attributes

Abstract

The invention discloses lossless encoding and decoding of attributes of image features. According to some embodiments, an exemplary decoding method may include receiving a data stream, the data stream including a plurality of parameters associated with a plurality of image features, respectively; dividing the plurality of image features into one or more clusters; determining a reference value for at least one of the one or more clusters; and determining respective attributes of the plurality of image features, where the attributes of the image features are determined based on: parameters associated with the respective image features, and reference values for the respective clusters.

Description

Technical Field

[0001] The present disclosure relates generally to image or video processing, and more particularly to methods and systems for representing, encoding, and decoding attributes of image features. Background Art

[0002] With the rapid development of devices and applications that use visual information, the demand for image / video processing and transmission capabilities has exploded. On the one hand, standards such as Advanced Video Coding (AVC), High Efficiency Video Coding (HEVC), and Versatile Video Coding (VVC) have been developed to significantly reduce the storage size and transmission bandwidth of image / video data while maximizing the subjective quality perceived by humans under certain specific bitrate conditions.

[0003] On the other hand, machine vision applications (such as object recognition and tracking, face recognition, image / video search, mobile augmented reality (MAR), autonomous vehicles, Internet of Things (IoT), image matching, 3D structure construction, stereo correspondence, and motion tracking) focus on the analysis and use of features extracted from image / video data rather than subjective quality. Therefore, there is an increasing demand for storing and transmitting data for quantifying and describing the extracted features (i.e., the attributes of the extracted features). To this end, the Moving Picture Experts Group (MPEG) established the Video Coding for Machines (VCM) Working Group to standardize new codecs and tools for encoding and decoding feature attributes (hereinafter also referred to as "feature data" or "feature streams"). Efforts have been made to improve the efficiency of these codecs and tools. Summary of the invention

[0004] Embodiments of the present disclosure relate to methods for representing and encoding (i.e., encoding or decoding) methods of feature attributes. In some embodiments, a computer-implemented method for decoding feature attributes is provided. The method includes: receiving a data stream, the data stream including a plurality of parameters respectively associated with a plurality of image features; dividing the plurality of image features into one or more clusters; determining a reference value for at least one of the one or more clusters; and determining corresponding attributes of the plurality of image features, wherein the attributes of the image features are determined based on: the parameters associated with the corresponding image features, and the reference value of the corresponding cluster.

[0005] In some embodiments, a computer-implemented method for encoding a feature attribute is provided. The method includes: dividing a plurality of image features into one or more clusters; determining a reference value for at least one of the one or more clusters; and generating a plurality of parameters respectively associated with the plurality of image features, wherein each of the plurality of parameters is generated based on: a value of an attribute of a corresponding image feature, and a reference value of a corresponding cluster.

[0006] In some embodiments, a device for decoding feature attributes is provided. The device includes: a memory for storing instructions; and a processor configured to execute the instructions to cause the device to: receive a data stream, the data stream including a plurality of parameters respectively associated with a plurality of image features; divide the plurality of image features into one or more clusters; determine a reference value of at least one of the one or more clusters; and determine corresponding attributes of the plurality of image features, wherein the attributes of the image features are determined based on: the parameters associated with the corresponding image features, and the reference value of the corresponding cluster.

[0007] In some embodiments, a device for encoding feature attributes is provided. The device includes: a memory for storing instructions; and a processor configured to execute the instructions so that the device: divides a plurality of image features into one or more clusters; determines a reference value of at least one of the one or more clusters; and generates a plurality of parameters respectively associated with the plurality of image features, wherein each of the plurality of parameters is generated based on: a value of an attribute of a corresponding image feature, and a reference value of a corresponding cluster.

[0008] Aspects of the disclosed embodiments may include a non-transitory tangible computer-readable medium storing software instructions that, when executed by one or more processors, are configured for and capable of performing and implementing one or more of the methods, operations, etc., in accordance with the disclosed embodiments. In addition, aspects of the disclosed embodiments may be performed by one or more processors configured as one (or more) special-purpose processors based on software instructions that are programmed with logic and instructions that, when executed, perform one or more operations in accordance with the disclosed embodiments.

[0009] Other purposes and advantages of the disclosed embodiments will be set forth in the following description, and in part will be apparent from the description, or may be learned through the practice of the embodiments. The purposes and advantages of the disclosed embodiments may be realized and obtained through the elements and combinations in the claims.

[0010] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosed embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 is a schematic diagram showing an exemplary encoding and decoding system for processing and transmitting image data and feature data according to an embodiment of the present disclosure.

[0012] Figure 2A is a schematic diagram showing an exemplary input image according to an embodiment of the present disclosure.

[0013] Figure 2B is a diagram showing the embodiment of the present disclosure. Figure 2A Schematic diagram of the key points identified on the input image shown.

[0014] Figure 3 is a schematic diagram showing exemplary key points in an image scale space according to an embodiment of the present disclosure.

[0015] Figure 4 is a table for describing exemplary feature vectors of key points extracted from image data according to an embodiment of the present disclosure.

[0016] Figure 5 is a schematic diagram showing clusters of key points according to an embodiment of the present disclosure.

[0017] Figure 6 is a schematic diagram showing a convex hull of a key point set according to an embodiment of the present disclosure.

[0018] Figure 7 is a schematic diagram showing an exemplary center point clustering process according to an embodiment of the present disclosure.

[0019] Figure 8 is a block diagram of an exemplary apparatus for encoding or decoding image feature attributes according to an embodiment of the present disclosure.

[0020] Fig. 9 is a flowchart illustrating an exemplary method for encoding image feature attributes according to an embodiment of the present disclosure.

[0021] Fig.10 According to the embodiment of the present disclosure Fig. 9 Flowchart of an exemplary hierarchical division method used in the encoding method shown in .

[0022] Fig.11 is a flow chart for determining parameters associated with non-center point key points according to an embodiment of the present disclosure.

[0023] Fig.12 is a flow chart illustrating an exemplary method for decoding image feature attributes according to an embodiment of the present disclosure.

[0024] Fig.13 According to the embodiment of the present disclosure Fig.12 Flowchart of an exemplary hierarchical partitioning method used in the decoding method shown in .

[0025] Fig.14 is a flow chart for determining the attribute value of a non-center key point according to an embodiment of the present disclosure.

[0026] Fig.15 is a histogram of scale values ​​of key points extracted from a sample image without using the disclosed method for representing image feature attributes.

[0027] Fig.16 is a histogram of scale values ​​of key points extracted from a sample image by using the disclosed method for representing image feature attributes.

[0028] Fig.17 is a flow chart of an exemplary Scale Invariant Feature Transform (SIFT) algorithm for extracting features from image data according to an embodiment of the present disclosure.

[0029] Fig.18 is a schematic diagram showing an exemplary Jarvis march algorithm according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0030] Reference will now be made in detail to embodiments of the present invention, examples of which are shown in the accompanying drawings. The following description refers to the accompanying drawings, in which the same numbers in different drawings represent the same or similar elements, unless otherwise specified. The implementations set forth in the following description of exemplary embodiments do not represent all implementations according to the present invention. On the contrary, they are merely examples of devices and methods according to aspects related to the present invention as described in the appended claims.

[0031] The present disclosure provides systems and methods for representing and encoding (i.e., encoding or decoding) the attributes of features extracted from image data. Image data may include one image, multiple images, or a video. An image is a static picture (e.g., a frame). Multiple images may be spatially or temporally related or unrelated. A video includes a set of images arranged in chronological order. The features of an image are information used to describe the local attributes of an image. For example, a feature may be a local structure, pattern, texture, or shape in an image. The attributes of a feature are data used to quantify and describe the features extracted from the image data, and are also referred to as "image feature attributes," "feature attributes," or "feature data" in the present disclosure.

[0032] According to the present disclosure, a feature can be any type of region of interest or point of interest extracted from image data, such as a key point, blob, edge, corner, vertex, point, ridge, object, etc. Various types of attributes can be used to describe these features, such as scale, orientation, curvature, peak response of Laplacian of Gaussian (LoG), distance to the center of the image, gradient, etc. A specific embodiment of the encoding and decoding scale of the scale-invariant feature transform (SIFT) key point is described below. However, the feature attribute encoding and decoding system and method provided by the present disclosure is not limited to SIFT key points.

[0033] Furthermore, according to the present disclosure, feature attributes may be represented using any data structure (e.g., parameters, coefficients, data arrays, vectors, matrices, or any other data structure suitable for quantizing features), and feature attributes may be stored or transmitted as a coded data stream, which is also referred to as a "feature stream" or "feature bitstream" in the present disclosure.

[0034] Figure 1 is a block diagram illustrating an encoding and decoding system 100 that may be used to process and transmit image data and feature data according to some disclosed embodiments. Figure 1 As shown, system 100 includes a source device 120 that provides encoded image data and / or feature data to be decoded by a target device 140. According to the disclosed embodiments, each of source device 120 and target device 140 may include any of a variety of devices, including a desktop computer, a notebook (e.g., laptop) computer, a server, a tablet computer, a set-top box, a mobile phone, a vehicle, a camera, an image sensor, a robot, a television, a camera, a wearable device (e.g., a smart watch or a wearable camera), a display device, a digital media player, a video game console, a video streaming device, etc. Source device 120 and target device 140 may be equipped for wireless or wired communication.

[0035] refer to Figure 1, the source device 120 may include an image / video encoder 122, a feature extractor 124, a feature encoder 126, and an output interface 128. According to the disclosed embodiments, the source device 120 may further include various devices (not shown) for providing the original image or video to be processed by the image / video encoder 122, the feature extractor 124, and / or the feature encoder 126. The device for providing the original image or video may include an image / video capture device, such as a camera, an image / video storage device containing previously captured images / videos, or an image / video feed interface for receiving images / videos from an image / video content provider. As another alternative, the source device 120 may generate computer graphics-based data as the source image / video, or a combination of real-time images / videos, archived images / videos, and computer-generated images / videos. The captured, pre-captured, or computer-generated images / videos may be encoded by the image / video encoder 122 to generate an encoded image / video bitstream 162, or analyzed by the feature extractor 124 to extract features. The attributes of the extracted features are encoded by the feature encoder 126 to generate an encoded feature bitstream 164. In some embodiments, the feature extractor 124 can also be configured to extract features from the output of the image / video encoder 122 (i.e., the encoded image / video bitstream 162). In addition or alternatively, the features extracted by the feature extractor 124 can be used by the image / video encoder 122 to assist in the encoding of the image data. For example, before encoding the image data, the image / video encoder 122 can remove certain redundancies from the image data based on the extracted features (such as size / shape information or brightness gradient). Redundancy can include details in the image data that are imperceptible to the human eye (e.g., background brightness, local texture, temporal changes, spatial-temporal frequency information, etc.). According to the disclosed embodiments, the encoded image / video bitstream 162 and the encoded feature bitstream 164 are output to the communication medium 160 via the output interface 126.

[0036] The output interface 128 may include any type of medium or device capable of transmitting the encoded image / video bitstream 162 and / or the feature bitstream 164 from the source device 120 to the target device 140. For example, the output interface 128 may include a transmitter or a transceiver configured to directly transmit the encoded image / video bitstream 162 and / or the feature bitstream 164 from the source device 120 to the target device 140. The encoded image / video bitstream 162 and / or the feature bitstream 164 may be modulated according to a communication standard (e.g., a wireless communication protocol) and transmitted to the target device 140.

[0037] The communication medium 160 may include a transient medium, such as a wireless broadcast or a wired network transmission. For example, the communication medium 160 may include a radio frequency (RF) spectrum or one or more physical transmission lines (e.g., a cable). The communication medium 160 may form part of a packet-based network (e.g., a local area network, a wide area network, or a global network such as the Internet). In some embodiments, the communication medium 160 may include a router, a switch, a base station, or any other device that can be used to facilitate communication from the source device 120 to the target device 140. For example, a network server (not shown) may receive a coded image / video bitstream 162 and / or a feature bitstream 164 from the source device 120 and provide the coded image / video bitstream 162 and / or the feature bitstream 164 to the target device 140, for example, via a network transmission.

[0038] Communication medium 160 may also be in the form of a storage medium (e.g., a non-transitory storage medium), such as a hard disk, a flash drive, a compact disk, a digital video disk, a Blu-ray disk, a volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data. In some embodiments, a computing device of a media production device (e.g., a disk stamping device) may receive the encoded video data from source device 120 and generate a disk containing the encoded video data.

[0039] The target device 140 may include an input interface 148, an image / video decoder 142, and a feature decoder 146. The input interface 148 of the target device 140 receives an encoded image / video bitstream 162 and / or an encoded feature bitstream 164 from the communication medium 160. The encoded image / video bitstream 162 is decoded by the image / video decoder 142 to generate decoded image data, and the encoded feature bitstream 164 is decoded by the feature decoder 146 to generate decoded feature attributes. The target device 140 may also include or be connected to various devices (not shown) for utilizing the decoded image data or feature attributes. For example, the various devices may include a display device for displaying the decoded image data to a user, and may include any of various display devices (e.g., a cathode ray tube (CRT), a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display device). As another example, various devices may include one or more processors configured to use the decoded feature attributes to perform various computer vision applications, such as object recognition and tracking, facial recognition, image matching, image / video search, augmented reality, robotic vision and navigation, autonomous driving, three-dimensional structure construction, stereo matching, motion tracking, etc.

[0040] The image / video encoder 122, the feature encoder 126, the image / video decoder 142, and the feature decoder 146 may each be implemented as any of a variety of suitable encoder / decoder circuits (e.g., one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof). When the technique is implemented partially in software, the device may store instructions for the software in a suitable, non-transitory computer-readable medium, and use one or more processors to execute the instructions in hardware to perform the techniques of the present disclosure. Each of the image / video encoder 122, the feature encoder 126, the image / video decoder 142, and the feature decoder 146 may be included in one or more encoders or decoders, any of which may be integrated as part of a combined encoder / decoder (CODEC) in a corresponding device.

[0041] The image / video encoder 122, the feature encoder 126, the image / video decoder 142, and the feature decoder 146 may operate according to any encoding standard (e.g., the MPEG-7 Compact Descriptors for Visual Search (CDVS) ​​standard, the MPEG-7 Video Analysis Compact Descriptors standard, the MPEG-7 standard on metadata, the Versatile Video Coding (VVC / H.266) standard, the High Efficiency Video Coding (HEVC / H.265) standard, the ITU-T H.264 (also known as MPEG-4) standard, etc.). Although not described in Figure 1 As shown in the figure, in some embodiments, some or all of the image / video encoder 122, the feature encoder 126, the image / video decoder 142 and the feature decoder 146 can be integrated with the audio encoder and decoder, and can include appropriate multiplexing-demultiplexing (MUX-DEMUX) units or other hardware and software to handle the encoding of audio data, image data and / or feature data in a common data stream or separate data streams.

[0042] Still reference Figure 1 For example, the coded image / video bitstream 162 includes syntax information including syntax elements describing characteristics or processing of coded image blocks or units. The syntax information in the coded image / video bitstream 162 is defined by the image / video encoder 122 and used by the image / video decoder 142. In addition, the coded feature bitstream 164 includes syntax information including syntax elements describing characteristics or processing of coded feature attributes. The syntax information in the coded feature bitstream 164 is defined by the feature encoder 126 and used by the feature decoder 146. Although Figure 1While the system 100 is shown to be capable of compressing and transmitting both image data and feature attributes, some embodiments only require the use of one of the image data and feature attributes. For example, object tracking is a computer vision application that involves extracting features using a sensor or camera and sending the encoded feature attributes to a server for analysis. Therefore, in these embodiments, only the encoded feature bitstream 164 needs to be transmitted from the source device 120 to the target device 140.

[0043] According to the disclosed embodiments, the technology for encoding and decoding feature attributes can comply with video coding standards, such as the MPEG-7CDVS standard or the MPEG-7CDVA standard. The MPEG-7CDVS standard was developed to use local descriptors to describe features in static images, while the MPEG-7CDVA standard was developed to use local descriptors to describe features in video sequences (i.e., sequences of consecutive frames). Both the MPEG-7CDVS and MPEG-7CDVA standards are based on the Scale-Invariant Feature Transform (SIFT) technology, which extracts key points from an image and describes the key points using certain predefined attributes or descriptors. DG Lowe introduced the SIFT technology in "Distinctive Image Features from Scale-Invariant Keypoints" in International Journal of Computer Vision, Vol. 60, No. 2, pp. 91-110, 2004.

[0044] To illustrate the concept of key points, Figure 2A is a schematic diagram showing an exemplary input image according to an embodiment of the present disclosure. And Figure 2B A plurality of key points 210 extracted from an input image 200 are shown. Each key point 210 is shown as a circle surrounding a local area with rich features. SIFT technology has been used in the art to extract features and generate key points 210, which are invariant to image translation, image rotation, image scaling, and / or changes such as viewpoint and illumination. Therefore, these key points are also referred to as "SIFT key points".

[0045] To illustrate the properties used to describe key points, Figure 3 3 is a schematic diagram showing exemplary key points 310 in the scale space of an input image according to an embodiment of the present disclosure. The scale space is described by a function generated from the convolution of a Gaussian kernel (at different scales) and the input image. Figure 3As shown, the key point 310 can be described by various parameters, including, for example: the key point position (x, y) in the image scale space where the key point 310 is detected, the scale σ (the radius of the key point in the scale space) and the direction θ (the angle of the key point in the image scale space, expressed in radians). The scale or radius σ describes the extent of the area occupied by the key point 310 in the scale space and depends on the position (x, y) of the key point in the image scale space. In some embodiments, the key point 310 also includes a descriptor (not shown) of the area occupied by the key point 310. The descriptor is a three-dimensional spatial histogram of image gradients that characterizes the appearance of the key point. The gradient at each pixel is regarded as a sample of a three-dimensional basic feature vector formed by the pixel position and the gradient direction. The samples are weighted by the gradient criterion and accumulated into a three-dimensional histogram to form a SIFT descriptor for the area. An additional Gaussian weighting function can be applied to the SIFT descriptor to reduce the importance of gradients away from the key point.

[0046] Back to Figure 2B , multiple key points can be extracted from the image. Each key point can be described by a feature vector, which includes: the key point position (x, y), scale radius σ, direction θ, and / or descriptor. The scale radius σ, direction θ, and descriptor are also called the "attributes" of the key point. For example, for a 1920×1088 image, about 18,000 key points can be extracted, and each key point is described by a feature vector representing the associated feature attributes. Some of these feature vectors are as follows Figure 4 As shown. Figure 4 The feature vectors in do not include information associated with the keypoint descriptors, but it can be considered that each of these feature vectors may include additional elements representing the associated keypoint descriptors.

[0047] Fig.17 1 is a flow chart of an exemplary scale-invariant feature transform (SIFT) algorithm 1700 for extracting features from image data according to an embodiment of the present disclosure. For example, the algorithm 1700 may be composed of a feature encoder (such as the feature encoder 126 ( Figure 1 ))Execute. Fig.17As shown, the SIFT algorithm 1700 may include four steps: 1710-extreme value detection, 1720-key point positioning, 1730-direction assignment, and 1740-key point descriptor generation. Specifically, in step 1710, the feature encoder examines the input image at various scales to separate the points of the input image from their surroundings. These points (called extrema) are potential candidates for image features. In step 1720 (key point detection), the feature encoder selects some of these points (extreme values) as key points. This selection eliminates extreme values ​​caused by image edges and low-contrast points. In step 1730 (direction assignment), the feature encoder converts each key point and its neighboring points into a vector set by calculating the scale σ and direction θ. In step 1740 (key point descriptor generation), the feature encoder obtains a set of vectors in the neighborhood of each key point and merges this information into a vector set, called a descriptor. Each descriptor is converted into a feature by calculating the normalized sum of these vectors.

[0048] The feature vectors can be analyzed and used by various computer vision applications, such as object detection, recognition, and tracking. The feature vectors can be stored / transmitted in their original form. Optionally and additionally, the feature vectors can be described in a stream syntax or encapsulated with additional information (e.g., a checksum).

[0049] The increase in demand for using feature attributes or feature streams has increased the amount of feature data that needs to be stored or transmitted. However, the above-mentioned feature attributes may occupy a large data range, resulting in huge data storage and transmission costs. Therefore, it is desirable to develop effective techniques for storing and transmitting feature data. The present disclosure provides some embodiments that improve the efficiency of storing and transmitting feature data by modifying the representation of feature data to reduce its dynamic range, thereby reducing the cost of storing and transmitting feature data.

[0050] Specifically, in the disclosed embodiments, the key points extracted from the image are divided into a plurality of clusters based on the key point positions. The key points in each cluster have better matching local image attributes, so the difference in attribute values ​​is smaller. The dynamic range of the attribute values ​​in each cluster can be reduced by subtracting the reference value associated with the corresponding cluster from each attribute value. For example, in some embodiments, the attribute value of the medoid of the cluster is selected as the reference value of the cluster. The medoid of the cluster is defined as one of the key points of the cluster, and the sum of its distances to all other key points in the cluster is the smallest. In the cluster, the attribute value of the non-medoid key point is subtracted from the attribute value of the medoid, and the attribute value of the medoid key point represents the attribute value of the cluster, so the attribute value of the medoid key point remains unchanged. By this differential representation of the attribute value of the non-medoid key point relative to the attribute value of the corresponding medoid key point, the dynamic range of the attribute value is reduced.

[0051] Further in Figure 5 The above concept of differential representation of non-center keypoints is illustrated in , which shows a cluster consisting of 8 keypoints represented by solid dark circles. One of the keypoints is determined to be the center point of the cluster, and the sum of the distances from the center point to all other keypoints in the cluster is the smallest. The scale value of the non-center keypoint can be subtracted from the scale value of the center keypoint to reduce their dynamic range. The scale value of the center keypoint remains unchanged, so it can be retained as the reference value of the cluster. Since in the SIFT algorithm, the keypoints in a cluster are usually indicated at the same level of the scale pyramid, the scale value of the center keypoint can well identify the scale value of the keypoints in the same cluster. Although Figure 5 Only the differential representation of the scale values ​​of non-center keypoints is shown, but it can be considered that the differential representation can also be applied to the direction values ​​in the same way.

[0052] In some embodiments, a medoid clustering algorithm (e.g., Partitioning Around Medoids (PAM) algorithm) is used to partition key points and determine the medoids of the clusters. The PAM algorithm searches for a set of representative data points in the input data set, and then assigns the remaining data points to the nearest medoids to create clusters. The representative data point of a cluster is also referred to as the "medoid" of the cluster, which is the mathematical center of the cluster. The remaining data points in the cluster are referred to as "non-medoids". In some embodiments, the PAM algorithm can be implemented in three steps. In the first step, the PAM algorithm selects one or more data points from the input data set as initial medoids. One or more initial medoids correspond to one or more clusters, respectively. In the second step, the PAM algorithm associates each remaining data point with its nearest initial medoid. In the third step, the PAM algorithm searches for the best medoid of the cluster by minimizing the cost function of the sum of the differences between the measuring medoid and other data points in the same cluster.

[0053] In some embodiments, keypoints located on the convex hull of the input keypoints are used as the initial center points for the center point clustering algorithm. This approach ensures that the encoder and decoder use the same method to determine the initial center points, thus resulting in the same clusters and center points being generated on both the encoder and decoder sides. Figure 6 As shown, the convex hull of a set of key points is the smallest convex polygon containing all the key points in the set. Specifically, the set of key points is shown as a gray filled area, and the convex hull is shown as a solid polygon containing all the key points. A convex hull determination algorithm (e.g., Jarvis march algorithm) can be used to determine the convex hull. Fig.18 is a schematic diagram showing an exemplary method for implementing the Jarvis march algorithm. Fig.18 As shown in , the Jarvis march algorithm starts from the leftmost key point P0 and searches for convex hull key points in the counter-clockwise direction. Specifically, the Jarvis march algorithm starts from the leftmost key point P0 and finds the angle ( Fig.18 Then, the Jarvis march algorithm starts from key point P1 and finds the angle ( Fig.18 The next convex hull keypoint P2 with the largest angle in the simulation is selected using the origin. The angle between each other point in the simulation is calculated using the origin. The Jarvis march algorithm repeats the above process until it returns to the leftmost keypoint P0. The set of keypoints selected in this way constitutes the convex hull.

[0054] Keypoints located on the convex hull can be used as initial center points for the disclosed clustering process (eg, the PAM algorithm), the output of which is a set of clusters with center points. Figure 7 FIG. 1 is a schematic diagram showing a center point clustering process according to an embodiment of the present disclosure. Figure 7 As shown, a set of keypoints is shown as a gray area. Keypoints located on the convex hull of the set of keypoints are shown as asterisks and are used as the initial center points for the center point clustering process. The cross marks correspond to the center points of the clusters output by the center point clustering process.

[0055] In some embodiments, a hierarchical clustering process can be used to partition key points. For example, for any given cluster, the ratio of the total number of key points in the cluster to the number of key points on the convex hull of the cluster is determined. If the ratio is greater than or equal to the number of key points on the convex hull of the cluster, the cluster is further divided into smaller clusters; otherwise, the cluster is not further divided. Therefore, the final output of the hierarchical clustering process is a plurality of clusters that are considered indivisible. This ensures the clarity of the partitioning and achieves a high level of granularity so that the reference values ​​of the resulting clusters (e.g., the attribute values ​​of the center point key points) can better match the local scene attributes.

[0056] In summary, the differential representation of the attribute values ​​of the non-center key points described above is a lossless hierarchical coding method. It does not lose any useful feature attribute information. It can also be reversibly encoded and decoded. In addition, in addition to the attribute value itself, no additional information needs to be sent to the decoder for decoding. Therefore, the disclosed technology for encoding attribute values ​​is efficient. The technology can be applied to tools that use SIFT, such as direct SIFT, CDVS, CDVA, etc.

[0057] The disclosed techniques for encoding property values ​​may be implemented in an encoder or a decoder. Figure 8 8 is a block diagram of an exemplary apparatus for encoding or decoding image feature attributes according to an embodiment of the present disclosure. For example, the feature encoder 126 and the feature decoder 146 can be implemented as apparatus 800 for performing the above method. Figure 8As shown, the device 800 may include a processor 802. When the processor 802 executes the instructions described herein, the device 800 may become a special-purpose machine for encoding or decoding feature attributes. The processor 802 may be any type of circuit capable of manipulating or processing information. For example, the processor 802 may include any number of central processing units (or "CPUs"), graphics processing units (or "GPUs"), neural processing units ("NPUs"), microcontroller units ("MCUs"), optical processors, programmable logic controllers, microcontrollers, microprocessors, digital signal processors, intellectual property (IP) cores, programmable logic arrays (PLAs), programmable array logic (PALs), general array logic (GALs), complex programmable logic devices (CPLDs), field programmable gate arrays (FPGAs), systems on chips (SoCs), application-specific integrated circuits (ASICs), and the like. In some embodiments, the processor 802 may also be a group of processors grouped into a single logical component. For example, as Figure 8 As shown, processor 802 may include multiple processors, including processor 802a, processor 802b, and processor 802n.

[0058] The apparatus 800 may also include a memory 804 configured to store data (eg, instruction sets, computer codes, intermediate data, etc.). Figure 8 As shown, the stored data may include program instructions (e.g., program instructions for implementing encoding or decoding of feature attributes) and data for processing (e.g., image data, features extracted from image data, or encoded feature bit streams). The processor 802 may access the program instructions and data for processing (e.g., via bus 810), and execute the program instructions to perform operations or manipulations on the data for processing. The memory 804 may include a high-speed random access memory device or a non-volatile memory device. In some embodiments, the memory 804 may include any number of random access memories (RAMs), read-only memories (ROMs), optical disks, magnetic disks, hard disk drives, solid-state drives, flash drives, secure digital (SD) cards, memory sticks, compact flash (CF) cards, etc. Any combination. The memory 804 may also be a group of memories ( Figure 8 not shown).

[0059] The bus 810 may be a communication device for transmitting data between components within the apparatus 800 , such as an internal bus (eg, a CPU memory bus), an external bus (eg, a universal serial bus port, a peripheral component interconnect express port), etc.

[0060] For ease of explanation and without ambiguity, the processor 802 and other data processing circuits are collectively referred to as "data processing circuits" in this disclosure. The data processing circuits may be implemented entirely as hardware, or as a combination of software, hardware, or firmware. In some embodiments, the data processing circuit may be a single stand-alone module, or may be fully or partially combined into any other component of the device 800.

[0061] The device 800 may also include a network interface 806 to provide wired or wireless communication with a network (e.g., the Internet, an intranet, a local area network, a mobile communication network, etc.). In some embodiments, the network interface 806 may include any number of network interface controllers (NICs), radio frequency (RF) modules, transponders, transceivers, modems, routers, gateways, wired network adapters, wireless network adapters, Bluetooth adapters, infrared adapters, near field communication (“NFC”) adapters, cellular network chips, etc.

[0062] In some embodiments, the apparatus 800 may also include a peripheral interface 808 to provide a connection to one or more peripheral devices. Figure 8 As shown, peripheral devices may include, but are not limited to, a cursor control device (e.g., a mouse, touchpad, or touch screen), a keyboard, a display (e.g., a cathode ray tube display, a liquid crystal display, or a light emitting diode display), an image / video input device (e.g., a camera or input interface coupled to an image / video archive), and the like.

[0063] Fig. 9 8 is a flowchart showing a method 900 for encoding image feature attributes according to an embodiment of the present disclosure. For example, the method 900 may be performed by a feature encoder (eg, a processor device (eg, processor 802) in the feature encoder). Fig. 9 As shown, method 900 includes the following steps 902-906.

[0064] In step 902, multiple image features are divided into one or more clusters. For example, the image features can be scale invariant feature transform (SIFT) key points extracted from the image. Each key point can be described by various attributes (e.g., including scale and direction). The division is based on the location of the key point. In some embodiments, a partition around center point (PAM) algorithm is performed by a feature encoder to divide multiple image features. The cluster output by the PAM algorithm includes a center point key point and one or more non-center point key points.

[0065] In some embodiments, in step 902, the feature encoder performs the partitioning using a hierarchical partitioning process. Fig.10 According to the embodiment of the present disclosure, in step 902 ( Fig. 9) is a flowchart of an exemplary hierarchical partitioning method used in FIG. For example, method 1000 may be executed by a processor (e.g., processor 802) in a feature encoder. Fig.10 As shown, method 1000 can be performed multiple times in iterations. In step 1002, the feature encoder receives multiple input key points, which are image regions generated by applying a feature extraction algorithm (e.g., a SIFT algorithm) to an image. Multiple input key points may belong to clusters generated by previous iterations of method 1000. Then, in step 1004, the feature encoder determines the total number of input key points. In this counting process, key points (x, y) located at the same position are counted as separate key points. The total number counted is represented as "N" in the following description. Then, in step 1006, the feature encoder determines the convex hull of the input key points. For example, a Jarvis march algorithm can be performed to determine the convex hull. In step 1008, the feature encoder identifies the number of key points located on the convex hull. In the following description, the number of key points on the convex hull is represented as "M". In step 1010, the feature encoder determines The value of , which is the ratio of the total number of input keypoints to the number of keypoints located on the convex hull. In one embodiment, the feature encoder can The value of is rounded down to the nearest integer. After the quotient, the feature encoder will is compared with M (the number of key points on the convex hull). The feature encoder determines that the input keypoint can be further divided and proceeds to step 1012 to divide the input keypoint; or if In this case, the feature encoder determines that the input key points form an indivisible cluster and proceeds to step 1014 to determine parameters representing the attributes of the input key points. The details of the operations involved in step 1014 will be described below in conjunction with Fig.11 Provide explanation.

[0066] In one embodiment, if The feature encoder determines that the input keypoint can be further divided and proceeds to step 1012 to divide the input keypoint; or if In this case, the feature encoder determines that the input keypoints form an indivisible cluster and proceeds to step 1014 to determine parameters representing attributes of the input keypoints.

[0067] Return reference method 1000( Fig.10), in step 1012, the feature encoder divides the input key points into a plurality of clusters (e.g., by performing a partition around center point (PAM) algorithm). The division is based on the position of the input key points. M key points located on the convex hull (determined in step 1008) can be used as the initial center points of the PAM algorithm so that the algorithm divides the input key points into M clusters. As described above, in order to perform clear clustering on the encoder side and the decoder side without the need to send additional information to the decoder, it may be necessary to accurately define the initial center points of the PAM algorithm used by the encoder and the decoder. By defining the initial center point as the center point located on the convex hull, it is ensured that the encoder and the decoder divide the key points in the same manner without the need for additional encoding information sent from the encoder to the decoder. In some embodiments, the information of the initial center point can be provided to the decoder to facilitate the identification of the key points. For example, the information provided to the decoder can include the number or initial position of the initial center points. Such information enables the decoder to use the same initial center point as the encoder.

[0068] After step 1012, the feature encoder returns to step 1002 and initiates a new iteration of method 1000 on each of the plurality of clusters obtained from step 1012. When no more clusters can be further divided, the feature encoder ends method 1000.

[0069] Return to reference method 900 ( Fig. 9 ), in step 904, the feature encoder determines a reference value of at least one of the one or more clusters in the plurality of clusters. For example, the reference value of the cluster can be determined to be equal to the attribute value (e.g., scale value or direction value) of the center point key point of the cluster.

[0070] In step 906, the feature encoder generates a plurality of parameters respectively associated with the plurality of clusters. The plurality of parameters represent image features of the key points. Each of the plurality of parameters is generated based on a value of an attribute of the corresponding image feature and a reference value of the corresponding cluster. According to some embodiments, the value of the parameter associated with each center point feature is equal to the reference value of the corresponding cluster (i.e., the attribute value of the center point of the corresponding cluster). And the value of the parameter associated with each non-center point feature is equal to the difference between the attribute value of the corresponding non-center point feature and the reference value of the corresponding cluster (i.e., the attribute value of the center point of the corresponding cluster).

[0071] The operations involved in steps 904 and 906 are further described as follows: Fig.11 As shown, Fig.11 A method 1100 for determining parameters representing attributes of a cluster of key points is shown. For example, the method 1100 may be performed by a processor (e.g., processor 802) in a feature encoder. Fig.11 As shown, method 1100 includes the following steps 1102-1106.

[0072] In step 1102, the feature encoder determines the center point of the cluster of key points. The attribute value of the center point is used as a reference value for the cluster. For example, the attribute of interest may be the scale of the key point, and the scale value of the center point is used as a reference scale value for the cluster.

[0073] In step 1104, for each non-center keypoint of the cluster, the feature encoder determines a parameter representing the attribute value of the non-center keypoint. In some embodiments, the value of the parameter is determined to be equal to the difference between the attribute value of the non-center keypoint and the reference value of the corresponding cluster. For example, if the attribute of interest is keypoint scale, then the parameter for each non-center keypoint is determined to be equal to σ 非中心点{i} -σ 簇的中心点 , where σ 非中心点{i} represents the scale value of the i-th non-center key point, and σ 簇的中心点 represents the scale value of the center point keypoint. As another example, if the attribute of interest is the keypoint direction, then the parameter of each non-center point keypoint is determined to be equal to θ 非中心点{i} -θ 簇的中心点 , where θ 非中心点{i} represents the direction value of the i-th non-center key point, and θ 簇的中心点 Indicates the orientation value of the center point keypoint.

[0074] In step 1106, the feature encoder determines whether parameters have been determined for all non-center keypoints in the cluster. If not, the feature encoder returns to step 1104 and determines the parameters for the next non-center keypoint in the cluster. If the last non-center keypoint in the cluster has been reached, the feature encoder ends method 1100 and returns the determined parameters.

[0075] According to the disclosed embodiment, in method 1000 ( Fig.10 ) executes method 1100 at step 1014 to determine the parameter representing the attribute of the non-center key point. In addition, the parameter representing the attribute of the center key point is made equal to the attribute value of the center key point. Return to reference method 900 ( Fig. 9 ), at the end of step 906, the generated multiple parameters can be inserted into the feature bitstream to be sent to the decoder. As described above, because the data range of the generated multiple parameters is smaller than the range of the original attribute value, using the generated multiple parameters to represent the attribute value can reduce storage and transmission costs.

[0076] Fig.121 is a flow chart showing a method 1200 for decoding an image feature attribute according to an embodiment of the present disclosure. For example, the method 1200 may be performed by a feature decoder (eg, a processor device (eg, processor 802) in the feature decoder). Fig.12 As shown, method 1200 includes the following steps 1202-1208.

[0077] In step 1202, a feature decoder receives a data stream including a plurality of parameters respectively associated with a plurality of image features. For example, the image features may be scale-invariant feature transform (SIFT) keypoints extracted from an image. Each keypoint may be described by various attributes (e.g., including scale and orientation).

[0078] In step 1204, the feature decoder divides the plurality of image features into one or more clusters. The division is based on the location of the keypoints. In some embodiments, a partitioning around center point (PAM) algorithm is performed by the feature decoder to divide the plurality of image features. The clusters output by the PAM algorithm include a center point keypoint and one or more non-center point keypoints.

[0079] In some embodiments, the feature decoder performs the partitioning in step 1204 using a hierarchical partitioning process. Fig.13 According to step 1204 ( Fig.12 ) is a flowchart of an exemplary hierarchical partitioning method 1300 used in FIG. For example, the method 1300 may be performed by a feature decoder (eg, a processor device (eg, processor 802) in a feature decoder). Fig.13 As shown, method 1300 can be performed multiple times in iterations. In step 1302, the feature decoder receives multiple input key points. The multiple input key points may belong to clusters generated by previous iterations of method 1300. Then, in step 1304, the feature decoder determines the total number of input key points. In this counting process, key points (x, y) located at the same position are counted separately. The total number of counts is represented as "N" in the following description. In step 1306, the feature decoder determines the convex hull of the input key points. For example, the Jarvis march algorithm can be performed to determine the convex hull. In step 1308, the feature decoder identifies the number of key points located on the convex hull. In the following description, the number of key points on the convex hull is represented as "M". In step 1310, the feature decoder determines The value of is the ratio of the total number of input keypoints to the number of keypoints lying on the convex hull. In one embodiment, the feature decoder can The value of is rounded down to the nearest integer. After the quotient, the processor 802 will is compared with M (the number of key points on the convex hull). The feature decoder determines that the input keypoint can be further divided and proceeds to step 1312 to divide the input keypoint; or if The feature decoder determines that the input key points form an indivisible cluster and proceeds to step 1314 to determine the attribute values ​​of the input key points. The detailed operations involved in step 1314 will be described below in conjunction with Fig.14 Provide explanation.

[0080] In one embodiment, if The feature encoder determines that the input keypoint can be further divided and proceeds to step 1012 to divide the input keypoint; or if In this case, the feature encoder determines that the input keypoints form an indivisible cluster and proceeds to step 1014 to determine parameters representing attributes of the input keypoints.

[0081] Return to reference method 1300 ( Fig.13 ), in step 1312, the feature decoder divides the input key points into a plurality of clusters (e.g., by executing a partition around center point (PAM) algorithm). The division is based on the position of the input key points. M key points located on the convex hull (determined in step 1308) can be used as the initial center points of the PAM algorithm so that the algorithm divides the input key points into M clusters. As described above, in order to perform explicit clustering on the encoder side and the decoder side without sending additional information to the decoder, it may be necessary to accurately define the initial center points of the PAM algorithm used by the encoder and the decoder. By defining the initial center point as the center point located on the convex hull, it is ensured that the encoder and the decoder divide the key points in the same manner without the need for additional encoding information sent from the encoder to the decoder. In some embodiments, the encoder may provide the information of the initial center point directly to the decoder to facilitate identification of the key points. For example, the information provided to the decoder may include the number or initial position of the initial center point.

[0082] After step 1312, the feature encoder returns to step 1302 and initiates a new iteration of method 1300 on each of the plurality of clusters obtained from step 1312. When no more clusters can be further divided, the feature encoder ends method 1300.

[0083] Return to reference method 1200 ( Fig.12 ), in step 1206, the feature decoder determines a reference value of at least one cluster of one or more clusters in the plurality of clusters. For example, the reference value of the cluster may be determined to be equal to the value of the parameter associated with the center point key point of the cluster.

[0084] In step 1208, the feature decoder determines respective attributes of the plurality of image features. The attributes of the image features are determined based on: a parameter associated with the respective image feature and a reference value for the respective cluster. According to some embodiments, the value of the attribute of each center point feature is determined to be equal to the reference value for the respective cluster. And the value of the attribute of each non-center point feature is determined to be equal to the sum of: the value of the parameter associated with the respective non-center point feature; and the reference value for the respective cluster (i.e., the value of the parameter associated with the center point of the respective cluster).

[0085] In some embodiments, the operation involved in step 1208 may be determining the attributes of all image features, or determining the attributes of some image features.

[0086] The operations involved in steps 1206 and 1208 are further described as follows: Fig.14 As shown, Fig.14 A method 1400 for determining attribute values ​​of a cluster of key points is shown. For example, the method 1400 may be performed by a processor (e.g., processor 802) in a feature decoder. Fig.14 As shown, method 1400 includes the following steps 1402-1406.

[0087] In step 1402, the feature decoder determines the center point of the cluster of key points. The value of the parameter associated with the center point is used as the reference value of the cluster.

[0088] In step 1404, for each non-center keypoint of the cluster, the feature decoder determines an attribute value of the non-center keypoint. Specifically, the attribute value is determined to be equal to the sum of the value of the parameter associated with the non-center keypoint associated with the parameter and the reference value of the corresponding cluster. For example, the scale value of each non-center keypoint is determined to be equal to parameter 非中心点{i} +parameter 簇的中心点 , where parameter 非中心点{i} represents the value of the parameter associated with the i-th non-center keypoint, and parameter 簇的中心点 Represents the value of the parameter associated with the center point keypoint (i.e., the reference value of the corresponding cluster).

[0089] In step 1406, the feature decoder determines whether attribute values ​​have been determined for all non-center keypoints in the cluster. If not, the feature decoder returns to step 1404 and determines the attribute value of the next non-center keypoint in the cluster. If the last non-center keypoint in the cluster has been reached, the feature decoder ends method 1400 and returns the determined attribute value.

[0090] According to the disclosed embodiments, in method 1300 ( Fig.13) is executed at step 1314 of the method 1400 to determine the attribute value of the non-center point key point. In addition, the attribute value of the center point key point is made equal to the value of the parameter associated with the center point key point.

[0091] The above-mentioned method for encoding and decoding feature attributes can reduce the total cost of representing attribute values ​​(e.g., scale values ​​or orientation values ​​of SIFT key points). Fig.15 and Fig.16 shown. Specifically, Fig.15 shows a histogram of σ values ​​for SIFT keypoints extracted from a sample Poznan parking lot frame without using the present scheme for representing scale values. Fig.16 A histogram of σ values ​​of SIFT keypoints extracted from the same sample Poznan parking lot frame using the present scheme for representing scale values ​​is shown. Fig.15 and Fig.16 The conditions for generating the histogram in include: quantization parameter (QP) = 47, 5996 key points. In addition, Fig.16 The key points in are divided into 24 clusters. Fig.15 and Fig.16 In the plot, the box plot covers 50% of the observations; the dotted line in the box plot is the median of these 50% observations; and the continuous kernel density estimation (KDE) plot uses a Gaussian kernel to smooth the observations to produce a continuous density estimate. Fig.15 In comparison, the dynamic range of the 50%-observation box plot is narrower. Therefore, the disclosed method for representing the attribute value of a key point and the disclosed method for encoding and decoding feature data can reduce the dynamic range of the encoded feature data.

[0092] In some embodiments, a non-transitory computer-readable storage medium is also provided, which includes instructions, and the instructions can be executed by a device (e.g., the disclosed encoder and decoder) for performing the above method. Common forms of non-transitory media include: for example, a floppy disk, a floppy disk, a hard disk, a solid-state drive, a tape or any other magnetic data storage medium, a CD-ROM, any other optical data storage medium, any physical medium with a hole pattern, RAM, PROM, EPROM, FLASH-EPROM or any other flash memory, NVRAM, cache, registers, any other memory chip or cartridge memory, and networked versions thereof. The device may include one or more processors (CPU), an input / output interface, a network interface, and / or a memory.

[0093] It may be noted that relational terms such as "first" and "second" herein are used only to distinguish an entity or operation from another entity or operation, without requiring or implying any actual relationship or order between these entities or operations. In addition, the words "include", "have", "contain", and "comprise" and other similar forms are intended to represent equivalent meanings and are open-ended, and the one or more items following any one of the words do not mean an exhaustive list of these items, nor are they limited to the one or more items listed.

[0094] The term "or" used herein encompasses all possible combinations unless otherwise specifically stated or not feasible. For example, if it is stated that a database may include A or B, then unless otherwise specifically stated or not feasible, the database may include A, or B, or A and B. As a second example, if it is stated that a database may include A, B, or C, then unless otherwise specifically stated or not feasible, the database may include A, or B, or C, or A and B, or A and C, or B and C, or A, B, and C.

[0095] It is to be understood that the present invention is not limited to the exact structures described above and shown in the drawings, and various modifications and changes may be made without departing from the scope of the present invention. The scope of the present invention shall be limited only by the appended claims.

Claims

1. A computer-implemented decoding method, comprising: receiving a data stream, the data stream comprising a plurality of parameters respectively associated with a plurality of image features; dividing the plurality of image features into one or more clusters; determining a reference value for at least one of the one or more clusters; as well as Determine corresponding attributes of the plurality of image features, wherein the attributes of the image features are determined based on: parameters associated with the corresponding image features, and Reference value of the corresponding cluster.

2. The method according to claim 1, wherein: The image features are scale invariant feature transform (SIFT) key points on the image.

3. The method according to claim 1 or 2, wherein: The attributes of the plurality of image features include at least one of the following: the scales of the plurality of image features; or The directions of the plurality of image features.

4. The method according to any one of claims 1 to 3, wherein: The segmentation is based on positions of the plurality of image features on the image.

5. The method according to any one of claims 1 to 4, wherein: The plurality of image features are divided using a PAM algorithm divided around a center point.

6. The method according to any one of claims 1 to 5, wherein: The image features in a cluster include a center point feature, and a reference value for the cluster is determined as a value of a parameter associated with the corresponding center point feature.

7. The method according to any one of claims 1 to 6, wherein: The image features in the cluster include a center point feature and one or more non-center point features, wherein determining the attributes of the plurality of image features includes: determining the attribute of the center point feature to have a value equal to a reference value of the corresponding cluster; and The attribute of the non-center point feature is determined to have a value equal to the sum of: the value of the parameter associated with the corresponding non-midpoint feature, and The reference value of the corresponding cluster.

8. The method according to claim 7, wherein: The center point feature of the cluster is the center point of the image feature in the corresponding cluster.

9. The method according to any one of claims 1 to 8, further comprising: Before the segmentation, determining the convex hull of the plurality of image features, The image features on the convex hull act on the initial center point of the division.

10. The method according to claim 9, wherein: The convex hull is determined using the Jarvis march algorithm.

11. The method according to any one of claims 1 to 10, wherein: The dividing comprises: performing an iterative hierarchical division process on the plurality of image features, wherein performing the iterative hierarchical division process comprises: Performing a first iteration of partitioning to generate a first cluster of image features, wherein there are N total image features in the first cluster and M image features located on the convex hull of the first cluster, N and M being positive integers; and In response to is greater than or equal to M, performing a second iteration of partitioning to partition the first cluster, wherein when When it is less than M, no further partitioning is performed on the first cluster.

12. The method according to any one of claims 1 to 10, wherein: The dividing includes performing an iterative hierarchical division process on the plurality of image features, wherein performing the iterative hierarchical division process includes: Performing a first iteration of partitioning to generate a first cluster of image features, wherein there are N total image features in the first cluster and M image features located on the convex hull of the first cluster, N and M being positive integers; and In response to N being greater than or equal to M*M, a second iteration of partitioning is performed to partition the first cluster, wherein when N is less than M*M, no further partitioning is performed on the first cluster.

13. The method according to claim 11 or 12, wherein: The M image features located on the convex hull of the first cluster are used as initial center points for dividing the first cluster.

14. A computer-implemented encoding method comprising: dividing a plurality of image features into one or more clusters; determining a reference value for at least one of the one or more clusters; as well as Generate a plurality of parameters respectively associated with the plurality of image features, wherein each parameter of the plurality of parameters is generated based on: the value of the attribute of the corresponding image feature, and Reference value of the corresponding cluster.

15. The method according to claim 14, further comprising: The plurality of parameters are sent in a data stream to a decoder.

16. The method according to claim 14 or 15, wherein: The image features are scale invariant feature transform (SIFT) key points on the image.

17. The method according to any one of claims 14 to 16, wherein: The attributes of the plurality of image features include at least one of the following: the scales of the plurality of image features; or The directions of the plurality of image features.

18. The method according to any one of claims 14 to 17, wherein: The segmentation is based on positions of the plurality of image features on the image.

19. The method according to any one of claims 14 to 18, wherein: The plurality of image features are divided using a PAM algorithm divided around a center point.

20. The method according to any one of claims 14 to 19, wherein: The image features in the cluster include a center point feature, and a reference value of the cluster is determined as a value of an attribute corresponding to the center point feature.

21. The method according to any one of claims 14 to 20, wherein: The image features in a cluster include a center point feature and one or more non-center point features, where: The parameter associated with the center point feature has a value equal to a reference value of the corresponding cluster; and The parameter associated with the non-center point feature has a value equal to the difference between: the value of the attribute at the corresponding non-central point, and Reference value of the corresponding cluster.

22. The method according to claim 21, wherein: The center point feature of the cluster is the center point of the image feature in the corresponding cluster.

23. The method according to any one of claims 14 to 22, further comprising: Before the segmentation, determining the convex hull of the plurality of image features, The image features on the convex hull are used as initial center points for the division.

24. The method according to claim 23, wherein: The convex hull is determined using the Jarvis march algorithm.

25. The method according to any one of claims 14 to 24, wherein: The dividing includes performing an iterative hierarchical division process on the plurality of image features, wherein performing the iterative hierarchical division process includes: Performing a first iteration of partitioning to generate a first cluster of image features, wherein there are N total image features in the first cluster and M image features located on the convex hull of the first cluster, N and M being positive integers; and In response to is greater than or equal to M, performing a second iteration of partitioning to partition the first cluster, wherein when When it is less than M, no further partitioning is performed on the first cluster.

26. The method according to any one of claims 14 to 24, wherein: The dividing includes performing an iterative hierarchical division process on the plurality of image features, wherein performing the iterative hierarchical division process includes: Performing a first iteration of partitioning to generate a first cluster of image features, wherein there are N total image features in the first cluster and M image features located on the convex hull of the first cluster, N and M being positive integers; and In response to N being greater than or equal to M*M, a second iteration of partitioning is performed to partition the first cluster, wherein when N is less than M*M, no further partitioning is performed on the first cluster.

27. The method according to claim 25 or 26, wherein: The M image features located on the convex hull of the first cluster are used as initial center points for dividing the first cluster.

28. An apparatus comprising: A memory for storing instructions; as well as a processor configured to execute the instructions so that the apparatus: receiving a data stream, the data stream comprising a plurality of parameters respectively associated with a plurality of image features; dividing the plurality of image features into one or more clusters; determining a reference value for at least one of the one or more clusters; as well as Determining corresponding attributes of the plurality of image features, wherein the attributes of the image features are determined based on: parameters associated with the corresponding image features, and Reference value of the corresponding cluster.

29. The device according to claim 28, wherein The image features are scale invariant feature transform (SIFT) key points on the image.

30. The device according to claim 28 or 29, wherein: The attributes of the plurality of image features include at least one of the following: the scales of the plurality of image features; or The directions of the plurality of image features.

31. The device according to any one of claims 28 to 30, wherein: The processor is configured to execute the instructions to cause the apparatus to divide the plurality of image features based on positions of the plurality of image features on the image.

32. The device according to any one of claims 28 to 31, wherein: The processor is configured to execute the instructions to cause the apparatus to segment the plurality of image features using a PAM algorithm segmented around a center point.

33. The device according to any one of claims 28 to 32, wherein: The image features in the cluster include a center point feature, and the processor is configured to execute the instructions to cause the apparatus to determine a reference value for the cluster as a value of a parameter associated with the corresponding center point feature.

34. The device according to any one of claims 28 to 33, wherein: The image features in the cluster include a center point feature and one or more non-center point features, wherein when determining the attributes of the plurality of image features, the processor is configured to execute the instructions to enable the apparatus to: determining the attribute of the center point feature to have a value equal to a reference value of the corresponding cluster; and The attribute of the non-center point feature is determined to have a value equal to the sum of: the value of the parameter associated with the corresponding non-midpoint feature, and Reference value of the corresponding cluster.

35. The device according to claim 34, wherein The center point feature of the cluster is the center point of the image feature in the corresponding cluster.

36. The device according to any one of claims 28 to 35, wherein The processor is configured to execute the instructions to cause the apparatus to: Before the segmentation, determining the convex hull of the plurality of image features, The image features on the convex hull are used as initial center points for the segmentation.

37. The device according to claim 36, wherein: The processor is configured to execute the instructions to cause the apparatus to determine the convex hull using a Jarvis march algorithm.

38. The device according to any one of claims 28 to 37, wherein: The processor is configured to execute the instructions to enable the apparatus to divide the plurality of image features by performing an iterative hierarchical division process, the iterative hierarchical division process comprising: Performing a first iteration of partitioning to generate a first cluster of image features, wherein there are N total image features in the first cluster and M image features located on the convex hull of the first cluster, N and M being positive integers; and In response to is greater than or equal to M, performing a second iteration of partitioning to partition the first cluster, wherein when When it is less than M, no further partitioning is performed on the first cluster.

39. The device according to any one of claims 28 to 37, wherein: The processor is configured to execute the instructions to enable the apparatus to divide the plurality of image features by performing an iterative hierarchical division process, the iterative hierarchical division process comprising: Performing a first iteration of partitioning to generate a first cluster of image features, wherein there are N total image features in the first cluster and M image features located on the convex hull of the first cluster, N and M being positive integers; and In response to N being greater than or equal to M*M, a second iteration of partitioning is performed to partition the first cluster, wherein when N is less than M*M, no further partitioning is performed on the first cluster.

40. The device according to claim 38 or 39, wherein: The M image features located on the convex hull of the first cluster are used as initial center points for dividing the first cluster.

41. An apparatus comprising: A memory for storing instructions; as well as a processor configured to execute the instructions so that the apparatus: dividing a plurality of image features into one or more clusters; determining a reference value for at least one of the one or more clusters; as well as Generate a plurality of parameters respectively associated with the plurality of image features, wherein each parameter of the plurality of parameters is generated based on: the value of the attribute of the corresponding image feature, and Reference value of the corresponding cluster.

42. The device according to claim 41, wherein The processor is configured to execute the instructions to cause the apparatus to: The plurality of parameters are sent in a data stream to a decoder.

43. The device according to claim 41 or 42, wherein: The image features are scale invariant feature transform (SIFT) key points on the image.

44. The device according to any one of claims 41 to 43, wherein: The attributes of the plurality of image features include at least one of the following: the scales of the plurality of image features; or The directions of the plurality of image features.

45. The device according to any one of claims 41 to 44, wherein The processor is configured to execute the instructions to cause the apparatus to divide the plurality of image features based on positions of the plurality of image features on the image.

46. ​​The device according to any one of claims 41 to 45, wherein The processor is configured to execute the instructions to cause the apparatus to segment the plurality of image features using a PAM algorithm segmented around a center point.

47. The device according to any one of claims 41 to 46, wherein The image features in the cluster include a center point feature, and the processor is configured to execute the instructions to cause the apparatus to determine a reference value of the cluster as a value of an attribute of the corresponding center point feature.

48. The device according to any one of claims 41 to 47, wherein The image features in a cluster include a center point feature and one or more non-center point features, where: The parameter associated with the center point feature has a value equal to a reference value of the corresponding cluster; and The parameter associated with the non-center point feature has a value equal to the difference between: the value of the attribute at the corresponding non-central point, and Reference value of the corresponding cluster.

49. The apparatus of claim 48, wherein: The center point feature of the cluster is the center point of the image feature in the corresponding cluster.

50. The device according to any one of claims 41 to 49, wherein The processor is configured to execute the instructions to cause the apparatus to: Before the segmentation, determining the convex hull of the plurality of image features, The image features on the convex hull are used as initial center points for the segmentation.

51. The apparatus of claim 50, wherein: The processor is configured to execute the instructions to cause the apparatus to determine the convex hull using a Jarvis march algorithm.

52. The device according to any one of claims 41 to 51, wherein: The processor is configured to execute the instructions to enable the apparatus to divide the plurality of image features by performing an iterative hierarchical division process, the iterative hierarchical division process comprising: Performing a first iteration of partitioning to generate a first cluster of image features, wherein there are N total image features in the first cluster and M image features located on the convex hull of the first cluster, N and M being positive integers; and In response to is greater than or equal to M, performing a second iteration of partitioning to partition the first cluster, wherein when When it is less than M, no further partitioning is performed on the first cluster.

53. The device according to any one of claims 41 to 51, wherein: The processor is configured to execute the instructions to enable the apparatus to divide the plurality of image features by performing an iterative hierarchical division process, the iterative hierarchical division process comprising: Performing a first iteration of partitioning to generate a first cluster of image features, wherein there are N total image features in the first cluster and M image features located on the convex hull of the first cluster, N and M being positive integers; and In response to N being greater than or equal to M*M, a second iteration of partitioning is performed to partition the first cluster, wherein when N is less than M*M, no further partitioning is performed on the first cluster.

54. The apparatus of claim 52 or 53, wherein: The M image features located on the convex hull of the first cluster are used as initial center points for dividing the first cluster.

55. A non-transitory computer-readable medium storing a set of instructions, the set of instructions being executed by one or more processors of a device to cause the device to perform a method, the method comprising: receiving a data stream, the data stream comprising a plurality of parameters respectively associated with a plurality of image features; dividing the plurality of image features into one or more clusters; determining a reference value for at least one of the one or more clusters; as well as Determine corresponding attributes of the plurality of image features, wherein the attributes of the image features are determined based on: parameters associated with the corresponding image features, and Reference value of the corresponding cluster.

56. The medium of claim 55, wherein: The image features are scale invariant feature transform (SIFT) key points on the image.

57. A medium according to claim 55 or 56, wherein: The attributes of the plurality of image features include at least one of the following: the scales of the plurality of image features; or The directions of the plurality of image features.

58. A medium according to any one of claims 55 to 57, wherein The segmentation is based on positions of the plurality of image features on the image.

59. A medium according to any one of claims 55 to 58, wherein The plurality of image features are divided using a PAM algorithm divided around a center point.

60. A medium according to any one of claims 55 to 59, wherein The image features in a cluster include a center point feature, and the reference value of the cluster is determined as the value of the parameter associated with the corresponding center point feature.

61. A medium according to any one of claims 55 to 60, wherein The image features in the cluster include a center point feature and one or more non-center point features, wherein determining the attributes of the plurality of image features includes: determining the attribute of the center point feature to have a value equal to a reference value of the corresponding cluster; and The attribute of the non-center point feature is determined to have a value equal to the sum of: the value of the parameter associated with the corresponding non-midpoint feature, and Reference value of the corresponding cluster.

62. The medium of claim 61, wherein: The center point feature of the cluster is the center point of the image feature in the corresponding cluster.

63. The medium of any one of claims 55 to 62, wherein the set of instructions causes the apparatus to perform: Before the segmentation, determining the convex hull of the plurality of image features, in, Image features on the convex hull are used as initial center points for the partitioning.

64. The medium of claim 63, wherein: The convex hull is determined using the Jarvis march algorithm.

65. A medium according to any one of claims 51 to 60, wherein The dividing includes performing an iterative hierarchical division process on the plurality of image features, wherein performing the iterative hierarchical division process includes: Performing a first iteration of partitioning to generate a first cluster of image features, wherein there are N total image features in the first cluster and M image features located on the convex hull of the first cluster, N and M being positive integers; and In response to is greater than or equal to M, performing a second iteration of partitioning to partition the first cluster, wherein when When it is less than M, no further partitioning is performed on the first cluster.

66. A medium according to any one of claims 51 to 60, wherein The dividing includes performing an iterative hierarchical division process on the plurality of image features, wherein performing the iterative hierarchical division process includes: Performing a first iteration of partitioning to generate a first cluster of image features, wherein there are N total image features in the first cluster and M image features located on the convex hull of the first cluster, N and M being positive integers; and In response to N being greater than or equal to M*M, a second iteration of partitioning is performed to partition the first cluster, wherein when N is less than M*M, no further partitioning is performed on the first cluster.

67. A medium according to claim 65 or 66, wherein The M image features located on the convex hull of the first cluster are used as initial center points for dividing the first cluster.

68. A non-transitory computer readable medium storing a set of instructions, the set of instructions being executed by one or more processors of a device to cause the device to perform a method comprising: dividing a plurality of image features into one or more clusters; determining a reference value for at least one of the one or more clusters; as well as Generate a plurality of parameters respectively associated with the plurality of image features, wherein each parameter of the plurality of parameters is generated based on: the value of the attribute of the corresponding image feature, and Reference value of the corresponding cluster.

69. The medium of claim 68, wherein the set of instructions causes the apparatus to perform: The plurality of parameters are sent in a data stream to a decoder.

70. The medium of claim 68 or 69, wherein: The image features are scale invariant feature transform (SIFT) key points on the image.

71. A medium according to any one of claims 68 to 70, wherein The attributes of the plurality of image features include at least one of the following: the scales of the plurality of image features; or The directions of the plurality of image features.

72. A medium according to any one of claims 68 to 71, wherein The segmentation is based on positions of the plurality of image features on the image.

73. A medium according to any one of claims 68 to 72, wherein The plurality of image features are divided using a PAM algorithm divided around a center point.

74. A medium according to any one of claims 68 to 73, wherein The image features in a cluster include a center point feature, and a reference value of the cluster is determined as a value of an attribute corresponding to the center point feature.

75. A medium according to any one of claims 68 to 74, wherein The image features in a cluster include a center point feature and one or more non-center point features, where: The parameter associated with the center point feature has a value equal to a reference value of the corresponding cluster; and The parameter associated with the non-center point feature has a value equal to the difference between: the value of the attribute at the corresponding non-central point, and Reference value of the corresponding cluster.

76. The medium of claim 75, wherein: The center point feature of the cluster is the center point of the image feature in the corresponding cluster.

77. The medium of any one of claims 68 to 76, wherein the set of instructions causes the apparatus to perform: Before the segmentation, determining the convex hull of the plurality of image features, in, Image features on the convex hull are used as initial center points for the partitioning.

78. The medium of claim 77, wherein: The convex hull is determined using the Jarvis march algorithm.

79. A medium according to any one of claims 68 to 78, wherein The dividing includes performing an iterative hierarchical division process on the plurality of image features, wherein performing the iterative hierarchical division process includes: Performing a first iteration of partitioning to generate a first cluster of image features, wherein there are N total image features in the first cluster and M image features located on the convex hull of the first cluster, N and M being positive integers; and In response to is greater than or equal to M, performing a second iteration of partitioning to partition the first cluster, wherein when When it is less than M, no further partitioning is performed on the first cluster.

80. A medium according to any one of claims 68 to 78, wherein The dividing includes performing an iterative hierarchical division process on the plurality of image features, wherein performing the iterative hierarchical division process includes: Performing a first iteration of partitioning to generate a first cluster of image features, wherein there are N total image features in the first cluster and M image features located on the convex hull of the first cluster, N and M being positive integers; and In response to N being greater than or equal to M*M, a second iteration of partitioning is performed to partition the first cluster, wherein when N is less than M*M, no further partitioning is performed on the first cluster.

81. The medium of claim 79 or 80, wherein: The M image features located on the convex hull of the first cluster are used as initial center points for dividing the first cluster.

Images