Machine video feature coding method and device, electronic equipment and storage medium

Abstract

Embodiments of the present disclosure provide a machine video feature encoding method, a decoding method, an apparatus, a device, a medium and a program product. The method converts a machine video feature group into a feature tensor graph group, and respectively performs two rounds of rearrangement on the feature tensor graph and each feature tensor in the feature tensor graph to generate a second feature graph. Meanwhile, the position change information of the rearrangement is recorded through a serial number matrix. Based on the second feature graph and the serial number matrix, a compressed code stream is obtained. Through the rearrangement process, the generated second feature graph is suitable for encoding and decoding by using a video coding algorithm, thereby saving bandwidth, improving transmission speed and reducing the cost of related processing without changing the existing video codec hardware system.

Description

Technical Field

[0001] This disclosure relates to the field of video encoding and decoding technology, and more specifically, to a machine video feature encoding and decoding method, apparatus, electronic device, storage medium, and computer program product. Background Technology

[0002] Machine vision feature coding technology aims to compress and transmit features by extracting and reducing their dimensions, combined with existing video codec standards (such as VVC, HEVC, and AVC). However, these methods lack optimized arrangement of features for different channels, resulting in limited compression efficiency. Compared to traditional video data, feature data is sparse and redundant, making it difficult to achieve optimal compression results by directly applying video codecs. Specifically, the similarity between feature channels is low, making it difficult to stitch them into a continuous video; the spatial redundancy pattern is complex, increasing the encoding difficulty; and the amount of mid-level feature data is enormous, far exceeding the original input.

[0003] To overcome the aforementioned challenges and fully utilize existing video compression frameworks and hardware, it is necessary to modify the data format of features. This includes designing specific quantization and permutation strategies to reduce the amount of data transmitted and improving methods for removing spatial redundancy of features. In this way, compression performance can be improved, ensuring efficient transmission of feature maps while maintaining high reconstruction quality, thereby supporting a wider range of application scenarios. Summary of the Invention

[0004] This disclosure provides a machine video feature encoding / decoding method, apparatus, electronic device, storage medium, and computer program product.

[0005] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0006] According to one aspect of this disclosure, a machine video feature encoding method is provided, the method comprising: converting a machine video feature set into a feature tensor map set; the machine video feature set corresponding to at least one video frame; the machine video feature set comprising N... w ×N h Each video frame has N channels; the feature tensor map set includes N channels corresponding to each video frame. w ×N hThe video stream consists of W×H feature tensor maps for each channel. The feature tensor maps are grouped by channel, and a reference feature map is generated based on the feature tensor maps of the same channel. The reference feature map corresponds to a channel. The feature tensor maps are arranged according to the similarity between the feature tensor maps and the reference feature map to obtain a first feature map. The first feature map corresponds to each video frame. A sequence matrix is ​​generated based on the arrangement position of the feature tensor maps in the first feature map. The sequence matrix corresponds to each video frame. The feature tensors in each feature tensor map are stitched together to obtain a second feature map. A compressed bitstream is obtained based on the second feature map and the sequence matrix.

[0007] In an exemplary embodiment, generating a reference feature map based on feature tensor maps of the same channel includes: calculating the average value of feature tensors corresponding to different video frames in the same channel; and generating the reference feature map based on the average value of the feature tensors.

[0008] In an exemplary embodiment, arranging the feature tensor map according to the similarity between the feature tensor map and the reference feature map to obtain a first feature map includes: sorting the feature tensor map according to the similarity between the feature tensor map and the reference feature map, arranging the feature tensor map row by row from left to right to obtain the first feature map.

[0009] In an exemplary embodiment, arranging the feature tensor maps according to the similarity between the feature tensor maps and the reference feature maps to obtain a first feature map includes: arranging the first feature tensor map with the highest similarity to the reference feature map in the upper left corner of the first feature map; determining a second feature tensor map based on its similarity to the feature tensor map above it and arranging it on the far left of its row; determining a third feature tensor map based on its similarity to the feature tensor map to the left and arranging it on the right side of the feature tensor map to the left; and repeating the above process to obtain the first feature map.

[0010] In an exemplary embodiment, arranging the feature tensor maps according to the similarity between the feature tensor maps and the reference feature maps to obtain a first feature map includes: arranging the first feature tensor map with the highest similarity to the reference feature map in the upper left corner of the first feature map; determining a second feature tensor map based on its similarity to the feature tensor map above it and arranging it on the far left of its row; determining a third feature tensor map based on its similarity to the feature tensor maps to the left and above it and arranging it on the right side of the feature tensor map to the left; and repeating the above process to obtain the first feature map.

[0011] In an exemplary embodiment, arranging the feature tensor maps according to the similarity between the feature tensor maps and the reference feature maps to obtain a first feature map includes: arranging the first feature tensor map with the highest similarity to the reference feature map in the upper left corner of the first feature map; determining a second feature tensor map based on its similarity to the feature tensor map above it and arranging it on the far left of its row; determining a third feature tensor map based on its similarity to the feature tensor maps to the left, above, and upper left and arranging it to the right of the feature tensor map to the left; and repeating the above process to obtain the first feature map.

[0012] In an exemplary embodiment, the similarity between the feature tensor map and the reference feature map includes: comparing each feature tensor in the feature tensor map with the reference feature value at the corresponding position in the reference feature map; binary labeling the position according to the magnitude relationship between the feature tensor and the reference feature value to obtain a binary feature map corresponding to the feature tensor map; and calculating the similarity between the two feature tensors based on the Hamming distance between the binary feature maps corresponding to the two feature tensors.

[0013] In an exemplary embodiment, the step of stitching together the feature tensors in each of the feature tensor maps to obtain a second feature map includes: rearranging the feature tensors at corresponding positions in adjacent feature tensor maps of the first feature map to obtain the second feature map.

[0014] According to another aspect of this disclosure, a machine video feature decoding method is provided, the method comprising: decoding a compressed bitstream to obtain a second feature map and a sequence matrix; the second feature map and the sequence matrix corresponding to each video frame; the sequence matrix being used to record the arrangement position information of feature tensors in the first feature map; performing a descrambling process on the second feature map to obtain the first feature map; rearranging the first feature map according to the sequence matrix to obtain a feature tensor map group; the feature tensor map group comprising N corresponding to each video frame. w ×N h A feature tensor map of each channel; each feature tensor map includes W×H feature tensors; the feature tensor map group is converted into a machine video feature group.

[0015] In an exemplary embodiment, the step of unstitching the second feature map to obtain the first feature map includes: rearranging the feature tensors at corresponding positions in adjacent feature tensor maps of the second feature map to obtain the first feature map.

[0016] In an exemplary embodiment, rearranging the first feature map according to the sequence matrix to obtain a feature tensor map group includes: obtaining a fourth feature tensor map from the first feature map; determining the channel information corresponding to the fourth feature tensor map according to the sequence matrix; the channel information includes at least: video frame information and channel position information; and inserting the fourth feature tensor map into the feature tensor map group according to the video frame information and channel position information.

[0017] According to another aspect of this disclosure, a machine video feature encoding apparatus is provided, comprising: a feature tensor map group encoding module configured to convert a machine video feature group into a feature tensor map group; the machine video feature group corresponds to at least one video frame; the machine video feature group includes N... w ×N h Each video frame has N channels; the feature tensor map set includes N channels corresponding to each video frame. w ×N h A feature tensor map of each channel; each feature tensor map includes W×H feature tensors; a reference feature map module is configured to group the feature tensor maps by channel and generate a reference feature map based on the feature tensor maps of the same channel; the reference feature map corresponds to a channel; a first feature map encoding module is configured to arrange the feature tensor maps according to the similarity between the feature tensor maps and the reference feature maps to obtain a first feature map; the first feature map corresponds to each video frame; a sequence matrix module is configured to generate a sequence matrix according to the arrangement position of the feature tensor maps in the first feature map; the sequence matrix corresponds to each video frame; a second feature map encoding module is configured to stitch the feature tensors in each of the feature tensor maps to obtain a second feature map; a compressed bitstream module is configured to obtain a compressed bitstream based on the second feature map and the sequence matrix.

[0018] According to another aspect of this disclosure, a machine video feature decoding apparatus is provided, comprising: a decoding bitstream module configured to decode a compressed bitstream to obtain a second feature map and a sequence number matrix; the second feature map and the sequence number matrix corresponding to each video frame; the sequence number matrix being used to record the arrangement position information of feature tensors in the first feature map; a first feature map decoding module configured to perform desizing and stitching processing on the second feature map to obtain the first feature map; and a feature tensor map group decoding module configured to rearrange the first feature map according to the sequence number matrix to obtain a feature tensor map group; the feature tensor map group including N corresponding to each video frame. w ×N h A feature tensor map of each channel; each feature tensor map includes W×H feature tensors; a machine video feature group module is configured to convert the feature tensor map group into a machine video feature group.

[0019] According to another aspect of this disclosure, an electronic device is provided, comprising: one or more processors; and a storage device configured to store one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement the machine video feature encoding method or the machine video feature decoding method as described in the above embodiments.

[0020] According to another aspect of this disclosure, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program that, when executed by a processor, implements the machine video feature encoding method or the machine video feature decoding method as described in the above embodiments.

[0021] According to another aspect of this disclosure, a computer program product is provided, including a computer program / signaling, characterized in that, when the computer program / signaling is executed by a processor, it implements the machine video feature encoding method or the machine video feature decoding method as described in the above embodiments.

[0022] The machine video feature encoding method provided in this disclosure converts a machine video feature set into a feature tensor map set, and performs two rounds of rearrangement on the feature tensor map and each feature tensor in the feature tensor map to generate a second feature map. The positional changes of this rearrangement are recorded using an index matrix. Based on the second feature map and the index matrix, a compressed bitstream is obtained. This method, through this rearrangement process, makes the generated second feature map suitable for encoding and decoding using video encoding and decoding algorithms, thereby saving bandwidth, increasing transmission speed, and reducing related processing costs without changing the existing video codec hardware architecture.

[0023] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0024] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0025] Figure 1 A flowchart of a machine video feature encoding method according to an embodiment of the present disclosure is shown;

[0026] Figure 2 A flowchart illustrating the feature tensor graph arrangement process according to an embodiment of this disclosure is shown;

[0027] Figure 3 A schematic diagram illustrating the feature tensor graph arrangement process according to an embodiment of the present disclosure is shown;

[0028] Figure 4 A flowchart of a similarity calculation method according to an embodiment of this disclosure is shown;

[0029] Figure 5 A flowchart illustrating the suturing process according to an embodiment of this disclosure is shown;

[0030] Figure 6 A flowchart of a machine video feature decoding method according to an embodiment of the present disclosure is shown;

[0031] Figure 7 A flowchart illustrating the first feature map rearrangement process according to an embodiment of this disclosure is shown;

[0032] Figure 8 A schematic diagram of the unstitching process according to an embodiment of the present disclosure is shown;

[0033] Figure 9 A schematic diagram of the machine video feature decoding process according to an embodiment of the present disclosure is shown;

[0034] Figure 10 A schematic diagram of the structure of a machine video feature encoding apparatus according to an embodiment of the present disclosure is shown;

[0035] Figure 11 A schematic diagram of the structure of a machine video feature decoding apparatus according to an embodiment of the present disclosure is shown;

[0036] Figure 12 A schematic diagram of the structure of an electronic device suitable for implementing exemplary embodiments of the present disclosure is shown. Detailed Implementation

[0037] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0038] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0039] It should be noted that the ordinal numbers such as "first" and "second" mentioned in the embodiments of this disclosure are used to distinguish multiple objects, and are not used to limit the order, timing, priority or importance of multiple objects. Furthermore, the descriptions of "first" and "second" do not limit the objects to necessarily being different.

[0040] Figure 1 A flowchart illustrating a machine video feature encoding method according to an embodiment of this disclosure is shown. Figure 1 As shown, the machine video feature encoding method may include the following steps.

[0041] In step S110, the machine video feature set is converted into a feature tensor map set; the machine video feature set corresponds to at least one video frame; the machine video feature set includes N w ×N h Each video frame has N channels; the feature tensor map set includes N channels corresponding to each video frame. w ×N h A feature tensor map of each channel; each feature tensor map contains W×H feature tensors.

[0042] In the fields of machine vision and deep learning, unlike humans who directly process visual images or videos, machines need to extract machine video feature sets from traditional video information, and then perform analysis and processing based on these video features. This process first involves analyzing each frame of the input video to extract feature vectors that describe the content of the frame. These feature vector sets constitute the machine video feature set. The feature set corresponding to each video frame contains N... w ×N h Each channel is a different dimension or layer that constitutes the feature map of each video frame. Each channel carries specific type of feature information about that frame (such as edges, texture, or color distribution), and together they form a multi-channel feature tensor map. Where N... w and N h These represent the number of channels in the width and height directions of a feature map of size W×H, respectively.

[0043] Next, to better utilize existing image and video compression techniques, such as VVC (Versatile Video Coding), HEVC (High Efficiency Video Coding), or AVC (Advanced Video Coding), these feature sets need to be transformed into feature tensor map sets more suitable for compression algorithms. Each video frame corresponds to a feature tensor map set consisting of N... w ×N hThe feature tensor map consists of W × H channels, and each feature tensor map contains W × H feature tensors. Here, W and H represent the spatial dimensions of each feature map, that is, the width and height of the feature map. Each feature tensor not only retains the information of each channel in the original feature group, but also enhances the correlation between different channels through a specific arrangement, making the feature data easier for compression algorithms to recognize and process. This transformation helps improve compression efficiency while maintaining the effectiveness and accuracy of feature representation.

[0044] In an exemplary embodiment, the process of converting a set of machine video features into a set of feature tensors may include:

[0045] The input raw video feature set is cropped, scaled, and quantized to integers, preferably using the following formula:

[0046]

[0047] Where x is the input feature value, c max c min These are the maximum and minimum values ​​to be cropped, respectively. N is the preset quantization level, and round(·) represents rounding the result to an integer. clp It is a quantitative result.

[0048] The channel mean distribution of features in all frames of the statistically quantized video feature group is analyzed. For channels whose mean value exceeds a preset threshold but whose data frequency is lower than a preset threshold, a reference channel is used to replace the channel containing the mean value. Preferably, the mean value of the feature in each channel is calculated using the following formula:

[0049]

[0050] Where x is the input feature value, t represents a frame of the video, c represents the channel number, w represents the width of the video feature, h represents the height of the video feature, F represents the total number of frames of the video feature, Widh represents the total width of the video feature, and Height represents the total height of the video feature.

[0051] In step S120, the feature tensor maps are grouped according to channels, and a reference feature map is generated based on the feature tensor maps of the same channel; the reference feature map corresponds to the channel.

[0052] The feature tensor map set obtained through the above steps corresponds to each video frame, and includes features corresponding to N. w ×N h A feature tensor map of each channel. Each feature tensor map contains W×H feature tensors.

[0053] In this embodiment, the feature tensors of all video frames are grouped according to their channel numbers, such that feature tensors of the same channel are grouped together. For example, the first channel of all video frames is grouped into one group, the second channel into another, and so on. Based on the feature tensors of the same channel in each group, a corresponding reference feature map is generated by calculating statistics (such as mean, median, or maximum). The reference feature map corresponds to a specific channel and comprehensively reflects the typical feature performance of that channel across all video frames.

[0054] In an exemplary embodiment, the average value of the feature tensors is calculated based on the feature tensors corresponding to different video frames in the same channel; the reference feature map is then generated based on the average value of the feature tensors. By calculating the average value of the feature tensors of different video frames in the same channel, information from each frame can be synthesized to extract a reference feature map representing the typical characteristics of the channel, which can then be used to evaluate each feature tensor map subsequently.

[0055] In step S130, the feature tensor map is arranged according to the similarity between the feature tensor map and the reference feature map to obtain a first feature map; the first feature map corresponds to each video frame.

[0056] In this embodiment, to reduce channel redundancy of depth features for video compression tasks in machine vision without altering the original model structure, the feature tensor map needs to be rearranged. In this disclosure, the feature tensor map is sorted at the block level based on the similarity between the feature tensor map and the reference feature map, in accordance with the characteristics of existing video compression algorithms.

[0057] First, the similarity between the feature tensor map corresponding to each video frame in the same channel and the reference feature map of the corresponding channel is calculated. This process typically involves comparing the feature distributions between the two, possibly using metrics such as cosine similarity or Euclidean distance to quantify the similarity. Through this comparison, the degree of similarity between each feature tensor map and the reference feature map can be determined. Next, the feature tensors are sorted based on the aforementioned similarity. Then, based on this sorting position, the feature tensors are inserted into specific positions within the first feature map, thus obtaining the first feature map. This first feature map corresponds to each video frame. That is, feature tensors corresponding to different channels of the same video frame are arranged within the same first feature map.

[0058] In an exemplary embodiment, the arrangement process of the feature tensor maps may include: sorting the feature tensor maps according to their similarity to a reference feature map, and arranging the feature tensor maps row by row from left to right to obtain the first feature map. In this process, the feature tensor maps are arranged row by row from left to right according to their similarity. That is, sorted by similarity, the feature tensor maps are first arranged from left to right in the first row until the first row is full, and then the feature tensor maps are arranged from left to right in the second row until the second row is full. This process is repeated, arranging the feature tensor maps row by row from top to bottom until the entire first feature map is filled, thus obtaining the first feature map. The width of the first feature map is W×N. w The height is H×N h .

[0059] In step S140, an index matrix is ​​generated based on the arrangement position of the feature tensor map in the first feature map; the index matrix corresponds to each video frame.

[0060] In this embodiment of the present disclosure, during the aforementioned step S130 feature tensor map arrangement process, a corresponding sequence number matrix is ​​generated simultaneously based on the arrangement position of each feature tensor map in the first feature map. This sequence number matrix is ​​used to record the correspondence between the channel information of the feature tensor map in the feature tensor map group and its position information in the first feature map. This sequence number matrix allows for the decoding and recovery of the feature tensor map's position during the decoding process. The channel information can be the sequence number of the feature tensor map, or it can include video frame information and channel position information. The video frame information and channel position information are used to represent the video frame corresponding to the feature tensor map and its channel position in the feature tensor map group. This sequence number matrix and the first feature map... Figure 1 One corresponds to another, and also to each video frame.

[0061] In step S150, the feature tensors in each of the feature tensor maps are stitched together to obtain a second feature map.

[0062] In this embodiment of the disclosure, for the purpose of video compression for machine vision tasks, not only is block-level reordering as described in step S130 above necessary, but also pixel-level reordering within blocks is required. Here, each pixel within a block is equivalent to each feature tensor in the feature tensor map.

[0063] In this embodiment, based on the first feature map obtained above, the feature tensors in each feature tensor map are stitched together, i.e., the positions of the feature tensors are rearranged to obtain a second feature map. This stitching process is a technique for enhancing feature map representation; it creates larger feature blocks by rearranging and combining pixels from adjacent feature maps. In video coding or deep learning applications, this process helps improve the consistency and continuity of feature representation, thereby improving compression efficiency or model performance.

[0064] On the one hand, this stitching process can effectively utilize the enhanced correlation between the sorted feature maps, ensuring that the generated feature blocks are more coherent, which is beneficial for subsequent encoding or decoding operations. On the other hand, this stitching process can also effectively reduce the amount of data transmitted without significantly increasing computational complexity, thereby improving encoding efficiency.

[0065] In step S160, a compressed bitstream is obtained based on the second feature map and the sequence number matrix.

[0066] In this embodiment of the disclosure, encoding is performed based on the aforementioned second feature map and sequence number matrix to generate a compressed bitstream.

[0067] In the exemplary embodiment, since the machine video feature group has undergone the aforementioned feature map transformation, the resulting second feature map is suitable for encoding and decoding using existing video codec algorithms, achieving good bitrate compression. Based on this, the aforementioned encoding process can be performed using existing video codecs. This solution saves bandwidth and increases transmission speed without changing the existing video codec hardware architecture, while maintaining the accuracy of machine vision tasks, resulting in low energy consumption and low cost.

[0068] The machine video feature encoding method provided in this disclosure converts a machine video feature set into a feature tensor map set, and performs two rounds of rearrangement on the feature tensor map and each feature tensor in the feature tensor map to generate a second feature map. The positional changes of this rearrangement are recorded using an index matrix. Based on the second feature map and the index matrix, a compressed bitstream is obtained. This method, through this rearrangement process, makes the generated second feature map suitable for encoding and decoding using video encoding and decoding algorithms, thereby saving bandwidth, increasing transmission speed, and reducing related processing costs without changing the existing video codec hardware architecture.

[0069] Figure 2 A flowchart illustrating the feature tensor graph arrangement process according to an embodiment of the present disclosure is shown. Figure 3 A schematic diagram illustrating the arrangement process of the feature tensor graphs according to an embodiment of this disclosure is shown. Figure 2 , 3 As shown, in Figure 1Based on the machine video feature encoding method shown, step S130 may include the following steps.

[0070] In step S210, the first feature tensor map with the highest similarity to the reference feature map is arranged in the upper left corner of the first feature map.

[0071] In this embodiment of the disclosure, the arrangement of the feature tensor map can be achieved using various methods. Here, through... Figure 2 , 3 The illustrated embodiment describes feasible arrangement methods.

[0072] In this embodiment of the disclosure, the arrangement process arranges the feature tensors from top to bottom and from left to right. First, the feature tensor with the highest similarity to the reference feature map is selected as the first feature tensor and arranged in the upper left corner of the first feature map, i.e., the leftmost position of the first row. The position of this first feature tensor can be denoted as P. 0，0 .

[0073] In this embodiment, after determining the first feature tensor map in the upper left corner, the remaining feature tensors in the first row are arranged based on the first feature tensor map. The next right-side feature map to be arranged is determined based on its similarity to the feature maps already placed on the left, ensuring it is as close as possible to the adjacent left-side feature maps. This process continues to arrange to the right until all feature maps in the row are placed. This process ensures that the local similarity between feature tensors is maximized, thus providing a good foundation for subsequent compression.

[0074] In step S220, a second feature tensor is determined based on its similarity to the feature tensor above, and is arranged on the far left of its row.

[0075] In this embodiment of the disclosure, after all feature maps in a row are arranged, the leftmost feature map of the next row depends on the similarity evaluation of the feature maps above it. The feature tensor map with the highest similarity to the leftmost feature tensor map of the previous row is selected as the second feature tensor map and arranged on the leftmost side of the next row as the starting feature map of the next row.

[0076] like Figure 3 As shown in Figure a, assume the leftmost feature tensor graph in the top row is P. m-1，n-1 After the (n-1)th row is sorted, based on the feature tensor P... m-1，n-1 The similarity is used to determine the leftmost feature tensor P in the nth row. m-1，n .

[0077] In step S230, a third feature tensor is determined based on its similarity to the left feature tensor and is arranged to the right of the left feature tensor.

[0078] In this embodiment, a third feature tensor is determined based on its similarity to the feature map already placed on the left, ensuring it is as close as possible to the adjacent left feature map, and is placed to the right of the left feature tensor. This process continues to the right until all feature tensors in the row have been placed.

[0079] like Figure 3 As shown in Figure a, in determining the leftmost feature tensor P in the nth row... m-1，n Subsequently, based on the feature tensor P... m-1，n Similarity is used to determine the right-hand feature tensor P. m，n .

[0080] By repeating steps S220 and S230 until the entire first feature map is filled, the first feature map is obtained. It should be noted that the execution order of steps S220 and S230 can also be reversed, i.e., arranging rows first and then columns, or arranging columns first and then rows, both of which should be within the protection scope of this disclosure.

[0081] At the same time, when arranging non-opening feature tensors in a row, the referenced feature tensors can also be adjusted as needed.

[0082] In an exemplary embodiment, such as Figure 3 As shown in Figure b, the third feature tensor is determined based on its similarity to the feature maps already placed to the left and above, ensuring it is as close as possible to the adjacent feature maps to the left and above, and is placed to the right of the left feature tensor. This process continues to the right until all feature tensors in the row are placed. As shown in the figure, when determining the left feature tensor P in the nth row... m-1，n and the feature tensor P above m，n-1 Subsequently, based on the feature tensor P... m-1，n and P m，n-1 The similarity is used to determine the right-side feature tensor P. m，n .

[0083] In an exemplary embodiment, such as Figure 3 As shown in Figure c, the third feature tensor is determined based on its similarity to the feature maps already placed to the left, top, and top-left, ensuring it is as close as possible to the adjacent left, top, and top-left feature maps, and placed to the right of the left feature tensor. This process continues to the right until all feature tensors in the row are placed. As shown in the figure, when determining the left feature tensor P in the nth row... m-1，n The upper feature tensor P m，n-1 and the feature tensor P in the upper left corner m-1，n-1 Subsequently, based on the feature tensor P... m-1，n P m，n-1 and P m-1，n-1The similarity is used to determine the right-side feature tensor P. m，n .

[0084] The machine video feature encoding method provided in this disclosure offers multiple ways to arrange feature tensors. By calculating the similarity between adjacent feature tensors, feature tensors with high similarity are arranged in close proximity. This method improves the correlation and continuity of adjacent regions in the feature maps, thereby improving compression efficiency or model performance, and is suitable for encoding processing with existing video encoding and decoding algorithms.

[0085] Figure 4 A flowchart illustrating a similarity calculation method according to an embodiment of this disclosure is shown. Figure 4 As shown, in Figure 1 Based on the machine video feature encoding method shown, the similarity calculation method between the feature tensor map and the reference feature map in step S130 may include the following steps.

[0086] In step S410, each feature tensor in the feature tensor map is compared with the reference feature value at the corresponding position in the reference feature map.

[0087] In this embodiment of the disclosure, a reference feature value is compared between the corresponding position of the feature tensor map of each channel and the reference feature map. That is, the feature tensor map of the feature tensor map is compared one by one with the reference feature value at the corresponding position in the reference feature map. This process ensures that details at the pixel level are taken into account, thereby accurately measuring the similarity between the two.

[0088] In step S420, the position is binary-labeled according to the relationship between the feature tensor and the reference feature value to obtain a binary feature map corresponding to the feature tensor map.

[0089] In this embodiment, the corresponding positions are binary-labeled based on the comparison between the feature tensor and the reference feature value. Specifically, if the feature tensor is greater than the corresponding reference feature value, it is labeled as 1; otherwise, it is labeled as 0. This label is then recorded at the corresponding position. In this way, the feature tensor map can be converted into binary form, forming the corresponding binary feature map.

[0090] In step S430, the similarity between the two feature tensors is calculated based on the Hamming distance between the binary feature maps corresponding to the two feature tensors.

[0091] In this embodiment of the disclosure, binary feature maps corresponding to each feature tensor map can be generated through step S420. Based on the binary feature maps corresponding to any two feature tensors, the similarity between the two feature tensors is calculated.

[0092] Specifically, we first compare their corresponding binary feature maps. We quantify their differences by calculating the Hamming distance between these two binary feature maps, that is, by counting the number of different values ​​at corresponding positions. The Hamming distance refers to the number of different characters at the same position in two strings of equal length, and is commonly used to measure the differences between binary strings or feature vectors. Based on the Hamming distance between the two feature tensors, we measure the overall similarity between them. The smaller the Hamming distance, the smaller the difference between the two feature tensors, and the higher the similarity. Conversely, the larger the distance, the lower the similarity.

[0093] The similarity calculation method provided in this disclosure can be used to calculate the similarity between any two feature tensor maps, or the similarity between a feature tensor map and a reference feature map. Based on the relevant similarity comparison, the aforementioned permutation operation can be performed.

[0094] Figure 5 A flowchart illustrating the suturing process according to an embodiment of this disclosure is shown. Figure 5 As shown, in Figure 1 Based on the machine video feature encoding method shown, step S150 may include the following steps.

[0095] In step S510, the feature tensors at corresponding positions in adjacent feature tensor maps of the first feature map are rearranged to obtain the second feature map.

[0096] As mentioned above, based on the first feature map obtained above, the feature tensors in each feature tensor map are stitched together, that is, the positions of the feature tensors are rearranged to obtain the second feature map.

[0097] In this embodiment of the disclosure, for each feature tensor in the first feature map, it is reordered with the feature tensors at adjacent positions according to a certain optimization rule. This process ensures that tensors with similar features can be clustered more closely together, thereby enhancing the local correlation between feature maps. Through this rearrangement, the consistency and coherence of feature representations can be improved, which helps to improve the efficiency and accuracy of subsequent encoding or analysis steps, ultimately forming an optimized second feature map.

[0098] In an exemplary embodiment, the rearrangement process can be performed using pixel-level stitching at twice the size, merging adjacent feature tensors into a larger feature block, specifically a feature block four times the size of the original. This can specifically include the following process:

[0099] Use P m，n Let p represent a feature tensor graph, where (m, n) represent its x and y coordinates in the first feature graph. 2m，2n [σ][τ] Characteristic Tensor P2m，2n The characteristic tensor with x and y axes of σ and T is p. 2m-1，2n [σ][τ],p 2m，2n-1 [σ][τ],p 2m-1，2n-1 [σ][τ],p 2m，2n [σ][τ] are rearranged and combined into a feature block B of size 4*P.

[0100] With B m，n This represents a feature block with coordinates (m, n) after rearrangement, denoted by b. m，n [σ][τ] represents feature block B m，n If the characteristic tensor with x and y coordinates of σ and τ is given, then b m，n [σ][τ] and p 2m，2n The correspondence between [σ][τ] is as follows:

[0101] b m，n [2σ][2τ]=p 2m，2n [σ][τ]

[0102] b m，n [2σ+1][τ]=p 2m-1，2n [σ][τ]

[0103] b m，n [2σ][2τ+1]=p 2m，2n-1 [σ][τ]

[0104] b m，n [2σ+1][2τ+1]=p 2m-1，2n-1 [σ][τ]

[0105] It should be noted that the above is only one feasible exemplary suturing method. Depending on actual needs, other rearrangement methods or suturing multiples may also be used, all of which should be considered within the scope of protection of this disclosure.

[0106] Figure 6 A flowchart illustrating a machine video feature decoding method according to an embodiment of this disclosure is shown. Figure 6 As shown, the machine video feature decoding method may include the following steps.

[0107] In step S610, the compressed bitstream is decoded to obtain a second feature map and a sequence number matrix; the second feature map and the sequence number matrix correspond to each video frame; the sequence number matrix is ​​used to record the arrangement position information of the feature tensor map in the first feature map.

[0108] As mentioned earlier, the video feature group has undergone the aforementioned feature map transformation, making it suitable for encoding and decoding using existing video codec algorithms. Therefore, after receiving the compressed bitstream, the decoding end can perform the above decoding process using an existing video codec to obtain the second feature map and the sequence number matrix. This second feature map and sequence number matrix are the same as those described in the encoding method section and will not be repeated here. The second feature map and sequence number matrix correspond to each video frame. The sequence number matrix records the arrangement position information of the feature tensor map in the first feature map. The sequence number matrix also records the correspondence between the channel information of the feature tensor map in the feature tensor map group and its position information in the first feature map.

[0109] In step S620, the second feature map is unstitched to obtain the first feature map.

[0110] In this embodiment, the decoding end performs desizing on the obtained second feature map. This desizing is the reverse of the stitching process described at the encoding end, and its specific process will not be described here. The decoding end can determine the stitching algorithm used at the encoding end based on a pre-agreed agreement or a specific flag indication, and then adopt the corresponding desizing algorithm. By performing desizing on the second feature map, a first feature map is obtained. This first feature map is the same as the first feature map described in the encoding method above, and will not be repeated here.

[0111] In an exemplary embodiment, such as Figure 8 As shown, the unstitching process may include: rearranging the feature tensors at corresponding positions in adjacent feature tensor maps of the second feature map to obtain the first feature map.

[0112] In step S630, the first feature map is rearranged according to the sequence number matrix to obtain a feature tensor map group; the feature tensor map group includes N corresponding to each video frame. w ×N h A feature tensor map of each channel; each feature tensor map contains W×H feature tensors.

[0113] In this embodiment, since the sequence matrix records the correspondence between the channel information of the feature tensor map in the feature tensor map group and the position information in the first feature map, the first feature map can be rearranged according to the position information recorded in the sequence matrix. This rearrangement process is the reverse of step S130 in the aforementioned encoding method, and its specific processing will not be described here. As mentioned above, the rearranged feature tensor map group includes N corresponding to each video frame. w ×N hA feature tensor map of each channel. Each feature tensor map contains W×H feature tensors.

[0114] In step S640, the feature tensor map group is converted into a machine video feature group.

[0115] In this embodiment, the feature tensor map group is transformed to obtain the corresponding machine video feature group. This transformation process is the reverse of step S110 in the aforementioned encoding method, and its specific processing procedure will not be described here.

[0116] like Figure 9 As shown, the machine video feature decoding process described above decodes the compressed data in the compressed bitstream back into a machine video feature set that can be processed by the machine. Based on this machine video feature set, the machine can perform relevant video analysis and processing operations, which will not be elaborated further here.

[0117] The machine video feature decoding method provided in this disclosure recovers machine video feature groups that can be processed by a machine by decoding from a compressed bitstream using the reverse process of the encoding method. This rearrangement process makes the generated second feature map suitable for encoding and decoding using video encoding and decoding algorithms, thereby saving bandwidth, increasing transmission speed, and reducing related processing costs without changing the existing video codec hardware architecture.

[0118] Figure 7 A flowchart illustrating the first feature map rearrangement process according to an embodiment of this disclosure is shown. Figure 7 As shown, in Figure 6 Based on the machine video feature decoding method shown, step S630 may include the following steps.

[0119] In step S710, a fourth feature tensor map is obtained from the first feature map.

[0120] In this embodiment of the disclosure, a fourth feature tensor map is obtained from the first feature map. The fourth feature tensor map is any feature tensor map in the first feature map.

[0121] In step S720, the channel information corresponding to the fourth feature tensor is determined according to the sequence number matrix; the channel information includes at least: video frame information and channel position information.

[0122] In this embodiment, the sequence matrix records the correspondence between the channel information of the feature tensor in the feature tensor group and its position information in the first feature tensor. The channel information of the fourth feature tensor is determined based on its position in the first feature tensor. This channel information includes at least: video frame information and channel position information. The video frame information represents the video frame corresponding to the feature tensor. The channel position information represents the channel position of the feature tensor in the feature tensor group.

[0123] In step S730, the fourth feature tensor map is inserted into the feature tensor map group according to the video frame information and channel position information.

[0124] In this embodiment of the disclosure, the corresponding video frame and channel position of the fourth feature tensor map in the feature tensor map group can be determined based on the video frame information and channel position information. Based on this, the fourth feature tensor map can be inserted into the feature tensor map group to be generated.

[0125] By repeating the above steps S710-730 until all feature tensors in the first feature map are acquired and inserted into the feature tensor map group, the feature tensor map group is generated.

[0126] Based on the same inventive concept, this disclosure provides a machine video feature encoding device and a machine video feature decoding device, as described in the following embodiments. Since the principle by which this device embodiment solves the problem is similar to that of the above method embodiments, the implementation of this communication authentication device embodiment can be directly referred to in the above method embodiments, and repeated details will not be elaborated further.

[0127] Figure 10 A schematic diagram of the structure of a machine video feature encoding apparatus according to an embodiment of the present disclosure is shown. Figure 10 As shown, the machine video feature encoding device 1000 may include: a feature tensor map group encoding module 1010, a reference feature map module 1020, a first feature map encoding module 1030, a sequence matrix module 1040, a second feature map encoding module 1050, and a compressed bitstream module 1060.

[0128] Feature Tensor Map Group Encoding Module 1010 is configured to convert machine video feature groups into feature tensor map groups; the machine video feature groups correspond to at least one video frame; the machine video feature groups include N w ×N h Each video frame has N channels; the feature tensor map set includes N channels corresponding to each video frame. w ×N h A feature tensor map of each channel; each feature tensor map contains W×H feature tensors;

[0129] The reference feature map module 1020 is configured to group the feature tensor maps according to channels and generate reference feature maps based on the feature tensor maps of the same channel; the reference feature maps correspond to the channels.

[0130] The first feature map encoding module 1030 is configured to arrange the feature tensor map according to the similarity between the feature tensor map and the reference feature map to obtain a first feature map; the first feature map corresponds to each video frame;

[0131] The sequence number matrix module 1040 is configured to generate a sequence number matrix based on the arrangement position of the feature tensor map in the first feature map; the sequence number matrix corresponds to each video frame.

[0132] The second feature map encoding module 1050 is configured to stitch together the feature tensors in each of the feature tensor maps to obtain a second feature map.

[0133] The compressed bitstream module 1060 is configured to obtain a compressed bitstream based on the second feature map and the sequence number matrix.

[0134] In an exemplary embodiment, generating a reference feature map based on feature tensor maps of the same channel includes: calculating the average value of feature tensors corresponding to different video frames in the same channel; and generating the reference feature map based on the average value of the feature tensors.

[0135] In an exemplary embodiment, arranging the feature tensor map according to the similarity between the feature tensor map and the reference feature map to obtain a first feature map includes: sorting the feature tensor map according to the similarity between the feature tensor map and the reference feature map, arranging the feature tensor map row by row from left to right to obtain the first feature map.

[0136] In an exemplary embodiment, arranging the feature tensor maps according to the similarity between the feature tensor maps and the reference feature maps to obtain a first feature map includes: arranging the first feature tensor map with the highest similarity to the reference feature map in the upper left corner of the first feature map; determining a second feature tensor map based on its similarity to the feature tensor map above it and arranging it on the far left of its row; determining a third feature tensor map based on its similarity to the feature tensor map to the left and arranging it on the right side of the feature tensor map to the left; and repeating the above process to obtain the first feature map.

[0137] In an exemplary embodiment, arranging the feature tensor maps according to the similarity between the feature tensor maps and the reference feature maps to obtain a first feature map includes: arranging the first feature tensor map with the highest similarity to the reference feature map in the upper left corner of the first feature map; determining a second feature tensor map based on its similarity to the feature tensor map above it and arranging it on the far left of its row; determining a third feature tensor map based on its similarity to the feature tensor maps to the left and above it and arranging it on the right side of the feature tensor map to the left; and repeating the above process to obtain the first feature map.

[0138] In an exemplary embodiment, arranging the feature tensor maps according to the similarity between the feature tensor maps and the reference feature maps to obtain a first feature map includes: arranging the first feature tensor map with the highest similarity to the reference feature map in the upper left corner of the first feature map; determining a second feature tensor map based on its similarity to the feature tensor map above it and arranging it on the far left of its row; determining a third feature tensor map based on its similarity to the feature tensor maps to the left, above, and upper left and arranging it to the right of the feature tensor map to the left; and repeating the above process to obtain the first feature map.

[0139] In an exemplary embodiment, the similarity between the feature tensor map and the reference feature map includes: comparing each feature tensor in the feature tensor map with the reference feature value at the corresponding position in the reference feature map; binary labeling the position according to the magnitude relationship between the feature tensor and the reference feature value to obtain a binary feature map corresponding to the feature tensor map; and calculating the similarity between the two feature tensors based on the Hamming distance between the binary feature maps corresponding to the two feature tensors.

[0140] In an exemplary embodiment, the step of stitching together the feature tensors in each of the feature tensor maps to obtain a second feature map includes: rearranging the feature tensors at corresponding positions in adjacent feature tensor maps of the first feature map to obtain the second feature map.

[0141] Figure 11 A schematic diagram of the structure of a machine video feature decoding apparatus according to an embodiment of the present disclosure is shown. Figure 11 As shown, the machine video feature decoding device 1100 may include: a decoding bitstream module 1110, a first feature map decoding module 1120, a feature tensor map group decoding module 1130, and a machine video feature group module 1140.

[0142] The decoding stream module 1110 is configured to decode the compressed stream to obtain a second feature map and a sequence number matrix; the second feature map and the sequence number matrix correspond to each video frame; the sequence number matrix is ​​used to record the arrangement position information of the feature tensor map in the first feature map;

[0143] The first feature map decoding module 1120 is configured to perform a deswapping process on the second feature map to obtain the first feature map.

[0144] The feature tensor map group decoding module 1130 is configured to rearrange the first feature map according to the sequence number matrix to obtain a feature tensor map group; the feature tensor map group includes N corresponding to each video frame. w ×N h A feature tensor map of each channel; each feature tensor map contains W×H feature tensors;

[0145] The machine video feature group module 1140 is configured to convert the feature tensor map group into a machine video feature group.

[0146] In an exemplary embodiment, the step of unstitching the second feature map to obtain the first feature map includes: rearranging the feature tensors at corresponding positions in adjacent feature tensor maps of the second feature map to obtain the first feature map.

[0147] In an exemplary embodiment, rearranging the first feature map according to the sequence matrix to obtain a feature tensor map group includes: obtaining a fourth feature tensor map from the first feature map; determining the channel information corresponding to the fourth feature tensor map according to the sequence matrix; the channel information includes at least: video frame information and channel position information; and inserting the fourth feature tensor map into the feature tensor map group according to the video frame information and channel position information.

[0148] Figure 12 A schematic diagram of the structure of an electronic device suitable for implementing exemplary embodiments of the present disclosure is shown. Referring below... Figure 12 To describe an electronic device 1200 according to this embodiment of the present invention. Figure 12 The electronic device 1200 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0149] like Figure 12 As shown, the electronic device 1200 is manifested in the form of a general-purpose computing device. The components of the electronic device 1200 may include, but are not limited to: at least one processing unit 1210, at least one storage unit 1220, a bus 1230 connecting different system components (including storage unit 1220 and processing unit 1210), and a display unit 1240.

[0150] Storage unit 1220 may include a readable medium in the form of a volatile storage unit, such as random access memory (RAM) 12201 and / or cache memory 12202, and may further include a read-only memory (ROM) 12203.

[0151] Storage unit 1220 may also include a program / utility 12204 having a set (at least one) of program modules 12205, such program modules 12205 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0152] Bus 1230 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0153] Electronic device 1200 can also communicate with one or more external devices 1270 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 1200, and / or with any device that enables electronic device 1200 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 1250. Furthermore, electronic device 1200 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 1260. As shown, network adapter 1260 communicates with other modules of electronic device 1200 via bus 1230. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 1200, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0154] In exemplary embodiments of this disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the methods described above is stored.

[0155] In some possible implementations, various aspects of the present invention may also be implemented as a program product comprising program code that, when the program product is run on a terminal device, causes the terminal device to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of the present invention.

[0156] According to embodiments of the present invention, a program product for implementing the above-described method may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, a readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with a signaling execution system, apparatus, or device.

[0157] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0158] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with a signaling execution system, apparatus, or device.

[0159] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0160] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0161] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0162] Furthermore, although the steps of the method in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.

[0163] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several signaling instructions to cause a computing device (such as a personal computer, server, mobile terminal, or network device, etc.) to execute the method according to the embodiments of this disclosure.

[0164] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.

[0165] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A machine video feature encoding method, characterized in that, The method includes: The machine video feature set is converted into a feature tensor map set; the machine video feature set corresponds to at least one video frame; the machine video feature set includes N. w ×N h Each video frame has N channels; the feature tensor map set includes N channels corresponding to each video frame. w ×N h A feature tensor map of each channel; each feature tensor map contains W×H feature tensors; The feature tensor maps are grouped according to channels, and reference feature maps are generated based on the feature tensor maps of the same channel; the reference feature maps correspond to the channels. Based on the similarity between the feature tensor map and the reference feature map, the feature tensor map is arranged to obtain a first feature map; the first feature map corresponds to each video frame. A sequence number matrix is ​​generated based on the arrangement position of the feature tensor map in the first feature map; the sequence number matrix corresponds to each video frame. The feature tensors in each of the aforementioned feature tensor maps are stitched together to obtain a second feature map; the stitching process involves rearranging the feature tensors at corresponding positions in adjacent feature tensor maps of the first feature map to obtain the second feature map. Based on the second feature map and the sequence number matrix, the compressed bitstream is obtained.

2. The method according to claim 1, characterized in that, The generation of a reference feature map based on the feature tensor map of the same channel includes: Calculate the average value of the feature tensors based on the feature tensors corresponding to different video frames in the same channel; The reference feature map is generated based on the average value of the feature tensor.

3. The method according to claim 1, characterized in that, The step of arranging the feature tensor map according to the similarity between the feature tensor map and the reference feature map to obtain the first feature map includes: Based on the similarity between the feature tensor map and the reference feature map, the feature tensor maps are arranged row by row from left to right to obtain the first feature map.

4. The method according to claim 1, characterized in that, The step of arranging the feature tensor map according to the similarity between the feature tensor map and the reference feature map to obtain the first feature map includes: The first feature tensor map with the highest similarity to the reference feature map is arranged in the upper left corner of the first feature map; Based on the similarity with the feature tensor above, the second feature tensor is determined and arranged on the far left of its row; Based on the similarity with the left-hand feature tensor, a third feature tensor is determined and arranged to the right of the left-hand feature tensor. Repeat the above process to obtain the first feature map.

5. The method according to claim 1, characterized in that, The step of arranging the feature tensor map according to the similarity between the feature tensor map and the reference feature map to obtain the first feature map includes: The first feature tensor map with the highest similarity to the reference feature map is arranged in the upper left corner of the first feature map; Based on the similarity with the feature tensor above, the second feature tensor is determined and arranged on the far left of its row; Based on the similarity with the feature tensors on the left and top, a third feature tensor is determined and arranged to the right of the left feature tensor. Repeat the above process to obtain the first feature map.

6. The method according to claim 1, characterized in that, The step of arranging the feature tensor map according to the similarity between the feature tensor map and the reference feature map to obtain the first feature map includes: The first feature tensor map with the highest similarity to the reference feature map is arranged in the upper left corner of the first feature map; Based on the similarity with the feature tensor above, the second feature tensor is determined and arranged on the far left of its row; Based on the similarity with the feature tensors on the left, top, and top left, a third feature tensor is determined and arranged to the right of the left feature tensor. Repeat the above process to obtain the first feature map.

7. The method according to claim 1, characterized in that, The similarity between the feature tensor map and the reference feature map includes: Compare each feature tensor in the feature tensor map with the reference feature value at the corresponding position in the reference feature map; Based on the relationship between the feature tensor and the reference feature value, the position is binary-labeled to obtain a binary feature map corresponding to the feature tensor map; The similarity between the two feature tensors is calculated based on the Hamming distance between the binary feature maps corresponding to the two feature tensors.

8. A method for decoding machine video features, characterized in that, The method includes: The compressed bitstream is decoded to obtain a second feature map and a sequence number matrix; the second feature map and the sequence number matrix correspond to each video frame; the sequence number matrix is ​​used to record the arrangement position information of the feature tensor map in the first feature map; The second feature map is unstitched to obtain the first feature map; the unstitching process is the inverse of the stitching process; the stitching process involves rearranging the feature tensors at corresponding positions in adjacent feature tensor maps of the first feature map. The first feature map is rearranged according to the index matrix to obtain a feature tensor map group; the feature tensor map group includes N corresponding to each video frame. w ×N h A feature tensor map of each channel; each feature tensor map contains W×H feature tensors; The feature tensor map group is converted into a machine video feature group.

9. The method according to claim 8, characterized in that, Based on the index matrix, the first feature map is rearranged to obtain a feature tensor map group, including: Obtain the fourth feature tensor map from the first feature map; Based on the sequence number matrix, the channel information corresponding to the fourth feature tensor is determined; the channel information includes at least: video frame information and channel position information; Based on the video frame information and channel position information, the fourth feature tensor map is inserted into the feature tensor map group.

10. A machine video feature encoding device, characterized in that, include: The feature tensor map group encoding module is configured to convert machine video feature groups into feature tensor map groups; The machine video feature group corresponds to at least one video frame; the machine video feature group includes N w ×N h Each video frame has N channels; the feature tensor map set includes N channels corresponding to each video frame. w ×N h Feature tensor map of each channel; Each feature tensor graph contains W×H feature tensors; The reference feature map module is configured to group the feature tensor maps by channel and generate a reference feature map based on the feature tensor maps of the same channel. The reference feature map corresponds to the channel; The first feature map encoding module is configured to arrange the feature tensor map according to the similarity between the feature tensor map and the reference feature map to obtain a first feature map; the first feature map corresponds to each video frame; The sequence matrix module is configured to generate a sequence matrix based on the arrangement position of the feature tensor map in the first feature map; The sequence number matrix corresponds to each video frame; The second feature map encoding module is configured to perform a stitching process on the feature tensors in each of the feature tensor maps to obtain a second feature map; the stitching process involves rearranging the feature tensors at corresponding positions in adjacent feature tensor maps in the first feature map to obtain the second feature map; The compressed bitstream module is configured to obtain a compressed bitstream based on the second feature map and the sequence matrix.

11. A machine video feature decoding device, characterized in that, include: The decoding stream module is configured to decode the compressed stream to obtain a second feature map and a sequence number matrix; the second feature map and the sequence number matrix correspond to each video frame. The sequence number matrix is ​​used to record the arrangement position information of the feature tensor map in the first feature map; The first feature map decoding module is configured to perform a desizing and stitching process on the second feature map to obtain the first feature map; the desizing and stitching process is the inverse process of the stitching process; the stitching process rearranges the feature tensors at corresponding positions in adjacent feature tensor maps of the first feature map. The feature tensor map group decoding module is configured to rearrange the first feature map according to the index matrix to obtain a feature tensor map group; the feature tensor map group includes N corresponding to each video frame. w ×N h A feature tensor map of each channel; each feature tensor map contains W×H feature tensors; The machine video feature group module is configured to convert the feature tensor map group into a machine video feature group.

12. An electronic device, characterized in that, include: One or more processors; A storage device configured to store one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the machine video feature encoding method as claimed in any one of claims 1 to 7 or the machine video feature decoding method as claimed in any one of claims 8 to 9.

13. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the machine video feature encoding method as described in any one of claims 1 to 7 or the machine video feature decoding method as described in any one of claims 8 to 9.

14. A computer program product, comprising a computer program / signaling, characterized in that, When the computer program / signaling is executed by the processor, it implements the machine video feature encoding method as described in any one of claims 1 to 7 or the machine video feature decoding method as described in any one of claims 8 to 9.

Images