Machine video feature coding and decoding method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a machine video feature coding method and device, a decoding method and device, equipment, a medium and a program product. The method comprises the following steps: converting a machine video feature group into a feature tensor graph group, and performing two rounds of rearrangement on a feature tensor graph and each feature tensor in the feature tensor graph to generate a second feature graph; meanwhile, the position change information of the rearrangement is recorded through a sequence number matrix. And obtaining a compressed code stream based on the second feature map and the serial number matrix. According to the method, through the rearrangement process, the generated second feature map is suitable for being coded and decoded by adopting a video coding and decoding algorithm, so that the bandwidth is saved, the transmission speed is improved, and the related processing cost is reduced under the condition of not changing the hardware system of the existing video coding and decoding device.

Description

Technical Field

[0001] The present disclosure relates to the field of video coding and decoding technology, and in particular to a machine video feature coding and decoding method, device, electronic device, storage medium and computer program product. Background Art

[0002] Machine vision feature coding technology aims to compress and transmit features by extracting and reducing dimensions, combined with existing video codec standards (such as VVC, HEVC, AVC). However, since these methods do not optimize the arrangement of features for different channels, the compression efficiency is limited. Compared with traditional video data, feature data is sparse and redundant, and directly applying video codecs cannot achieve the best compression effect. Specifically, the similarity between feature channels is low, making it difficult to splice into continuous videos; the spatial redundancy pattern is complex, which increases the difficulty of encoding; and the amount of mid-level feature data is huge, far exceeding the original input.

[0003] In order to overcome the above challenges and make full use of existing video compression frameworks and hardware facilities, it is necessary to transform the data form of features. This includes designing specific quantization and arrangement strategies to reduce the amount of transmitted data and improving the spatial redundancy removal method of features. In this way, compression performance can be improved, ensuring the effective transmission of feature maps while maintaining high reconstruction quality, thereby supporting a wider range of application scenarios. Summary of the invention

[0004] The embodiments of the present disclosure provide a machine video feature encoding and decoding method, device, electronic device, storage medium and computer program product.

[0005] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0006] According to one aspect of the present disclosure, a method for encoding machine video features is provided, the method comprising: converting a machine video feature group into a feature tensor graph group; the machine video feature group corresponds to at least one video frame; the machine video feature group includes N w ×N h channels; the feature tensor graph group includes N corresponding to each video frame w ×N hThe invention relates to a method for generating a feature tensor map of a channel; each feature tensor map includes W×H feature tensors; the feature tensor maps are grouped according to channels, and a reference feature map is generated based on the feature tensor maps of the same channel; the reference feature map corresponds to the channel; the feature tensor maps are arranged according to the similarity between the feature tensor map and the reference feature map to obtain a first feature map; the first feature map corresponds to each video frame; a serial number matrix is ​​generated according to the arrangement position of the feature tensor map in the first feature map; the serial number matrix corresponds to each video frame; the feature tensors in each feature tensor map are stitched to obtain a second feature map; based on the second feature map and the serial number matrix, a compressed code stream is obtained.

[0007] In an exemplary embodiment, the generating of a reference feature map based on a feature tensor map of the same channel includes: calculating an average value of feature tensors according to feature tensors corresponding to different video frames in the same channel; and generating the reference feature map based on the average value of feature tensors.

[0008] In an exemplary embodiment, the step of arranging the feature tensor graph according to the similarity between the feature tensor graph and a reference feature graph to obtain a first feature graph includes: arranging the feature tensor graph row by row from left to right according to the similarity between the feature tensor graph and the reference feature graph to obtain the first feature graph.

[0009] In an exemplary embodiment, the feature tensor graph is arranged according to the similarity between the feature tensor graph and the reference feature graph to obtain a first feature graph, including: arranging the first feature tensor graph having the highest similarity to the reference feature graph at the upper left corner of the first feature graph; determining the second feature tensor graph according to the similarity to the upper feature tensor graph, and arranging it at the leftmost side of the row; determining the third feature tensor graph according to the similarity to the left feature tensor graph, and arranging it to the right of the left feature tensor graph; repeating the above process to obtain the first feature graph.

[0010] In an exemplary embodiment, the feature tensor graph is arranged according to the similarity between the feature tensor graph and a reference feature graph to obtain a first feature graph, including: arranging the first feature tensor graph having the highest similarity to the reference feature graph at the upper left corner of the first feature graph; determining the second feature tensor graph according to the similarity to the feature tensor graph above, and arranging it at the leftmost side of the row; determining the third feature tensor graph according to the similarity to the feature tensor graphs to the left and above, and arranging it to the right of the left feature tensor graph; repeating the above process to obtain the first feature graph.

[0011] In an exemplary embodiment, the feature tensor graph is arranged according to the similarity between the feature tensor graph and a reference feature graph to obtain a first feature graph, including: arranging the first feature tensor graph having the highest similarity to the reference feature graph at the upper left corner of the first feature graph; determining the second feature tensor graph according to the similarity to the feature tensor graph above, and arranging it at the leftmost side of the row; determining the third feature tensor graph according to the similarity to the feature tensor graphs to the left, above, and upper left, and arranging it to the right of the left feature tensor graph; repeating the above process to obtain the first feature graph.

[0012] In an exemplary embodiment, the similarity between the feature tensor map and the reference feature map includes: comparing each feature tensor in the feature tensor map with the reference eigenvalue of the corresponding position of the reference feature map; based on the size relationship between the feature tensor and the reference eigenvalue, binary marking the position to obtain a binary feature map corresponding to the feature tensor map; and calculating the similarity between the two feature tensor maps based on the Hamming distance between the binary feature maps corresponding to the two feature tensor maps.

[0013] In an exemplary embodiment, stitching the feature tensors in each of the feature tensor maps to obtain the second feature map includes: rearranging the feature tensors at corresponding positions in adjacent feature tensor maps in the first feature map to obtain the second feature map.

[0014] According to another aspect of the present disclosure, a machine video feature decoding method is provided, the method comprising: decoding a compressed code stream to obtain a second feature map and a sequence number matrix; the second feature map and the sequence number matrix correspond to each video frame; the sequence number matrix is ​​used to record the arrangement position information of the feature tensor map in the first feature map; performing unstitching processing on the second feature map to obtain the first feature map; rearranging the first feature map according to the sequence number matrix to obtain a feature tensor map group; the feature tensor map group includes N corresponding to each video frame w ×N h The feature tensor maps include W×H feature tensors; and the feature tensor map group is converted into a machine video feature group.

[0015] In an exemplary embodiment, performing unstitching processing on the second feature map to obtain the first feature map includes: rearranging feature tensors at corresponding positions in adjacent feature tensor maps in the second feature map to obtain the first feature map.

[0016] In an exemplary embodiment, the first feature map is rearranged according to the serial number matrix to obtain a feature tensor map group, including: obtaining a fourth feature tensor map from the first feature map; determining channel information corresponding to the fourth feature tensor map according to the serial number matrix; the channel information includes at least: video frame information and channel position information; and inserting the fourth feature tensor map into the feature tensor map group according to the video frame information and the channel position information.

[0017] According to another aspect of the present disclosure, a machine video feature encoding device is provided, comprising: a feature tensor graph group encoding module, configured to convert a machine video feature group into a feature tensor graph group; the machine video feature group corresponds to at least one video frame; the machine video feature group includes N w ×N h channels; the feature tensor graph group includes N corresponding to each video frame w ×N h a feature tensor map of a channel; each feature tensor map includes W×H feature tensors; a reference feature map module is configured to group the feature tensor maps according to channels, and generate a reference feature map based on the feature tensor maps of the same channel; the reference feature map corresponds to the channel; a first feature map encoding module is configured to arrange the feature tensor maps according to the similarity between the feature tensor maps and the reference feature map to obtain a first feature map; the first feature map corresponds to each video frame; a sequence number matrix module is configured to generate a sequence number matrix according to the arrangement position of the feature tensor maps in the first feature map; the sequence number matrix corresponds to each video frame; a second feature map encoding module is configured to stitch the feature tensors in each feature tensor map to obtain a second feature map; a compressed code stream module is configured to obtain a compressed code stream based on the second feature map and the sequence number matrix.

[0018] According to another aspect of the present disclosure, a machine video feature decoding device is provided, comprising: a decoding code stream module, configured to decode a compressed code stream to obtain a second feature map and a sequence number matrix; the second feature map and the sequence number matrix correspond to each video frame; the sequence number matrix is ​​used to record the arrangement position information of the feature tensor map in the first feature map; a first feature map decoding module, configured to perform a de-stitching process on the second feature map to obtain the first feature map; a feature tensor map group decoding module, configured to rearrange the first feature map according to the sequence number matrix to obtain a feature tensor map group; the feature tensor map group includes N corresponding to each video frame w ×N h The invention relates to a machine video feature group module, wherein the machine video feature group module comprises a feature tensor map of channels; each feature tensor map comprises W×H feature tensors; and a machine video feature group module is configured to convert the feature tensor map group into a machine video feature group.

[0019] According to another aspect of the present disclosure, an electronic device is provided, comprising: one or more processors; and a storage device configured to store one or more programs, so that when the one or more programs are executed by the one or more processors, the one or more processors implement the machine video feature encoding method or the machine video feature decoding method as described in the above embodiments.

[0020] According to another aspect of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the machine video feature encoding method or the machine video feature decoding method as described in the above embodiments is implemented.

[0021] According to another aspect of the present disclosure, a computer program product is provided, including a computer program / signaling, characterized in that when the computer program / signaling is executed by a processor, the machine video feature encoding method or the machine video feature decoding method as described in the above embodiments is implemented.

[0022] The machine video feature encoding method provided by the embodiment of the present disclosure converts the machine video feature group into a feature tensor map group, and performs two rounds of rearrangement on the feature tensor map and each feature tensor in the feature tensor map to generate a second feature map. And the position change information of the rearrangement is recorded through a sequence matrix. Based on the second feature map and the sequence matrix, a compressed code stream is obtained. Through the rearrangement process, the method makes the generated second feature map suitable for encoding and decoding using a video codec algorithm, thereby saving bandwidth, improving transmission speed, and reducing the cost of related processing without changing the existing video codec hardware system.

[0023] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0025] Figure 1 A flow chart of a machine video feature encoding method according to an embodiment of the present disclosure is shown;

[0026] Figure 2 A flow chart showing a feature tensor graph arrangement process of an embodiment of the present disclosure is shown;

[0027] Figure 3 A schematic diagram showing a process of arranging a feature tensor graph according to an embodiment of the present disclosure is shown;

[0028] Figure 4 A flow chart showing a similarity calculation method according to an embodiment of the present disclosure is shown;

[0029] Figure 5 A flow chart showing a suturing process according to an embodiment of the present disclosure is shown;

[0030] Figure 6 A flow chart of a machine video feature decoding method according to an embodiment of the present disclosure is shown;

[0031] Figure 7 A flowchart showing a first feature map rearrangement process according to an embodiment of the present disclosure is shown;

[0032] Figure 8 A schematic diagram showing a de-stitching process according to an embodiment of the present disclosure;

[0033] Fig. 9 A schematic diagram showing a machine video feature decoding process according to an embodiment of the present disclosure;

[0034] Fig.10 A schematic diagram showing the structure of a machine video feature encoding device according to an embodiment of the present disclosure is shown;

[0035] Fig.11 A schematic diagram showing the structure of a machine video feature decoding device according to an embodiment of the present disclosure is shown;

[0036] Fig.12 A schematic diagram of the structure of an electronic device suitable for implementing an exemplary embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0037] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0038] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0039] It should be noted that the ordinal numbers such as "first" and "second" mentioned in the embodiments of the present disclosure are used to distinguish multiple objects, and are not used to limit the order, timing, priority or importance of multiple objects, and the descriptions of "first" and "second" do not limit the objects to be different.

[0040] Figure 1 FIG. 4 is a flow chart showing a method for encoding machine video features according to an embodiment of the present disclosure. Figure 1 As shown, the machine video feature encoding method may include the following steps.

[0041] In step S110, a machine video feature group is converted into a feature tensor map group; the machine video feature group corresponds to at least one video frame; the machine video feature group includes N w ×N h channels; the feature tensor graph group includes N corresponding to each video frame w ×N h The feature tensor map contains W×H feature tensors.

[0042] In the field of machine vision and deep learning, unlike humans who directly process visual images or videos, machines need to extract machine video feature groups from traditional video information and then analyze and process based on video features. This process first involves analyzing each frame of the input video and extracting feature vectors that can describe the frame content. These feature vector sets constitute the machine video feature group. The feature group corresponding to each video frame contains N w ×N h Channels. Channels refer to the different dimensions or layers that make up the feature map of each video frame. Each channel carries a specific type of feature information about the frame (such as edge, texture, or color distribution, etc.), and together they form a multi-channel feature tensor map. w and N h They represent the number of channels in the width and height directions of a feature map of size W×H respectively.

[0043] Next, in order to better utilize existing image and video compression technologies, such as VVC (Versatile Video Coding), HEVC (High Efficiency Video Coding) or AVC (Advanced Video Coding), these feature groups need to be converted into feature tensor graph groups that are more suitable for compression algorithm processing. Each video frame corresponds to a tensor graph consisting of N w ×N hThe feature tensor graph consists of channels, and each feature tensor graph contains W×H feature tensors. Here, W and H represent the spatial dimensions of each feature graph, that is, the width and height of the feature graph. Each feature tensor not only retains the information of each channel in the original feature group, but also enhances the correlation between different channels through a specific arrangement, making the feature data easier to be recognized and processed by the compression algorithm. This conversion helps to improve compression efficiency while maintaining the effectiveness and accuracy of feature expression.

[0044] In an exemplary embodiment, the process of converting the machine video feature set into the feature tensor map set may include:

[0045] The input original video feature group is trimmed, scaled and quantized into integers, preferably using the following formula for quantization:

[0046]

[0047] Where x is the input eigenvalue, c max , c min are the maximum and minimum values ​​of the clipping, N is the preset quantization level, round(·) means rounding the result to an integer, x clp It is a quantitative result.

[0048] The channel mean distribution of the features in all frames of the video feature group after statistical quantization is calculated, and the reference channel is used to replace the channel where the distance mean exceeds the preset threshold and the data frequency is lower than the preset threshold; preferably, the following formula is used to calculate the mean of the features in each channel:

[0049]

[0050] Among them, x is the input feature value, t represents a frame of the video, c represents the channel number, w represents the width of the video feature, h represents the height of the video feature, F represents the total number of frames of the video feature, Widh represents the total width of the video feature, and Height represents the total height of the video feature.

[0051] In step S120, the feature tensor maps are grouped according to channels, and a reference feature map is generated based on the feature tensor maps of the same channel; the reference feature map corresponds to the channel.

[0052] The feature tensor graph group obtained by the above steps corresponds to each video frame and includes N w ×N h The feature tensor map contains W×H feature tensors.

[0053] In the disclosed embodiment, the feature tensor graphs of all video frames are grouped according to the channel sequence number so that the feature tensor graphs of the same channel are grouped together. For example, the first channel of all video frames is grouped into one group, the second channel is grouped into another group, and so on. Based on each group of feature tensor graphs of the same channel, a corresponding reference feature graph is generated by calculating statistics (such as mean, median, or maximum value, etc.). The reference feature graph corresponds to a specific channel, and it comprehensively reflects the typical feature performance of the channel in all video frames.

[0054] In an exemplary embodiment, the feature tensor average is calculated based on the feature tensors corresponding to different video frames in the same channel; and the reference feature map is generated based on the feature tensor average. By calculating the feature tensor average of different video frames in the same channel, the information of each frame can be integrated to extract a reference feature map representing the typical features of the channel, so as to measure each feature tensor map later.

[0055] In step S130, the feature tensor map is arranged according to the similarity between the feature tensor map and the reference feature map to obtain a first feature map; the first feature map corresponds to each video frame.

[0056] In the embodiment of the present disclosure, in order to reduce the channel redundancy of the deep features without changing the original model structure to perform video compression for machine vision tasks, it is necessary to rearrange the feature tensor graph. In the present disclosure, the feature tensor graph is sorted at the block level according to the similarity between the feature tensor graph and the reference feature graph to meet the characteristics of the existing video compression algorithm.

[0057] First, the similarity between the feature tensor map corresponding to each video frame in the same channel and the reference feature map of the corresponding channel is calculated. This process usually involves comparing the feature distribution between the two, and may use metrics such as cosine similarity and Euclidean distance to quantify the similarity between the two. Through such a comparison, the degree of similarity between each feature tensor map and the reference feature map can be determined. Next, the feature tensor maps are sorted based on the above similarity. And based on the sorted position, the feature tensor map is inserted into a specific position of the first feature map to obtain the first feature map. The first feature map corresponds to each video frame. That is, the feature tensor maps corresponding to different channels of the same video frame are arranged in the same first feature map.

[0058] In an exemplary embodiment, the arrangement process of the feature tensor map may include: arranging the feature tensor map row by row from left to right according to the similarity sorting between the feature tensor map and the reference feature map, to obtain the first feature map. In this process, the feature tensor maps are arranged row by row from left to right according to the similarity sorting of each feature tensor map. That is, sorted by similarity, first arrange the feature tensor maps from left to right in the first row until the first row is full, and then arrange the feature tensor maps from left to right in the second row until the second row is full. Repeat this process, arranging the feature tensor maps row by row from top to bottom until the entire first feature map is filled, and the first feature map is obtained. Among them, the width of the first feature map is W×N w , height is H×N h .

[0059] In step S140, a serial number matrix is ​​generated according to the arrangement position of the feature tensor map in the first feature map; the serial number matrix corresponds to each video frame.

[0060] In the embodiment of the present disclosure, during the feature tensor map arrangement process of the aforementioned step S130, a corresponding serial number matrix is ​​recorded and generated according to the arrangement position of each feature tensor map in the first feature map. The serial number matrix is ​​used to record the correspondence between the channel information of the feature tensor map in the feature tensor map group and the position information in the first feature map. The position of the feature tensor map can be decoded and restored during the decoding process through the serial number matrix. Among them, the channel information can be the serial number of the feature tensor map, and can also include: video frame information and channel position information. The video frame information and channel position information are used to indicate the video frame corresponding to the feature tensor map, and the channel position in the feature tensor map group. The serial number matrix and the first feature Figure 1 One to one, and also corresponds to each video frame.

[0061] In step S150, the feature tensors in each of the feature tensor maps are stitched to obtain a second feature map.

[0062] In the embodiment of the present disclosure, for the purpose of video compression for machine vision tasks, not only block-level reordering as in the aforementioned step S130 is required, but also pixel-level reordering within the block is required. Here, each pixel within the block is each feature tensor in the feature tensor map.

[0063] In the disclosed embodiment, based on the first feature map obtained above, the feature tensors in each feature tensor map are stitched, that is, the positions of the feature tensors are rearranged, so as to obtain a second feature map. The stitching process is a technique for enhancing the representation of feature maps, which creates a larger feature block by rearranging and combining pixels of adjacent feature maps. In video coding or deep learning applications, this process helps to improve the consistency and continuity of feature expression, thereby improving compression efficiency or model performance.

[0064] On the one hand, the stitching process can effectively utilize the enhanced correlation between the sorted feature maps to ensure that the generated feature blocks are more coherent, which is beneficial to subsequent encoding or decoding operations. On the other hand, the stitching process can also effectively reduce the amount of transmitted data without significantly increasing the computational complexity, thereby improving the coding efficiency.

[0065] In step S160, a compressed code stream is obtained based on the second feature map and the sequence number matrix.

[0066] In the disclosed embodiment, encoding is performed based on the second feature map and sequence number matrix obtained above to generate a compressed code stream.

[0067] In an exemplary embodiment, since the machine video feature group is converted into the feature map, the second feature map obtained by the conversion can be suitable for encoding and decoding using an existing video encoding and decoding algorithm to obtain good bit rate compression. Based on this, the above encoding process can be performed on it using an existing video codec. Since this solution saves bandwidth and improves transmission speed without changing the existing video codec hardware system, while ensuring the accuracy of machine vision tasks, it has low energy consumption and low cost.

[0068] The machine video feature encoding method provided by the embodiment of the present disclosure converts the machine video feature group into a feature tensor map group, and performs two rounds of rearrangement on the feature tensor map and each feature tensor in the feature tensor map to generate a second feature map. And the position change information of the rearrangement is recorded through a sequence matrix. Based on the second feature map and the sequence matrix, a compressed code stream is obtained. Through the rearrangement process, the method makes the generated second feature map suitable for encoding and decoding using a video codec algorithm, thereby saving bandwidth, improving transmission speed, and reducing the cost of related processing without changing the existing video codec hardware system.

[0069] Figure 2 A flowchart of a feature tensor graph arrangement process according to an embodiment of the present disclosure is shown. Figure 3 FIG. 1 is a schematic diagram showing a process of arranging a feature tensor graph according to an embodiment of the present disclosure. Figure 2 , 3 As shown, in Figure 1Based on the machine video feature encoding method shown, step S130 may include the following steps.

[0070] In step S210, the first feature tensor map having the highest similarity to the reference feature map is arranged at the upper left corner of the first feature map.

[0071] In the embodiment of the present disclosure, various methods can be used to arrange the feature tensor graph. Figure 2 , 3 The illustrated embodiment introduces a feasible arrangement method.

[0072] In the embodiment of the present disclosure, the arrangement process arranges each feature tensor graph from top to bottom and from left to right. First, the feature tensor graph with the highest similarity to the reference feature graph is selected as the first feature tensor graph, and arranged in the upper left corner of the first feature graph, that is, the leftmost side of the first row. The position of the first feature tensor graph can be recorded as P 0，0 .

[0073] In the disclosed embodiment, after determining the first feature tensor map in the upper left corner, the remaining feature tensor maps in the first row are arranged based on the first feature tensor map. The next right feature map to be arranged is determined based on the similarity with the feature map already placed on the left, ensuring that it is as close as possible to the adjacent left feature map. And continue to arrange to the right in this way until all feature maps in the row are placed. This process ensures that the local similarity between feature tensor maps is maximized, thereby providing a good foundation for subsequent compression.

[0074] In step S220, a second feature tensor graph is determined based on the similarity with the upper feature tensor graph and arranged at the leftmost side of the row.

[0075] In the disclosed embodiment, when all feature graphs in a row are arranged, the leftmost feature graph of the next row depends on the similarity evaluation of the feature graph above it. The feature tensor graph with the highest similarity to the leftmost feature tensor graph of the previous row is selected as the second feature tensor graph, which is arranged at the leftmost of the next row as the starting feature graph of the next row.

[0076] like Figure 3 As shown in a, assuming that the leftmost feature tensor graph in the previous row is P m-1，n-1 After the arrangement of the n-1th row is completed, according to the characteristic tensor graph P m-1，n-1 Similarity, determine the feature tensor graph P on the left of the nth row m-1，n .

[0077] In step S230, a third feature tensor graph is determined according to the similarity with the left feature tensor graph, and is arranged on the right side of the left feature tensor graph.

[0078] In the disclosed embodiment, the third feature tensor map is determined based on the similarity with the feature map already placed on the left, ensuring that it is as close as possible to the adjacent left feature map, and is placed on the right side of the left feature tensor map. Continue to arrange to the right in this way until all feature tensor maps in the row are placed.

[0079] like Figure 3 As shown in a, when determining the leftmost feature tensor graph P of the nth row m-1，n Afterwards, based on the feature tensor graph P m-1，n Similarity, determine the feature tensor graph P on the right m，n .

[0080] By repeating the above steps S220 and S230 until the entire first feature map is filled, the first feature map is obtained. It should be noted that the execution order of steps S220 and S230 can also be swapped, that is, arranging the rows first and then the columns, or arranging the columns first and then the rows, which should be within the protection scope of the present disclosure.

[0081] At the same time, when arranging non-starting feature tensor graphs in a row, the feature tensor graphs they refer to can also be adjusted as needed.

[0082] In an exemplary embodiment, if Figure 3 As shown in b, the third feature tensor map is determined based on the similarity with the feature maps already placed on the left and above, ensuring that it is as close as possible to the adjacent left and above feature maps, and is placed on the right side of the left feature tensor map. Continue to arrange to the right in this way until all feature tensor maps in this row are placed. As shown in the figure, after determining the left feature tensor map P of the nth row m-1，n And the above feature tensor graph P m，n-1 Afterwards, based on the feature tensor graph P m-1，n and P m，n-1 The similarity of the right feature tensor graph P is determined m，n .

[0083] In an exemplary embodiment, if Figure 3 As shown in c, the third feature tensor map is determined based on the similarity with the feature maps already placed on the left, above, and upper left, ensuring that it is as close as possible to the adjacent left, above, and upper left feature maps, and is placed on the right side of the left feature tensor map. Continue to arrange to the right in this way until all feature tensor maps in this row are placed. As shown in the figure, after determining the left feature tensor map P of the nth row m-1，n , the upper feature tensor graph P m，n-1 And the upper left feature tensor graph P m-1，n-1 Afterwards, based on the feature tensor graph P m-1，n , P m，n-1 and P m-1，n-1The similarity of the right feature tensor graph P is determined m，n .

[0084] The machine video feature encoding method provided by the embodiment of the present disclosure provides a plurality of ways of arranging feature tensor graphs, and arranges feature tensor graphs with high similarity in adjacent positions by calculating the similarity between adjacent feature tensor graphs. In this way, the correlation and continuity of adjacent regions in the feature graph are improved, thereby improving the compression efficiency or model performance, and being suitable for encoding processing by existing video encoding and decoding algorithms.

[0085] Figure 4 FIG. 4 is a flow chart showing a similarity calculation method according to an embodiment of the present disclosure. Figure 4 As shown, in Figure 1 Based on the machine video feature encoding method shown, the similarity calculation method between the feature tensor graph and the reference feature graph in step S130 may include the following steps.

[0086] In step S410, each feature tensor in the feature tensor map is compared with a reference feature value at a corresponding position in the reference feature map.

[0087] In the disclosed embodiment, the reference eigenvalues ​​of the feature tensor map of each channel are compared with the reference eigenvalues ​​of the corresponding positions of the reference feature map. That is, the reference eigenvalues ​​of the feature tensor map of the feature tensor map and the corresponding positions of the reference feature map are compared one by one. This process ensures that the details at each pixel level are taken into account, so as to accurately measure the similarity between the two.

[0088] In step S420, the position is binary-labeled according to the magnitude relationship between the feature tensor and the reference eigenvalue to obtain a binary feature map corresponding to the feature tensor map.

[0089] In the disclosed embodiment, the corresponding position is marked with a binary value according to the size relationship between the above-mentioned feature tensor and the reference feature value. Specifically, if the feature tensor is greater than the corresponding reference feature value, it is marked as 1; conversely, if it is less than the reference feature value, it is marked as 0. And the mark is recorded in the corresponding position. In this way, the feature tensor map can be converted into a binary form to form a corresponding binary feature map.

[0090] In step S430, the similarity between the two feature tensor maps is calculated according to the Hamming distance between the binary feature maps corresponding to the two feature tensor maps.

[0091] In the disclosed embodiment, the binary feature maps corresponding to each feature tensor map can be generated by the method of step S420. According to the binary feature maps corresponding to any two feature tensor maps, the similarity between the two feature tensor maps is calculated.

[0092] Specifically, we first compare the corresponding binary feature maps. We quantify the difference between the two binary feature maps by measuring the Hamming distance between them, that is, counting the number of different values ​​on the corresponding bits. The so-called Hamming distance refers to the number of different characters in the same position of two strings of equal length, which is often used to measure the difference between binary strings or feature vectors. Based on the Hamming distance of two feature tensor maps, the overall similarity of the two feature tensor maps is measured. The smaller the Hamming distance, the smaller the difference between the two feature tensor maps and the higher the similarity. Conversely, the similarity is lower.

[0093] By using the similarity calculation method provided by the present disclosure, the similarity between any two feature tensor graphs, or the similarity between a feature tensor graph and a reference feature graph can be calculated. Based on the relevant similarity comparison, the aforementioned arrangement operation can be performed.

[0094] Figure 5 FIG. 4 is a flow chart showing the stitching process of the embodiment of the present disclosure. Figure 5 As shown, in Figure 1 Based on the machine video feature encoding method shown, step S150 may include the following steps.

[0095] In step S510, feature tensors at corresponding positions in adjacent feature tensor maps in the first feature map are rearranged to obtain the second feature map.

[0096] As mentioned above, based on the first feature map obtained above, the feature tensors in each feature tensor map are stitched, that is, the positions of the feature tensors are rearranged, so as to obtain the second feature map.

[0097] In the disclosed embodiment, for each feature tensor in the first feature map, it is reordered with the feature tensors at adjacent positions according to a certain optimization rule. This process ensures that tensors with similar features can be clustered more closely together, thereby enhancing the local correlation between feature maps. Through this rearrangement, the consistency and coherence of feature representation can be improved, which helps to improve the efficiency and accuracy of subsequent encoding or analysis steps, and finally forms an optimized second feature map.

[0098] In an exemplary embodiment, the rearrangement process may be performed by using 2 times pixel-level stitching to merge adjacent feature tensor images into a feature block of a larger size, specifically a feature block of 4 times the original size. Specifically, the following process may be included:

[0099] Use P m，n represents a feature tensor graph, and (m, n) represents the vertical and horizontal coordinates of its feature graph in the first feature graph. 2m，2n [σ][τ] Feature tensor map P2m，2n The horizontal and vertical coordinates are σ, the characteristic tensor of T. 2m-1，2n [σ][τ],p 2m，2n-1 [σ][τ],p 2m-1，2n-1 [σ][τ],p 2m，2n [σ][τ] are rearranged and combined into a feature block B of size 4*P.

[0100] B m，n Represents the feature block with the vertical and horizontal coordinates (m, n) after rearrangement, with b m，n [σ][τ] represents the feature block B m，n The horizontal and vertical coordinates are σ and τ of the characteristic tensor, then b m，n [σ][τ] and p 2m，2n The corresponding relationship between [σ][τ] is:

[0101] b m，n [2σ][2τ]=p 2m，2n [σ][τ]

[0102] b m，n [2σ+1][τ]=p 2m-1，2n [σ][τ]

[0103] b m，n [2σ][2τ+1]=p 2m，2n-1 [σ][τ]

[0104] b m，n [2σ+1][2τ+1]=p 2m-1，2n-1 [σ][τ]

[0105] It should be noted that the above is only a feasible exemplary suture processing method. According to actual needs, other rearrangement methods can be adopted, or other multiple suture processing can be performed, which should be considered within the scope of protection of the present disclosure.

[0106] Figure 6 FIG. 4 is a flow chart showing a method for decoding machine video features according to an embodiment of the present disclosure. Figure 6 As shown, the machine video feature decoding method may include the following steps.

[0107] In step S610, the compressed code stream is decoded to obtain a second feature map and a sequence number matrix; the second feature map and the sequence number matrix correspond to each video frame; the sequence number matrix is ​​used to record the arrangement position information of the feature tensor map in the first feature map.

[0108] As mentioned above, since the machine video feature group has been converted into the above-mentioned feature map, it is suitable for encoding and decoding using the existing video encoding and decoding algorithm. Therefore, after receiving the compressed code stream, the decoding end can perform the above-mentioned decoding processing on it through the existing video codec, thereby obtaining a second feature map and a serial number matrix. The second feature map and the serial number matrix are the second feature map and the serial number matrix introduced in the encoding method above, and will not be repeated here. Among them, the second feature map and the serial number matrix correspond to each video frame. The serial number matrix is ​​used to record the arrangement position information of the feature tensor map in the first feature map. The serial number matrix also records the correspondence between the channel information of the feature tensor map in the feature tensor map group and the position information in the first feature map.

[0109] In step S620, the second feature map is unstitched to obtain the first feature map.

[0110] In the disclosed embodiment, the decoding end performs a de-stitching process on the obtained second feature map. The de-stitching process is the inverse process of the stitching process described in the encoding end, and its specific process will not be introduced here. The decoding end can determine the stitching processing algorithm used at the encoding end based on a pre-agreed agreement or a specific flag indication, and then adopt the corresponding de-stitching processing algorithm. The first feature map is obtained by performing a de-stitching process on the second feature map. The first feature map is the first feature map introduced in the encoding method above, and will not be repeated here.

[0111] In an exemplary embodiment, if Figure 8 As shown, the unstitching process may include: rearranging the feature tensors of corresponding positions in adjacent feature tensor maps in the second feature map to obtain the first feature map.

[0112] In step S630, the first feature map is rearranged according to the sequence matrix to obtain a feature tensor map group; the feature tensor map group includes N corresponding to each video frame. w ×N h The feature tensor map contains W×H feature tensors.

[0113] In the disclosed embodiment, since the sequence number matrix records the correspondence between the channel information of the feature tensor map in the feature tensor map group and the position information in the first feature map, the first feature map can be rearranged according to the position information recorded in the sequence number matrix. The rearrangement process is the inverse process of step S130 in the aforementioned encoding method, and its specific processing process is not introduced here. As mentioned above, the feature tensor map group obtained by rearrangement includes N corresponding to each video frame. w ×N hThe feature tensor map contains W×H feature tensors.

[0114] In step S640, the feature tensor map group is converted into a machine video feature group.

[0115] In the embodiment of the present disclosure, the feature tensor graph group is converted to obtain a corresponding machine video feature group. The conversion process is the inverse process of step S110 in the aforementioned encoding method, and its specific processing process is not introduced here.

[0116] like Fig. 9 As shown, through the above machine video feature decoding process, the compressed data in the compressed code stream is decoded and restored into a machine video feature group that can be processed by the machine. Based on the machine video feature group, the machine can perform analysis and processing operations on related videos, which will not be repeated here.

[0117] The machine video feature decoding method provided by the embodiment of the present disclosure uses the inverse process of the encoding method to decode and restore the compressed code stream into a machine video feature group that can be processed by the machine. Through the rearrangement process, the method makes the generated second feature map suitable for encoding and decoding using a video encoding and decoding algorithm, thereby saving bandwidth, improving transmission speed, and reducing the cost of related processing without changing the existing video codec hardware system.

[0118] Figure 7 FIG. 4 is a flowchart showing a first feature map rearrangement process according to an embodiment of the present disclosure. Figure 7 As shown, in Figure 6 Based on the machine video feature decoding method shown, step S630 may include the following steps.

[0119] In step S710, a fourth feature tensor map is obtained from the first feature map.

[0120] In the disclosed embodiment, a fourth feature tensor map is obtained from the first feature map. The fourth feature tensor map is any feature tensor map in the first feature map.

[0121] In step S720, channel information corresponding to the fourth feature tensor map is determined according to the sequence matrix; the channel information at least includes: video frame information and channel position information.

[0122] In the disclosed embodiment, since the correspondence between the channel information of the feature tensor map in the feature tensor map group and the position information in the first feature map is recorded in the serial number matrix. According to the position of the fourth feature tensor map in the first feature map, the channel information of the fourth feature tensor map is determined. The channel information includes at least: video frame information and channel position information. The video frame information is used to indicate the video frame corresponding to the feature tensor map. The channel position information is used to indicate the channel position of the feature tensor map in the feature tensor map group.

[0123] In step S730, the fourth feature tensor map is inserted into the feature tensor map group according to the video frame information and the channel position information.

[0124] In the embodiment of the present disclosure, according to the video frame information and the channel position information, the corresponding video frame and the corresponding channel position of the fourth feature tensor map in the feature tensor map group can be determined. Based on this, the fourth feature tensor map can be inserted into the feature tensor map group to be generated.

[0125] By repeating the process of steps S710 - 730 above until all feature tensor maps in the first feature map are acquired and inserted into the feature tensor map group, the feature tensor map group is generated.

[0126] Based on the same inventive concept, the embodiments of the present disclosure provide a machine video feature encoding device and a machine video feature decoding device, as described in the following embodiments. Since the principle of solving the problem in the device embodiment is similar to that in the above method embodiment, the implementation of the communication authentication device embodiment can refer to the implementation of the above method embodiment, and the repeated parts will not be repeated.

[0127] Fig.10 FIG. 2 is a schematic diagram showing the structure of a machine video feature encoding device according to an embodiment of the present disclosure. Fig.10 As shown, the machine video feature encoding device 1000 may include: a feature tensor graph group encoding module 1010, a reference feature graph module 1020, a first feature graph encoding module 1030, a serial number matrix module 1040, a second feature graph encoding module 1050 and a compressed code stream module 1060.

[0128] The feature tensor graph group encoding module 1010 is configured to convert a machine video feature group into a feature tensor graph group; the machine video feature group corresponds to at least one video frame; the machine video feature group includes N w ×N h channels; the feature tensor graph group includes N corresponding to each video frame w ×N h The feature tensor map of each channel; each feature tensor map includes W×H feature tensors;

[0129] A reference feature map module 1020 is configured to group the feature tensor maps by channel and generate a reference feature map based on the feature tensor maps of the same channel; the reference feature map corresponds to the channel;

[0130] A first feature map encoding module 1030 is configured to arrange the feature tensor map according to the similarity between the feature tensor map and the reference feature map to obtain a first feature map; the first feature map corresponds to each video frame;

[0131] A sequence number matrix module 1040 is configured to generate a sequence number matrix according to the arrangement position of the feature tensor map in the first feature map; the sequence number matrix corresponds to each video frame;

[0132] A second feature map encoding module 1050 is configured to stitch the feature tensors in each of the feature tensor maps to obtain a second feature map;

[0133] The compressed code stream module 1060 is configured to obtain a compressed code stream based on the second feature map and the sequence number matrix.

[0134] In an exemplary embodiment, the generating of a reference feature map based on a feature tensor map of the same channel includes: calculating an average value of feature tensors according to feature tensors corresponding to different video frames in the same channel; and generating the reference feature map based on the average value of feature tensors.

[0135] In an exemplary embodiment, the step of arranging the feature tensor graph according to the similarity between the feature tensor graph and a reference feature graph to obtain a first feature graph includes: arranging the feature tensor graph row by row from left to right according to the similarity between the feature tensor graph and the reference feature graph to obtain the first feature graph.

[0136] In an exemplary embodiment, the feature tensor graph is arranged according to the similarity between the feature tensor graph and the reference feature graph to obtain a first feature graph, including: arranging the first feature tensor graph having the highest similarity to the reference feature graph at the upper left corner of the first feature graph; determining the second feature tensor graph according to the similarity to the upper feature tensor graph, and arranging it at the leftmost side of the row; determining the third feature tensor graph according to the similarity to the left feature tensor graph, and arranging it to the right of the left feature tensor graph; repeating the above process to obtain the first feature graph.

[0137] In an exemplary embodiment, the feature tensor graph is arranged according to the similarity between the feature tensor graph and a reference feature graph to obtain a first feature graph, including: arranging the first feature tensor graph having the highest similarity to the reference feature graph at the upper left corner of the first feature graph; determining the second feature tensor graph according to the similarity to the feature tensor graph above, and arranging it at the leftmost side of the row; determining the third feature tensor graph according to the similarity to the feature tensor graphs to the left and above, and arranging it to the right of the left feature tensor graph; repeating the above process to obtain the first feature graph.

[0138] In an exemplary embodiment, the feature tensor graph is arranged according to the similarity between the feature tensor graph and a reference feature graph to obtain a first feature graph, including: arranging the first feature tensor graph having the highest similarity to the reference feature graph at the upper left corner of the first feature graph; determining the second feature tensor graph according to the similarity to the feature tensor graph above, and arranging it at the leftmost side of the row; determining the third feature tensor graph according to the similarity to the feature tensor graphs to the left, above, and upper left, and arranging it to the right of the left feature tensor graph; repeating the above process to obtain the first feature graph.

[0139] In an exemplary embodiment, the similarity between the feature tensor map and the reference feature map includes: comparing each feature tensor in the feature tensor map with the reference eigenvalue of the corresponding position of the reference feature map; based on the size relationship between the feature tensor and the reference eigenvalue, binary marking the position to obtain a binary feature map corresponding to the feature tensor map; and calculating the similarity between the two feature tensor maps based on the Hamming distance between the binary feature maps corresponding to the two feature tensor maps.

[0140] In an exemplary embodiment, stitching the feature tensors in each of the feature tensor maps to obtain the second feature map includes: rearranging the feature tensors at corresponding positions in adjacent feature tensor maps in the first feature map to obtain the second feature map.

[0141] Fig.11 FIG. 2 is a schematic diagram showing the structure of a machine video feature decoding device according to an embodiment of the present disclosure. Fig.11 As shown, the machine video feature decoding device 1100 may include: a code stream decoding module 1110 , a first feature map decoding module 1120 , a feature tensor map group decoding module 1130 and a machine video feature group module 1140 .

[0142] The decoding code stream module 1110 is configured to decode the compressed code stream to obtain a second feature map and a sequence number matrix; the second feature map and the sequence number matrix correspond to each video frame; the sequence number matrix is ​​used to record the arrangement position information of the feature tensor map in the first feature map;

[0143] A first feature map decoding module 1120 is configured to perform a de-stitching process on the second feature map to obtain the first feature map;

[0144] The feature tensor map group decoding module 1130 is configured to rearrange the first feature map according to the sequence number matrix to obtain a feature tensor map group; the feature tensor map group includes N corresponding to each video frame w ×N h The feature tensor map of each channel; each feature tensor map includes W×H feature tensors;

[0145] The machine video feature group module 1140 is configured to convert the feature tensor map group into a machine video feature group.

[0146] In an exemplary embodiment, performing unstitching processing on the second feature map to obtain the first feature map includes: rearranging feature tensors at corresponding positions in adjacent feature tensor maps in the second feature map to obtain the first feature map.

[0147] In an exemplary embodiment, the first feature map is rearranged according to the serial number matrix to obtain a feature tensor map group, including: obtaining a fourth feature tensor map from the first feature map; determining channel information corresponding to the fourth feature tensor map according to the serial number matrix; the channel information includes at least: video frame information and channel position information; and inserting the fourth feature tensor map into the feature tensor map group according to the video frame information and the channel position information.

[0148] Fig.12 The structure diagram of the electronic device suitable for implementing the exemplary embodiment of the present disclosure is shown. Fig.12 The electronic device 1200 according to this embodiment of the present invention is described. Fig.12 The electronic device 1200 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0149] like Fig.12 As shown, the electronic device 1200 is in the form of a general computing device. The components of the electronic device 1200 may include, but are not limited to: the at least one processing unit 1210, the at least one storage unit 1220, a bus 1230 connecting different system components (including the storage unit 1220 and the processing unit 1210), and a display unit 1240.

[0150] The storage unit 1220 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 12201 and / or a cache storage unit 12202 , and may further include a read-only storage unit (ROM) 12203 .

[0151] The storage unit 1220 may also include a program / utility 12204 having a set (at least one) of program modules 12205, such program modules 12205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0152] Bus 1230 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0153] The electronic device 1200 may also communicate with one or more external devices 1270 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 1200, and / or communicate with any device that enables the electronic device 1200 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 1250. In addition, the electronic device 1200 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 1260. As shown, the network adapter 1260 communicates with other modules of the electronic device 1200 via a bus 1230. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 1200, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0154] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is further provided, on which a program product capable of implementing the above method of the present specification is stored.

[0155] In some possible implementations, various aspects of the present invention may also be implemented in the form of a program product, which includes program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps of various exemplary implementations of the present invention described in the above “Exemplary Methods” section of this specification.

[0156] The program product for implementing the above method according to the embodiment of the present invention may adopt a portable compact disk read-only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto, and in this document, a readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with a signaling execution system, apparatus, or device.

[0157] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0158] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with a signaling execution system, apparatus, or device.

[0159] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0160] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0161] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0162] In addition, although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0163] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation methods of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several signalings to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the implementation methods of the present disclosure.

[0164] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any modification, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the appended claims.

[0165] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A machine video feature encoding method, characterized in that: The method comprises: Converting a machine video feature group into a feature tensor map group; the machine video feature group corresponds to at least one video frame; the machine video feature group includes N w ×N h channels; the feature tensor graph group includes N corresponding to each video frame w ×N h The feature tensor map of each channel; each feature tensor map includes W×H feature tensors; The feature tensor maps are grouped according to channels, and reference feature maps are generated based on feature tensor maps of the same channel; the reference feature maps correspond to the channels; Arranging the feature tensor map according to the similarity between the feature tensor map and the reference feature map to obtain a first feature map; the first feature map corresponds to each video frame; Generate a sequence number matrix according to the arrangement position of the feature tensor map in the first feature map; the sequence number matrix corresponds to each video frame; Perform stitching processing on the feature tensors in each of the feature tensor maps to obtain a second feature map; Based on the second feature map and the sequence number matrix, a compressed code stream is obtained.

2. The method according to claim 1, characterized in that The generating a reference feature map based on the feature tensor map of the same channel includes: According to the feature tensors corresponding to different video frames in the same channel, the average value of the feature tensors is calculated; The reference feature map is generated based on the feature tensor average value.

3. The method according to claim 1, characterized in that: The step of arranging the feature tensor graph according to the similarity between the feature tensor graph and the reference feature graph to obtain a first feature graph includes: According to the similarity sorting between the feature tensor map and the reference feature map, the feature tensor map is arranged row by row from left to right to obtain the first feature map.

4. The method according to claim 1, characterized in that: The step of arranging the feature tensor graph according to the similarity between the feature tensor graph and the reference feature graph to obtain a first feature graph includes: Arrange the first feature tensor map having the highest similarity to the reference feature map at the upper left corner of the first feature map; According to the similarity with the feature tensor graph above, the second feature tensor graph is determined and arranged at the leftmost side of the row; Determine a third feature tensor graph according to the similarity with the left feature tensor graph, and arrange it on the right side of the left feature tensor graph; Repeat the above process to obtain the first feature map.

5. The method according to claim 1, characterized in that The step of arranging the feature tensor graph according to the similarity between the feature tensor graph and the reference feature graph to obtain a first feature graph includes: Arrange the first feature tensor map having the highest similarity to the reference feature map at the upper left corner of the first feature map; According to the similarity with the feature tensor graph above, the second feature tensor graph is determined and arranged at the leftmost side of the row; Determine a third feature tensor graph according to the similarity with the feature tensor graphs on the left and above, and arrange it on the right side of the feature tensor graph on the left; Repeat the above process to obtain the first feature map.

6. The method according to claim 1, characterized in that The step of arranging the feature tensor graph according to the similarity between the feature tensor graph and the reference feature graph to obtain a first feature graph includes: Arrange the first feature tensor map having the highest similarity to the reference feature map at the upper left corner of the first feature map; According to the similarity with the feature tensor graph above, the second feature tensor graph is determined and arranged at the leftmost side of the row; Determine a third feature tensor graph according to the similarity with the left, top, and top-left feature tensor graphs, and arrange it on the right side of the left feature tensor graph; Repeat the above process to obtain the first feature map.

7. The method according to claim 1, characterized in that The similarity between the feature tensor graph and the reference feature graph includes: Comparing each feature tensor in the feature tensor map with a reference feature value at a corresponding position in the reference feature map; According to the magnitude relationship between the feature tensor and the reference feature value, the position is binary-labeled to obtain a binary feature map corresponding to the feature tensor map; The similarity between the two feature tensor maps is calculated according to the Hamming distance between the binary feature maps corresponding to the two feature tensor maps.

8. The method according to claim 1, characterized in that The stitching process of the feature tensors in each of the feature tensor graphs to obtain a second feature graph comprises: The feature tensors of corresponding positions in adjacent feature tensor maps in the first feature map are rearranged to obtain the second feature map.

9. A machine video feature decoding method, characterized in that: The method comprises: Decoding the compressed code stream to obtain a second feature map and a sequence number matrix; the second feature map and the sequence number matrix correspond to each video frame; the sequence number matrix is ​​used to record the arrangement position information of the feature tensor map in the first feature map; performing unstitching processing on the second feature map to obtain the first feature map; According to the sequence matrix, the first feature map is rearranged to obtain a feature tensor map group; the feature tensor map group includes N corresponding to each video frame w ×N h The feature tensor map of each channel; each feature tensor map includes W×H feature tensors; The feature tensor map group is converted into a machine video feature group.

10. The method according to claim 9, characterized in that The performing unstitching processing on the second feature map to obtain the first feature map includes: The feature tensors of corresponding positions in adjacent feature tensor maps in the second feature map are rearranged to obtain the first feature map.

11. The method according to claim 9, characterized in that According to the sequence matrix, the first feature map is rearranged to obtain a feature tensor map group, including: Obtaining a fourth feature tensor map from the first feature map; Determine, according to the sequence number matrix, channel information corresponding to the fourth feature tensor graph; the channel information at least includes: video frame information and channel position information; Inserting the fourth feature tensor map into the feature tensor map group according to the video frame information and the channel position information.

12. A machine video feature encoding device, characterized in that: include: a feature tensor map group encoding module configured to convert the machine video feature group into a feature tensor map group; The machine video feature group corresponds to at least one video frame; the machine video feature group includes N w ×N h channels; the feature tensor graph group includes N corresponding to each video frame w ×N h The feature tensor map of the channels; Each feature tensor graph includes W×H feature tensors; A reference feature map module is configured to group the feature tensor maps by channel and generate a reference feature map based on the feature tensor maps of the same channel; The reference feature map corresponds to the channel; A first feature map encoding module is configured to arrange the feature tensor map according to the similarity between the feature tensor map and the reference feature map to obtain a first feature map; the first feature map corresponds to each video frame; A serial number matrix module is configured to generate a serial number matrix according to the arrangement position of the feature tensor map in the first feature map; the serial number matrix corresponds to each video frame; A second feature map encoding module is configured to stitch the feature tensors in each of the feature tensor maps to obtain a second feature map; The compressed code stream module is configured to obtain a compressed code stream based on the second feature map and the sequence number matrix.

13. A machine video feature decoding device, characterized in that: include: A decoding code stream module is configured to decode the compressed code stream to obtain a second feature map and a sequence number matrix; the second feature map and the sequence number matrix correspond to each video frame; The serial number matrix is ​​used to record the arrangement position information of the feature tensor map in the first feature map; A first feature map decoding module is configured to perform a de-stitching process on the second feature map to obtain the first feature map; A feature tensor map group decoding module is configured to rearrange the first feature map according to the sequence matrix to obtain a feature tensor map group; the feature tensor map group includes N corresponding to each video frame w ×N h The feature tensor map of each channel; each feature tensor map includes W×H feature tensors; The machine video feature group module is configured to convert the feature tensor map group into a machine video feature group.

14. An electronic device, characterized in that: include: one or more processors; A storage device configured to store one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement the machine video feature encoding method as described in any one of claims 1 to 8 or the machine video feature decoding method as described in any one of claims 9 to 11.

15. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the machine video feature encoding method according to any one of claims 1 to 8 or the machine video feature decoding method according to any one of claims 9 to 11 is implemented.

16. A computer program product, comprising a computer program / signaling, characterized in that When the computer program / signaling is executed by a processor, the machine video feature encoding method according to any one of claims 1 to 8 or the machine video feature decoding method according to any one of claims 9 to 11 is implemented.

Images