Video coding method and device and video player

Abstract

The invention discloses a video coding method, a video coding device and a video player. The method comprises the following steps: acquiring target video data and target multi-dimensional perception data of a target object, wherein the target multi-dimensional perception data is information which is associated with the target video data and has a plurality of dimensional features and attributes; aligning the time sequence of the target multi-dimensional perception data with the time sequence of the target video data one by one to obtain reference multi-dimensional perception data; the time sequence of the reference multi-dimensional perception data is consistent with that of the target video data; and performing combined coding on the target video data and the reference multi-dimensional perception data to obtain coded video data. According to the embodiment of the invention, the problems of easy data loss and non-visual data viewing are solved, and subsequent efficient and rapid data analysis is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of video encoding and decoding, and in particular to a video encoding method, device and video player. Background Art

[0002] In the power industry, in order to better manage power equipment, video data of power equipment and multi-dimensional perception data such as dynamic environment data and temperature data of power equipment are collected. Usually, the storage of video data and multi-dimensional perception data is discrete. This storage method has the problems of easy data loss and non-intuitive data viewing when analyzing and searching historical data in the later stage. Summary of the invention

[0003] The present invention provides a video encoding method, device and video player to solve the problems of easy data loss and non-intuitive data viewing, and realize subsequent efficient and fast data analysis.

[0004] According to one aspect of the present invention, a video encoding method is provided, the method comprising:

[0005] Acquire target video data and target multi-dimensional perception data of a target object, wherein the target multi-dimensional perception data is information having multiple dimensional features and attributes associated with the target video data;

[0006] Aligning the time series of the target multidimensional perception data with the time series of the target video data one by one to obtain reference multidimensional perception data; the reference multidimensional perception data is consistent with the time series of the target video data;

[0007] The target video data and the reference multi-dimensional perception data are combined and encoded to obtain encoded video data.

[0008] According to another aspect of the present invention, a video encoding device is provided, the device comprising:

[0009] A data acquisition module, used to acquire target video data and target multi-dimensional perception data of a target object, wherein the target multi-dimensional perception data is information having multiple dimensional features and attributes associated with the target video data;

[0010] A data alignment module, used to align the time series of the target multi-dimensional perception data with the time series of the target video data one by one to obtain reference multi-dimensional perception data; the reference multi-dimensional perception data is consistent with the time series of the target video data;

[0011] The encoding module is used to combine and encode the target video data with the reference multi-dimensional perception data to obtain encoded video data.

[0012] According to another aspect of the present invention, a video player is provided, the video player comprising: a decoder, a display module and an analysis module;

[0013] The decoder is used to decode the target coded data to obtain decoded data; the target coded data is coded video data generated by the video coding method according to any one of claims 1 to 7, and the decoded data includes video data and multi-dimensional perception data;

[0014] The display module is used to display the decoded data in a display interface;

[0015] The analysis module is used to analyze the decoded data to obtain feature data related to the target object.

[0016] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0017] at least one processor; and

[0018] a memory communicatively connected to the at least one processor; wherein,

[0019] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the video encoding method described in any embodiment of the present invention.

[0020] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the video encoding method described in any embodiment of the present invention when executed.

[0021] The technical solution of the embodiment of the present invention obtains target video data and target multidimensional perception data of the target object, where the target multidimensional perception data is information with multiple dimensional features and attributes associated with the target video data, thereby ensuring the richness of the data; aligning the time series of the target multidimensional perception data with the time series of the target video data one by one to obtain reference multidimensional perception data, thereby ensuring that the time series of the multidimensional perception data is consistent with the time series of the target video data, so as to facilitate subsequent encoding; the reference multidimensional perception data is consistent with the time series of the target video data; the target video data and the reference multidimensional perception data are combined and encoded to obtain encoded video data, thereby realizing the encoding of video data and multidimensional perception data together, solving the problems of easy data loss and non-intuitive data viewing, and realizing subsequent efficient and fast data analysis.

[0022] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 is a flowchart of a video encoding method provided according to an embodiment of the present invention;

[0025] Figure 2 is a flowchart of another video encoding method provided according to an embodiment of the present invention;

[0026] Figure 3 is a structural schematic diagram of a video encoding device provided according to an embodiment of the present invention;

[0027] Figure 4 is a structural schematic diagram of a video player provided according to an embodiment of the present invention;

[0028] Figure 5 It is a structural schematic diagram of an electronic device implementing a video encoding method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0030] It should be noted that the terms "first", "second", "target", "reference", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0031] Embodiment 1

[0032] Figure 1 A flowchart of a video encoding method provided in an embodiment of the present invention. This embodiment is applicable to the case of combined encoding of video data and multi-dimensional perceptual data. The method can be executed by a video encoding device. The video encoding device can be implemented in the form of hardware and / or software. The video encoding device can be configured in any electronic device with network communication function.

[0033] like Figure 1 As shown, the video encoding method of the present invention includes the following process:

[0034] S110. Obtain target video data and target multi-dimensional perception data of the target object, where the target multi-dimensional perception data is information with multiple dimensional features and attributes associated with the target video data.

[0035] Among them, the target video data can be understood as the video shot by the camera of the target object, and the target multi-dimensional perception data can include the dynamic environment data, ambient temperature, humidity and other perception data of the target object. Dynamic environment data refers to power and environment data, which is the monitoring and recording of the power system required for the operation of various equipment and the relevant data of the environment in which it is located.

[0036] Specifically, the target object can be understood as a device that can generate multi-dimensional perception data, including but not limited to power grid equipment. Because the target multi-dimensional perception data has a great impact on subsequent data analysis, in addition to obtaining the target video data of the target object, it is still necessary to accurately obtain the target multi-dimensional perception data. The target video data can be obtained through the shooting device, and the target multi-dimensional perception data can be obtained through the sensor associated with the target object.

[0037] S120, aligning the time series of the target multidimensional perception data with the time series of the target video data one by one to obtain reference multidimensional perception data; the time series of the reference multidimensional perception data is consistent with the time series of the target video data.

[0038] Specifically, in order to facilitate subsequent encoding processing, the target multidimensional perception data is processed based on the time series of the target video data, that is, the time series of the target video data is known, the time series of the target multidimensional perception data is determined, and then the sequence in the time series of the target multidimensional perception data that is consistent with the time series of the target video data is saved as reference multidimensional perception data, thereby ensuring that the reference multidimensional perception data is consistent with the time series of the target video data.

[0039] As an optional but non-limiting manner, the time series of the target multi-dimensional perception data is aligned with the time series of the target video data one by one to obtain reference multi-dimensional perception data, including steps A1-A3:

[0040] Step A1: determine a first sampling rate of target video data and a second sampling rate of target multi-dimensional perception data.

[0041] Step A2: If the first sampling rate is greater than the second sampling rate, data amplification processing is performed on the target multidimensional perception data to obtain reference multidimensional perception data.

[0042] Specifically, if the first sampling rate is greater than the second sampling rate, it means that the target multidimensional perception data is less than the target video data, and the time series of the target multidimensional perception data is inconsistent with the target video data. Therefore, a generative adversarial network (GAN) is used to analyze the target multidimensional perception data based on the time series of the target video data to generate data consistent with the time series of the target video data, and use it as reference multidimensional perception data.

[0043] Generative adversarial network is a neural network model composed of generator and discriminator, which can be used to generate new data similar to the original data. By training GAN, the generator can generate more realistic multi-dimensional perceptual data, thereby increasing the amount and diversity of data.

[0044] As an optional embodiment, data augmentation processing is performed on the target multidimensional perception data to obtain reference multidimensional perception data, including: using a preset interpolation method to perform data augmentation processing on the target multidimensional perception data according to the time series of the target video data to obtain reference multidimensional perception data.

[0045] The preset interpolation method includes one of a zero-one interpolation method, a linear interpolation method and a quadratic interpolation method.

[0046] Specifically, a first time series that is missing from the target multidimensional perception data compared to the time series in the target video data is determined, and a preset interpolation method is used to generate first multidimensional perception data corresponding to the first time series. The first multidimensional perception data and the target multidimensional perception data constitute reference multidimensional perception data.

[0047] Optionally, the zero-one interpolation method can be understood as: interpolating between two sampling data; specifically: calculating the distance between the interpolated data and the sampling data according to the absolute value of the time difference, and the interpolated data has the same value as the sampling data closest to it.

[0048] Optionally, the linear interpolation method can be understood as: interpolating between two sampling data; specifically: constructing metadata with time and data value, establishing a linear function with the two metadata, and bringing the interpolation time into the linear function to obtain the interpolated data value.

[0049] Optionally, the quadratic interpolation method can be understood as: interpolating between three sampling data; specifically: constructing metadata with time and data value, establishing a quadratic function with three metadata, and bringing the interpolation time into the quadratic function to obtain the interpolated data value.

[0050] The technical solution of this embodiment adopts a preset interpolation method to perform data augmentation processing on the target multidimensional perception data according to the time series of the target video data to obtain reference multidimensional perception data. The preset interpolation method fills in the missing data points, increases the data density, improves the data resolution, and further ensures the consistency of the time series of the reference multidimensional perception data and the target video data, which is beneficial to subsequent data analysis. In addition, the interpolation method is highly adaptable and can be applied to different types of data, which can ensure the accuracy of data processing.

[0051] Step A3: If the first sampling rate is less than the second sampling rate, data cropping is performed on the target multidimensional perception data to obtain reference multidimensional perception data; the cropping process is to retain data that is consistent with the time series of the target video data.

[0052] Specifically, the first sampling rate is less than the second sampling rate, indicating that the target multidimensional perception data is more than the target video data. In order to ensure that the time series of the target multidimensional perception data is consistent with the target video data, it is necessary to trim the data whose time series in the target multidimensional perception data is inconsistent with the time series in the target video data.

[0053] The technical solution of this embodiment determines the size of the first sampling rate of the target video data and the second sampling rate of the target multidimensional perception data, thereby taking different processing methods for the target multidimensional perception data, thereby ensuring the accuracy of the obtained reference multidimensional perception data and its consistency with the time series of the target video data.

[0054] S130, combining and encoding the target video data and the reference multi-dimensional perception data to obtain encoded video data.

[0055] The encoding algorithms used in this embodiment include but are not limited to h264, 265, v8 and other encoding algorithms.

[0056] The combined coding of this embodiment can give full play to the advantages of various coding methods and improve coding efficiency. By selecting appropriate coding methods for combination, the length of the encoded data can be minimized under the premise of ensuring data integrity.

[0057] Specifically, a preset combined encoding method is set to obtain target video data and reference multi-dimensional perception data, and based on the preset combined encoding method, the target video data and the reference multi-dimensional perception data are encoded to obtain encoded video data, thereby ensuring that the target video data and the reference multi-dimensional perception data are saved together, improving the encoding efficiency, and facilitating subsequent data call analysis.

[0058] In this embodiment, after obtaining the encoded video data, a lossless compression algorithm is used to compress the encoded video data to obtain compressed target video encoded data for easy storage; the data size of the target video encoded data is smaller than the sum of the data sizes of the target video data and the target multi-dimensional perception data.

[0059] The technical solution of the embodiment of the present invention obtains target video data and target multidimensional perception data of the target object, where the target multidimensional perception data is information with multiple dimensional features and attributes associated with the target video data, thereby ensuring the richness of the data; aligning the time series of the target multidimensional perception data with the time series of the target video data one by one to obtain reference multidimensional perception data, thereby ensuring that the time series of the multidimensional perception data is consistent with the time series of the target video data, so as to facilitate subsequent encoding; the reference multidimensional perception data is consistent with the time series of the target video data; the target video data and the reference multidimensional perception data are combined and encoded to obtain encoded video data, thereby realizing the encoding of video data and multidimensional perception data together, solving the problems of easy data loss and non-intuitive data viewing, and realizing subsequent efficient and fast data analysis.

[0060] Embodiment 2

[0061] Figure 2 This is a flowchart of another video encoding method provided by an embodiment of the present invention. The technical solution of this embodiment further optimizes the processes of S120 and S130 in the aforementioned embodiment on the basis of the aforementioned embodiment. This embodiment can be combined with various optional solutions in one or more of the aforementioned embodiments.

[0062] like Figure 2 As shown, the video encoding method of the present invention includes the following process:

[0063] S210. Obtain target video data and target multi-dimensional perception data of the target object, where the target multi-dimensional perception data is information associated with the target video data and having multiple dimensional features and attributes.

[0064] S220, decomposing the target multi-dimensional perception data into a preset number of video frame sequences, and determining a target timestamp for each video frame sequence.

[0065] The timestamp can be an absolute time or a relative time relative to a certain reference time.

[0066] S230. Based on the target timestamp of each video frame sequence, the target multidimensional perception data is divided into a preset number of multidimensional perception data sequences, and the preset number of multidimensional perception data sequences are reference multidimensional perception data; the timestamps of the multidimensional perception data sequences correspond one-to-one to the target timestamps of the video frame sequences.

[0067] Specifically, based on the target timestamp of each video frame sequence, the target multidimensional perception data is divided into a preset number of multidimensional perception data sequences, that is, the data in the target multidimensional perception data corresponds to the target timestamp of each video frame sequence, and the data in the target multidimensional perception data that is the same as the target timestamp is taken as a multidimensional perception data sequence, so that the target multidimensional perception data can be divided into a preset number of multidimensional perception data sequences, and the timestamps corresponding to the multidimensional perception data sequences correspond one-to-one to the target timestamps of the video frame sequences, thereby ensuring the time series consistency between the reference multidimensional perception data and the target video data.

[0068] S240: Combining and encoding the target video data with the reference multi-dimensional perception data to obtain encoded video data.

[0069] Specifically, based on the algorithmic features of a preset encoding algorithm, each video frame sequence of the target video data is encoded to obtain a preset number of first encoded data, and a target frame sequence is reserved in the first encoded data; further determine the reference timestamp corresponding to the first encoded data, and use the multidimensional perception data sequence whose corresponding timestamp in the preset number of multidimensional perception data sequences is the same as the reference timestamp as the reference multidimensional perception data sequence; based on the algorithmic features of the target frame sequence in the first encoded data corresponding to the reference timestamp, encode the reference multidimensional perception data sequence to obtain second encoded data; insert the second encoded data into the target frame sequence in the first encoded data corresponding to the reference timestamp to obtain encoded video data, thereby ensuring that the encoded video data can be used in a general video player or in a player that can display multidimensional perception data, thereby achieving efficient storage of data.

[0070] For example, the target video data is obtained as h264 video, and the full radiation temperature data corresponding to its target timestamp, that is, the target multi-dimensional perception data. Based on the algorithm characteristics of the preset encoding algorithm, each video frame sequence of the target video data is encoded to obtain a preset number of first encoded data, and the reference timestamp corresponding to each first encoded data is determined, and the first encoded data reserves a section of the target frame sequence. If the first sampling rate is greater than the second sampling rate, the zero-one interpolation method can be used to amplify the full radiation temperature data to obtain the reference full radiation temperature data, and the reference full radiation temperature data is matched with the target video frame data one by one according to the reference timestamp to obtain the full radiation temperature sequence; each full radiation temperature sequence is traversed, and it is encoded as the second encoded data and inserted into the corresponding target frame sequence of the first encoded data to obtain the encoded video data.

[0071] In this embodiment, after obtaining the encoded video data, a lossless compression algorithm is used to compress the encoded video data to obtain compressed target video encoded data for easy storage; the data size of the target video encoded data is smaller than the sum of the data sizes of the target video data and the target multi-dimensional perception data.

[0072] The technical solution of the embodiment of the present invention obtains the target video data and target multidimensional perception data of the target object, the target multidimensional perception data is information with multiple dimensional features and attributes associated with the target video data, and ensures the richness of the data; decomposes the target multidimensional perception data into a preset number of video frame sequences, and determines the target timestamp of each video frame sequence, and divides the target multidimensional perception data into a preset number of multidimensional perception data sequences based on the target timestamp of each video frame sequence, and the preset number of multidimensional perception data sequences are reference multidimensional perception data; the timestamp of the multidimensional perception data sequence corresponds to the target timestamp of the video frame sequence one by one. , ensuring that the time series of the multidimensional perception data is unified with the time series of the target video data, so as to facilitate subsequent encoding; the target video data and the reference multidimensional perception data are combined and encoded to obtain the encoded video data, realizing encoding the video data and the multidimensional perception data together, solving the problem of easy data loss and unintuitive data viewing, and realizing subsequent efficient and fast data analysis.

[0073] Embodiment 3

[0074] Figure 3 A structural schematic diagram of a video encoding device provided in an embodiment of the present invention. This embodiment can be applied to the case of combined encoding of video data and multi-dimensional perception data. The XXX device can be implemented in the form of hardware and / or software. The video encoding device can be configured in any electronic device with network communication function.

[0075] like Figure 3As shown, the video encoding device of the present invention comprises:

[0076] A data acquisition module 310 is used to acquire target video data and target multi-dimensional perception data of a target object, wherein the target multi-dimensional perception data is information having multiple dimensional features and attributes associated with the target video data;

[0077] A data alignment module 320 is used to align the time series of the target multi-dimensional perception data with the time series of the target video data one by one to obtain reference multi-dimensional perception data; the reference multi-dimensional perception data is consistent with the time series of the target video data;

[0078] The encoding module 330 is used to combine and encode the target video data with the reference multi-dimensional perception data to obtain encoded video data.

[0079] Based on the above embodiment, optionally, the data alignment module includes:

[0080] A sampling rate determination unit, used to determine a first sampling rate of the target video data and a second sampling rate of the target multi-dimensional perception data;

[0081] a first judgment unit, configured to perform data augmentation processing on the target multidimensional perception data to obtain reference multidimensional perception data if the first sampling rate is greater than the second sampling rate;

[0082] The second judgment unit is used to perform data cropping processing on the target multi-dimensional perception data to obtain reference multi-dimensional perception data if the first sampling rate is less than the second sampling rate; the cropping processing is the processing of retaining data consistent with the time series of the target video data.

[0083] On the basis of the above embodiment, optionally, the first judgment unit is further used to: adopt a preset interpolation method to perform data augmentation processing on the target multidimensional perception data according to the time series of the target video data to obtain reference multidimensional perception data.

[0084] Based on the above embodiment, optionally, the preset interpolation method includes one of a zero-one interpolation method, a linear interpolation method and a quadratic interpolation method.

[0085] Based on the above embodiment, optionally, the data alignment module includes a data processing unit, and the data processing unit is used to:

[0086] Decomposing the target multi-dimensional perception data into a preset number of video frame sequences, and determining a target timestamp for each video frame sequence;

[0087] Based on the target timestamp of each video frame sequence, the target multidimensional perception data is divided into a preset number of multidimensional perception data sequences; the timestamps of the multidimensional perception data sequences correspond one-to-one to the target timestamps of the video frame sequences.

[0088] Based on the above embodiment, optionally, the encoding module is used to:

[0089] Based on the algorithm characteristics of the preset encoding algorithm, each video frame sequence of the target video data is encoded to obtain a preset number of first encoded data; the first encoded data reserves a section of the target frame sequence;

[0090] Determine a reference timestamp corresponding to the first coded data, and use a preset number of multidimensional perceptual data sequences whose corresponding timestamps are the same as the reference timestamp as reference multidimensional perceptual data sequences;

[0091] encoding the reference multi-dimensional perception data sequence based on an algorithm feature of a target frame sequence in the first coded data corresponding to the reference timestamp to obtain second coded data;

[0092] The second encoded data is inserted into a target frame sequence in the first encoded data corresponding to the reference timestamp to obtain encoded video data.

[0093] Based on the above embodiment, optionally, the encoding module also includes a compression unit, which is used to: compress the encoded video data using a lossless compression algorithm to obtain compressed target video encoding data; the data size of the target video encoding data is smaller than the sum of the data sizes of the target video data and the target multi-dimensional perception data.

[0094] The video encoding device provided in the embodiment of the present invention can execute the video encoding method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0095] The acquisition, storage, use, and processing of data in the technical solution of this application comply with the relevant provisions of national laws and regulations and do not violate public order and good morals.

[0096] Embodiment 4

[0097] Figure 4 The video player 40 of the present invention comprises: a decoder 410 , a display module 420 and an analysis module 430 .

[0098] The decoder 410 is used to decode the target coded data to obtain decoded data; the target coded data is the coded video data generated by the video coding method according to any one of claims 1-7, and the decoded data includes video data and multi-dimensional perception data.

[0099] The display module 420 is used to display the decoded data in a display interface.

[0100] The analysis module 430 is used to analyze the decoded data to obtain feature data related to the target object.

[0101] Optionally, the analysis module is also used to determine the target area in response to the first trigger operation, use the decoded data corresponding to the target area as the target decoded data, and determine the characteristic values ​​of the multidimensional perception data in the target area based on the target decoded data; the characteristic values ​​are used to describe the distribution of the multidimensional perception data; the target area is the area determined after executing the first trigger operation in the display interface.

[0102] For example, the encoded video data includes encoded h264 video data and encoded full radiation temperature data. The decoder in the video player in this embodiment is used to decode the encoded video data generated by any video encoding method of claims 1-7 to obtain decoded data, and the display module displays the h264 video data and full radiation temperature data in the display interface.

[0103] Furthermore, the first trigger operation may include but is not limited to operations such as moving a control displayed on the display interface, clicking operations, etc. The click operation may be performed to select a data range displayed on the display interface.

[0104] For example, the first trigger operation is moving the mouse. In response to the first trigger operation, the temperature value of the location where the first trigger operation is performed can be viewed in real time.

[0105] The first trigger operation is a click operation. In response to the first trigger operation, by clicking the dotted frame button on the player, a dotted frame and other graphic areas can be drawn on the display interface. The analysis module can calculate the maximum, minimum and average temperature in the drawn graphic area and display the results on the screen to analyze the changes in temperature data in the area.

[0106] The video player of the technical solution of this embodiment can not only decode the target encoded data containing multi-dimensional perception data, but also display the decoded data through the display module, that is, the display interface can display the video data, and at the same time can display the multi-dimensional perception data, thereby realizing a complete and intuitive display of the actual situation of the target device when acquiring the data; further, the analysis module in the video player can also perform feature analysis on the decoded data, and display the analysis results on the display interface using the display module, thereby making data analysis more convenient and efficient.

[0107] Embodiment 5

[0108] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0109] Figure 5 A schematic diagram of the structure of an electronic device that can be used to implement the video encoding method of an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0110] like Figure 5 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0111] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0112] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as a video encoding method.

[0113] In some embodiments, the video encoding method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the video encoding method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the video encoding method in any other appropriate manner (e.g., by means of firmware).

[0114] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0115] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0116] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0117] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0118] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0119] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0120] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0121] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A video encoding method, characterized in that: The method comprises: Acquire target video data and target multi-dimensional perception data of a target object, wherein the target multi-dimensional perception data is information having multiple dimensional features and attributes associated with the target video data; Aligning the time series of the target multidimensional perception data with the time series of the target video data one by one to obtain reference multidimensional perception data; the reference multidimensional perception data is consistent with the time series of the target video data; The target video data and the reference multi-dimensional perception data are combined and encoded to obtain encoded video data.

2. The method according to claim 1, characterized in that: Aligning the time series of the target multi-dimensional perception data with the time series of the target video data one by one to obtain reference multi-dimensional perception data, including: Determining a first sampling rate of the target video data and a second sampling rate of the target multi-dimensional perception data; If the first sampling rate is greater than the second sampling rate, performing data augmentation processing on the target multidimensional perception data to obtain reference multidimensional perception data; If the first sampling rate is less than the second sampling rate, data cropping is performed on the target multidimensional perception data to obtain reference multidimensional perception data; the cropping is a process of retaining data that is consistent with the time series of the target video data.

3. The method according to claim 2, characterized in that Performing data augmentation processing on the target multi-dimensional perception data to obtain reference multi-dimensional perception data includes: A preset interpolation method is used to perform data augmentation processing on the target multidimensional perception data according to the time series of the target video data to obtain reference multidimensional perception data.

4. The method according to claim 3, characterized in that The preset interpolation method includes one of a zero-one interpolation method, a linear interpolation method and a quadratic interpolation method.

5. The method according to claim 1, characterized in that Aligning the time series of the target multi-dimensional perception data with the time series of the target video data one by one to obtain reference multi-dimensional perception data, including: Decomposing the target multi-dimensional perception data into a preset number of video frame sequences, and determining a target timestamp for each video frame sequence; Based on the target timestamp of each video frame sequence, the target multidimensional perception data is divided into a preset number of multidimensional perception data sequences; the timestamps of the multidimensional perception data sequences correspond one-to-one to the target timestamps of the video frame sequences.

6. The method according to claim 5, characterized in that Combining and encoding the target video data with the reference multi-dimensional perception data to obtain encoded video data includes: Based on the algorithm characteristics of the preset encoding algorithm, each video frame sequence of the target video data is encoded to obtain a preset number of first encoded data; the first encoded data reserves a section of the target frame sequence; Determine a reference timestamp corresponding to the first coded data, and use a preset number of multidimensional perceptual data sequences whose corresponding timestamps are the same as the reference timestamp as reference multidimensional perceptual data sequences; encoding the reference multi-dimensional perception data sequence based on an algorithm feature of a target frame sequence in the first coded data corresponding to the reference timestamp to obtain second coded data; The second encoded data is inserted into a target frame sequence in the first encoded data corresponding to the reference timestamp to obtain encoded video data.

7. The method according to claim 1 or 6, characterized in that: After obtaining the encoded video data, the method further includes: The encoded video data is compressed using a lossless compression algorithm to obtain compressed target video encoded data; the data size of the target video encoded data is smaller than the sum of the data sizes of the target video data and the target multi-dimensional perception data.

8. A video encoding device, characterized in that: The device comprises: A data acquisition module, used to acquire target video data and target multi-dimensional perception data of a target object, wherein the target multi-dimensional perception data is information having multiple dimensional features and attributes associated with the target video data; A data alignment module, used to align the time series of the target multi-dimensional perception data with the time series of the target video data one by one to obtain reference multi-dimensional perception data; the reference multi-dimensional perception data is consistent with the time series of the target video data; The encoding module is used to combine and encode the target video data with the reference multi-dimensional perception data to obtain encoded video data.

9. A video player, characterized in that: The video player comprises: a decoder, a display module and an analysis module; The decoder is used to decode the target coded data to obtain decoded data; the target coded data is coded video data generated by the video coding method according to any one of claims 1 to 7, and the decoded data includes video data and multi-dimensional perception data; The display module is used to display the decoded data in a display interface; The analysis module is used to analyze the decoded data to obtain feature data related to the target object.

10. The video player according to claim 9, characterized in that: The analysis module is further configured to determine a target area in response to a first trigger operation, use the decoded data corresponding to the target area as target decoded data, and determine a characteristic value of the multidimensional perception data in the target area based on the target decoded data; The characteristic value is used to describe the distribution of multi-dimensional perception data; the target area is an area determined after the first trigger operation is performed in the display interface.

Images