Binocular image processing method and device, computer equipment and storage medium
By calculating and encoding the motion vector and disparity vector and their residuals in the binocular image data, the problem that traditional technology is difficult to effectively compress binocular image data is solved, and a more efficient compression effect is achieved.
Patent Information
- Application Number
- CN202411712807.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-26
- Publication Date
- 2025-03-04
AI Technical Summary
Traditional video compression technology is difficult to effectively process binocular image data, resulting in unsatisfactory compression speed and compression ratio.
By acquiring the front decoded frame and the current frame in the binocular image data, the motion vector and the disparity vector and their residuals are calculated, and these vectors and residuals are encoded to generate encoded data.
The correlation and redundancy of the two front-end decoded frames are utilized to improve the compression speed and compression ratio of the binocular image data.
Smart Images
Figure CN119562075A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image processing, and in particular to a binocular image processing method, device, computer equipment and storage medium. Background Art
[0002] With the development of image processing technology, more and more binocular cameras are used in the fields of smart medical care and smart finance. Binocular cameras are similar to human eyes and can calculate parallax and depth to detect obstacles. Due to the addition of depth information, the performance of binocular cameras is better than that of monocular cameras. With the development of binocular cameras, the binocular image data collected by binocular cameras has shown an exponential growth. For example, binocular cameras are used in autonomous driving, which generates a huge amount of binocular image data. Binocular image data includes image data collected by two cameras respectively, so binocular image data has correlation and redundancy. Traditional video compression technology is not suitable for processing binocular image data. Summary of the invention
[0003] Based on this, it is necessary to provide a binocular image processing method, device, computer equipment and storage medium to address the above technical problems, so as to improve the compression speed and compression ratio of binocular image data.
[0004] A binocular image processing method, comprising: Acquire binocular image data; the binocular image data includes a first previously decoded frame, a second previously decoded frame, and a first current frame to be compressed; Determine a first motion vector and a first motion residual according to the first previously decoded frame and the first current frame; Determine a first disparity vector and a first disparity residual according to the second previously decoded frame and the first current frame; The first motion vector, the first motion residual, the first disparity vector, and the first disparity residual are encoded to generate first encoded data of the first current frame.
[0005] A binocular image processing device, comprising: A binocular image data acquisition module is used to acquire binocular image data; the binocular image data includes a first previously decoded frame, a second previously decoded frame and a first current frame to be compressed; a motion vector residual module, configured to determine a first motion vector and a first motion residual according to the first previously decoded frame and the first current frame; a disparity vector residual module, configured to determine a first disparity vector and a first disparity residual according to the second previously decoded frame and the first current frame; The first encoding module is used to encode the first motion vector, the first motion residual, the first disparity vector and the first disparity residual to generate first encoded data of the first current frame.
[0006] A computer device comprises a memory, a processor and computer-readable instructions stored in the memory and executable on the processor, wherein the processor implements the above binocular image processing method when executing the computer-readable instructions.
[0007] One or more readable storage media storing computer-readable instructions, wherein when the computer-readable instructions are executed by one or more processors, the one or more processors execute the binocular image processing method as described above.
[0008] The above-mentioned binocular image processing method, device, computer equipment and storage medium compress the first current frame based on the first previously decoded frame and the second previously decoded frame to generate first encoded data of the first current frame, and utilize the correlation and redundancy of the two previously decoded frames to further improve the compression speed and the compression ratio. BRIEF DESCRIPTION OF THE DRAWINGS
[0009] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.
[0010] Figure 1 is a flow chart of a binocular image processing method according to an embodiment of the present invention; Figure 2 is a structural schematic diagram of a binocular image processing device in one embodiment of the present invention; Figure 3 is a schematic diagram of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION
[0011] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0012] The binocular image processing method provided in this embodiment can be applied to smart cars, smart wearable devices and other devices. Figure 1As shown, a binocular image processing method is provided, comprising the following steps S10 to S60.
[0013] S10, acquiring binocular image data; the binocular image data includes a first previously decoded frame, a second previously decoded frame, and a first current frame to be compressed.
[0014] Understandably, binocular image data refers to image data collected by a binocular camera, including two image frames, namely "left view" and "right view". Binocular image data can be a digital signal. There will be two pictures at each moment. The binocular image data includes the first previous decoded frame and the second previous decoded frame that have been encoded and decoded, and also includes the first current frame and the second current frame that have not been encoded. The serial numbers "first" and "second" before the image frame are used to identify the range of the camera. For example, the first previous decoded frame can be a left view image, and the second previous decoded frame can be a right view image; or, the second previous decoded frame can be a left view image, and the first previous decoded frame can be a right view image.
[0015] In some examples, the current frame is the tth frame, and the current frame includes a first current frame and a second current frame. If the first current frame is a left-view image and the second current frame is a right-view image, the current frame is represented as: ,in, is the first current frame, is the second current frame.
[0016] When encoding the first current frame, a decoded reference frame needs to be used. The decoded reference frame can be the previous frame of the current frame, that is, the first previously decoded frame and the second previously decoded frame. The decoded reference frame can be expressed as: ,in, is the first previously decoded frame, The second previous decoded frame.
[0017] S20. Determine a first motion vector and a first motion residual according to the first previously decoded frame and the first current frame.
[0018] It can be understood that motion estimation and motion compensation may be performed on the first previously decoded frame and the first current frame to generate a first motion vector and a first motion residual.
[0019] Optionally, step S20, i.e., determining a first motion vector and a first motion residual according to the first previously decoded frame and the first current frame, includes: S201, extracting a first previously decoded frame feature map from the first previously decoded frame through an autoencoder network; S202, extracting a first current frame feature map from the first current frame through the autoencoder network; S203. Determine the first motion vector according to the first previous frame decoding feature map and the first current frame feature map.
[0020] Understandably, the feature map of the first previously decoded frame can be extracted by the encoder of the auto-encoder network in the feature space to obtain the feature map of the first previously decoded frame. Unlike the pixel space, the feature space is a low-dimensional representation obtained by extracting features from the image. The auto-encoder network is a neural network model for unsupervised learning. The features in the feature map of the first previously decoded frame can be regarded as a meaningful abstract representation of the first previously decoded frame. These features can be high-level expressions such as edges, textures, and shapes of the image. Using the feature map of the first previously decoded frame to extract the first motion vector can reduce redundancy.
[0021] Similarly, the feature map of the first current frame can be extracted by the encoder of the autoencoder network in the feature space to obtain the feature map of the current frame. The features in the feature map of the current frame can be regarded as a meaningful abstract representation of the current frame.
[0022] The process of obtaining the first motion vector is generally called motion estimation. The conventional pixel space motion estimation is to divide each frame of the image into a number of non-overlapping macroblocks and assume that the displacement of all pixels in the macroblocks is the same. The motion estimation of this embodiment is performed in the feature space.
[0023] This embodiment uses the first previous frame decoding feature map and the first current frame feature map to determine and calculate the first motion vector of the first current frame, which can greatly reduce the amount of calculation and improve the encoding efficiency.
[0024] Optionally, step S203, i.e., determining the first motion vector according to the first previous frame decoding feature map and the first current frame feature map, includes: S2031, dividing the first current frame feature map into non-overlapping first feature blocks; the displacements of all pixels corresponding to the first feature blocks are the same; S2032, searching, according to a first motion matching rule, for a first most similar feature block of the first feature block within a first motion search range in the first previous frame decoding feature map; S2033: Determine the first motion vector according to the relative displacement between the first feature block and the first most approximate feature block.
[0025] It can be understood that dividing the first current frame feature map into non-overlapping first feature blocks means dividing the feature map according to a certain size, the pixels inside each first feature block are continuous, and there is no overlap between different first feature blocks.
[0026] The displacement of all pixels corresponding to the first feature block is the same, which means that after being processed by the autoencoder network, each first feature block in the first current frame feature map corresponds to a part of the current frame (original image), and these first feature blocks have the same displacement. This means that no matter where a first feature block appears in the first current frame, other identical first feature blocks will also appear in the corresponding first current frame feature map with the same displacement. This displacement of the first feature block is generally caused by the convolution operation. The convolution operation calculates on the current frame feature map by sliding a convolution kernel (filter), and each calculation corresponds to the features of a first feature block. During the sliding process, the displacement of the convolution kernel is fixed, so the displacement of the pixels corresponding to the first feature block is also the same. Since the first feature blocks have the same displacement, the autoencoder network can capture similar features at different positions on the first feature map. This way of sharing weights can improve the efficiency of the model, reduce the number of parameters, and help the model to be translationally invariant to the image.
[0027] For each first feature block, the first most similar feature block can be found in the first specified search range of the first previous frame decoding feature map according to the first preset matching rule. The first specified search range and the first preset matching rule can be set according to actual needs. The relative displacement between the first feature block and the first most similar feature block can be calculated, and the relative displacement is the first motion vector. Here, the first motion vector can be a global motion vector.
[0028] This embodiment realizes the acquisition of a first motion vector, and the first motion vector can be used to encode a first current frame.
[0029] Optionally, step S20, i.e., determining a first motion vector and a first motion residual according to the first previously decoded frame and the first current frame, further includes: S204, performing motion compensation on the first previously decoded frame feature map according to the first motion vector to generate a first current frame prediction feature map; S205. Determine the first motion residual according to the first current frame feature map and the first current frame prediction feature map.
[0030] Understandably, after obtaining the first motion vector, the first current frame prediction feature map can be obtained by motion compensation. Motion compensation is a method of describing the difference between adjacent frames. In the feature space, motion compensation describes how the feature block of the reference frame moves to a certain position in the first current frame feature map. Through motion compensation, the predicted value of the image feature map of the first current frame can be obtained, that is, the first current frame prediction feature map.
[0031] In the feature space, the first feature block in the first previously decoded frame feature map may be offset according to the first motion vector to obtain the first current frame predicted feature map.
[0032] Then, the first motion residual may be calculated in the feature space using the first current frame feature map and the first current frame predicted feature map.
[0033] This embodiment is used to generate a first motion residual, and the first motion residual can be used to encode a first current frame.
[0034] S30. Determine a first disparity vector and a first disparity residual according to the second previously decoded frame and the first current frame.
[0035] Understandably, the second previously decoded frame can be used as a reference frame to compare with the first current frame to calculate the first disparity vector. The calculation process of the first disparity vector is similar to the calculation process of the first motion vector, which will not be repeated here.
[0036] Optionally, step S30, i.e., determining a first disparity vector and a first disparity residual according to the second previously decoded frame and the first current frame, includes: S301, extracting a second previously decoded frame feature map from the second previously decoded frame through an autoencoder network; S302, searching for a first most similar disparity feature block of the first feature block within a first disparity search range in the second previous frame decoded feature map according to a first disparity matching rule; S303, determining the first disparity vector according to the relative displacement between the first feature block and the first most approximate disparity feature block; S304, performing disparity compensation on the second previously decoded frame feature map according to the first disparity vector to generate a first current frame disparity prediction feature map; S305 . Determine the first disparity residual according to the first current frame feature map and the first current frame disparity prediction feature map.
[0037] It is understandable that the feature map of the second previously decoded frame can be extracted by the encoder of the self-encoding network in the feature space to obtain the feature map of the second previously decoded frame.
[0038] The first feature block generated in step S2031 can be used to find the first most approximate disparity feature block of the first feature block in the first disparity search range of the second previous frame decoded feature map according to the first disparity matching rule, and then calculate the relative displacement between the first feature block and the first most approximate disparity feature block to obtain the first disparity vector, and then perform disparity compensation on the second previous decoded frame feature map according to the first disparity vector to generate the first current frame disparity prediction feature map, and finally calculate the difference between the first current frame feature map and the first current frame disparity prediction feature map to generate the first disparity residual. Among them, the first disparity matching rule and the first disparity search range can be set according to actual needs. The process of calculating the first disparity vector is similar to the process of calculating the first motion vector, and can refer to step S203. The process of calculating the first disparity residual is similar to the process of calculating the first motion residual, and can refer to steps S204~S205.
[0039] This embodiment is used to generate a first disparity vector and a first disparity residual, and the first disparity vector and the first disparity residual are used to encode a first current frame.
[0040] S40: Encode the first motion vector, the first motion residual, the first disparity vector, and the first disparity residual to generate first encoded data of the first current frame.
[0041] Understandably, after obtaining the first motion vector, the first motion residual, the first disparity vector and the first disparity residual, they can be encoded respectively to generate the first encoded data of the first current frame. In some examples, entropy coding can be used to encode the first motion vector, the first motion residual, the first disparity vector and the first disparity residual. Entropy coding can remove redundant information in the data to reduce the storage space or transmission bandwidth of the data. Entropy coding achieves efficient encoding of data by encoding symbols with higher frequency of occurrence for a shorter time and encoding symbols with lower frequency of occurrence for a longer time. Entropy coding can be Huffman coding.
[0042] In steps S10 to S40, the first current frame is compressed based on the first previously decoded frame and the second previously decoded frame to generate first encoded data of the first current frame, which utilizes the correlation and redundancy of the two previously decoded frames to further improve the compression speed and the compression ratio.
[0043] In some examples, the above binocular image processing method has the following applications in the field of smart finance: 1. Used for high-precision face recognition and identity authentication. By analyzing two images from different angles, a person's identity can be more accurately determined, improving the security of financial transactions, ATM withdrawals and other processes. 2. Used to detect the user's living features to prevent face recognition from being deceived. By observing indicators such as eye movements, pupil reactions, and expression changes, it can be determined whether a person is real and alive. 3. Used to monitor the security of financial venues. It can capture potential suspicious behaviors, abnormal events or unusual activities, and issue alarms in time to protect the interests of financial institutions and customers. 4. Using the stereoscopic vision capabilities of binocular cameras, more advanced video analysis and behavior recognition can be performed. For example, it can identify gatherings of people, queues, evaluate customer experience, analyze shopping behaviors, etc., providing financial institutions with deeper insights and decision-making basis. 5. Used to analyze customer behavior, emotions and reactions to optimize marketing strategies and improve customer service. By observing customers' facial expressions, eyes, behaviors and other characteristics, their needs and preferences can be understood, and more personalized financial products and services can be provided.
[0044] In some examples, the above binocular image processing method has the following applications in the field of smart medical care: 1. Used for three-dimensional reconstruction and navigation of human organs. By acquiring images from two different angles, an accurate three-dimensional model can be established to help doctors with surgical planning, intraoperative navigation and operation. 2. Using the imaging capability of the binocular camera, non-contact fundus examination and retinal imaging can be performed. This is very helpful for the early diagnosis and treatment of eye diseases (such as glaucoma, diabetic retinopathy, etc.). 3. Used for the diagnosis and monitoring of skin diseases. By capturing multi-angle, high-resolution images, doctors can more accurately assess the type, degree and changes of skin lesions, assist in diagnosis and formulate treatment plans. 4. Used for motion analysis and rehabilitation monitoring. By capturing the details and postures of human movements, motor skills, physical functions and rehabilitation progress can be evaluated, helping doctors and rehabilitation therapists to formulate personalized rehabilitation plans and monitor effects. 5. Used for gesture recognition and assisted operation. By recognizing gestures and actions, non-contact human-computer interaction can be achieved to help disabled people perform various operations in daily life (such as turning on and off lights, controlling TVs, etc.).
[0045] Optionally, after step S40, that is, after encoding the first motion vector, the first motion residual, the first disparity vector and the first disparity residual to generate first encoded data of the first current frame, the method further includes: S50: Decode the first encoded data to generate a first current decoded frame.
[0046] Understandably, the first encoded data may be decoded to generate a first current frame decoded feature map. Here, entropy coding may be used to decode the first encoded data, and the entropy decoding used is consistent with the probability model of the entropy coding in step S40. Then, the first current frame decoded feature map is restored to generate a restored image of the first current frame. .
[0047] This embodiment implements decoding of the first encoded data, obtains a restored image of the first current frame, and ensures high-precision restoration of the first current frame.
[0048] Optionally, the binocular image data further includes a second current frame to be compressed; After step S50, that is, after decoding the first coded data to generate the first current decoded frame, the method further includes: S60, determining a second motion vector and a second motion residual according to the second previously decoded frame and the second current frame; S70, determining a second disparity vector and a second disparity residual according to the first current decoded frame and the second current frame; S80. Encode the second motion vector, the second motion residual, the second disparity vector and the second disparity residual to generate second encoded data of the second current frame.
[0049] Understandably, after obtaining the first current decoded frame, the second motion vector and the second motion residual of the second current frame can be determined according to the second previous decoded frame and the second current frame. The process of calculating the second motion vector and the second motion residual can refer to steps S203 to S205, which will not be described in detail here. The second disparity vector and the second disparity residual of the second current frame can be determined according to the first current decoded frame and the second current frame. The process of calculating the second disparity vector and the second disparity residual can refer to steps S301 to S305, which will not be described in detail here.
[0050] This embodiment is used to generate the second coded data of the second current frame, and the coding of the two current frames is completed. After the coding of the second current frame is completed, the first subsequent frame and the second subsequent frame can be further coded. Therefore, this embodiment uses a left-right alternating method to encode the binocular image data, and utilizes the correlation and redundancy of the left and right frames of the image, which can further improve the compression speed and the compression ratio.
[0051] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.
[0052] In one embodiment, a binocular image processing device is provided, and the binocular image processing device corresponds one-to-one to the binocular image processing method in the above embodiment. Figure 2 As shown, the binocular image processing device includes a binocular image data acquisition module 10, a motion vector residual module 20, a disparity vector residual module 30 and a first encoding module 40. Each functional module is described in detail as follows: A binocular image data acquisition module 10 is used to acquire binocular image data; the binocular image data includes a first previously decoded frame, a second previously decoded frame and a first current frame to be compressed; A motion vector residual module 20, configured to determine a first motion vector and a first motion residual according to the first previously decoded frame and the first current frame; a disparity vector residual module 30, configured to determine a first disparity vector and a first disparity residual according to the second previously decoded frame and the first current frame; The first encoding module 40 is used to encode the first motion vector, the first motion residual, the first disparity vector and the first disparity residual to generate first encoded data of the first current frame.
[0053] Optionally, the binocular image processing device further includes: The first decoding module is used to decode the first encoded data to generate a first current decoded frame.
[0054] Optionally, the motion vector residual module 20 includes: A first previous frame feature map extraction unit, configured to extract a first previous decoded frame feature map from the first previous decoded frame through an autoencoder network; A first current frame feature map extraction unit, configured to extract a first current frame feature map from the first current frame through the autoencoder network; A first motion vector determination unit is used to determine the first motion vector according to the first previous frame decoding feature map and the first current frame feature map.
[0055] Optionally, the determining the first motion vector unit includes: A feature map segmentation unit, used for segmenting the first current frame feature map into non-overlapping first feature blocks; the displacements of all pixels corresponding to the first feature blocks are the same; A similar feature block searching unit is used to search, according to a first motion matching rule, a first most similar feature block of the first feature block in a first motion search range of the first previous frame decoding feature map; A first motion vector generating unit is configured to determine the first motion vector according to a relative displacement between the first feature block and the first most similar feature block.
[0056] Optionally, the motion vector residual module 20 further includes: a motion compensation unit, configured to perform motion compensation on the first previously decoded frame feature map according to the first motion vector to generate a first current frame prediction feature map; Determine a first motion residual unit, which is used to determine the first motion residual according to the first current frame feature map and the first current frame prediction feature map.
[0057] Optionally, the disparity vector residual module 30 includes: A second previous frame feature map extraction unit, configured to extract a second previous decoded frame feature map from the second previous decoded frame through an autoencoder network; Searching for a first most similar disparity feature block of the first feature block within a first disparity search range in the second previous frame decoded feature map according to a first disparity matching rule; a similar disparity feature block searching unit, configured to determine the first disparity vector according to a relative displacement between the first feature block and the first most similar disparity feature block; a disparity compensation unit, configured to perform disparity compensation on the second previously decoded frame feature map according to the first disparity vector to generate a first current frame disparity prediction feature map; A first disparity residual unit is determined, which is used to determine the first disparity residual according to the first current frame feature map and the first current frame disparity prediction feature map.
[0058] Optionally, the binocular image data further includes a second current frame to be compressed; and the binocular image processing device further includes: A second motion vector residual module, configured to determine a second motion vector and a second motion residual according to the second previously decoded frame and the second current frame; A second disparity vector residual module, configured to determine a second disparity vector and a second disparity residual according to the first current decoded frame and the second current frame; The second encoding module is used to encode the second motion vector, the second motion residual, the second disparity vector and the second disparity residual to generate second encoded data of the second current frame.
[0059] For the specific definition of the binocular image processing device, please refer to the definition of the binocular image processing method above, which will not be repeated here. Each module in the above binocular image processing device can be implemented in whole or in part by software, hardware and a combination thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.
[0060] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 3 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a readable storage medium and an internal memory. The readable storage medium stores an operating system, computer-readable instructions and a database. The internal memory provides an environment for the operation of the operating system and computer-readable instructions in the readable storage medium. The database of the computer device is used to store data involved in the binocular image processing method. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer-readable instructions are executed by the processor, a binocular image processing method is implemented. The readable storage medium provided in this embodiment includes a non-volatile readable storage medium and a volatile readable storage medium.
[0061] In one embodiment, a computer device is provided, including a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, wherein the processor executes the computer-readable instructions to implement the following steps: Acquire binocular image data; the binocular image data includes a first previously decoded frame, a second previously decoded frame, and a first current frame to be compressed; Determine a first motion vector and a first motion residual according to the first previously decoded frame and the first current frame; Determine a first disparity vector and a first disparity residual according to the second previously decoded frame and the first current frame; The first motion vector, the first motion residual, the first disparity vector, and the first disparity residual are encoded to generate first encoded data of the first current frame.
[0062] In one embodiment, one or more computer-readable storage media storing computer-readable instructions are provided. The readable storage media provided in this embodiment include non-volatile readable storage media and volatile readable storage media. The readable storage media store computer-readable instructions, and when the computer-readable instructions are executed by one or more processors, the following steps are implemented: Acquire binocular image data; the binocular image data includes a first previously decoded frame, a second previously decoded frame, and a first current frame to be compressed; Determine a first motion vector and a first motion residual according to the first previously decoded frame and the first current frame; Determine a first disparity vector and a first disparity residual according to the second previously decoded frame and the first current frame; The first motion vector, the first motion residual, the first disparity vector, and the first disparity residual are encoded to generate first encoded data of the first current frame.
[0063] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing related hardware through computer-readable instructions, and the computer-readable instructions can be stored in a non-volatile readable storage medium or a volatile readable storage medium. When the computer-readable instructions are executed, they may include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.
[0064] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.
[0065] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.
Claims
1. A binocular image processing method, characterized in that: include: Acquire binocular image data; the binocular image data includes a first previously decoded frame, a second previously decoded frame, and a first current frame to be compressed; Determine a first motion vector and a first motion residual according to the first previously decoded frame and the first current frame; Determine a first disparity vector and a first disparity residual according to the second previously decoded frame and the first current frame; The first motion vector, the first motion residual, the first disparity vector, and the first disparity residual are encoded to generate first encoded data of the first current frame.
2. The binocular image processing method according to claim 1, characterized in that: After encoding the first motion vector, the first motion residual, the first disparity vector and the first disparity residual to generate first encoded data of the first current frame, the method further includes: The first encoded data is decoded to generate a first current decoded frame.
3. The binocular image processing method according to claim 1, characterized in that: The determining a first motion vector and a first motion residual according to the first previously decoded frame and the first current frame comprises: Extracting a first previously decoded frame feature map from the first previously decoded frame through an autoencoder network; Extracting a first current frame feature map from the first current frame through the autoencoder network; The first motion vector is determined based on the first previous frame decoded feature map and the first current frame feature map.
4. The binocular image processing method according to claim 3, characterized in that: The determining the first motion vector according to the first previous frame decoding feature map and the first current frame feature map comprises: The first current frame feature map is divided into non-overlapping first feature blocks; the displacements of all pixels corresponding to the first feature blocks are the same; Finding a first most similar feature block of the first feature block within a first motion search range of the first previous frame decoding feature map according to a first motion matching rule; The first motion vector is determined according to a relative displacement between the first feature block and the first most similar feature block.
5. The binocular image processing method according to claim 4, characterized in that: The determining of a first motion vector and a first motion residual according to the first previously decoded frame and the first current frame further includes: Performing motion compensation on the first previously decoded frame feature map according to the first motion vector to generate a first current frame prediction feature map; The first motion residual is determined according to the first current frame feature map and the first current frame predicted feature map.
6. The binocular image processing method according to claim 4, characterized in that: The determining a first disparity vector and a first disparity residual according to the second previously decoded frame and the first current frame comprises: Extracting a second previously decoded frame feature map from the second previously decoded frame by an autoencoder network; Searching for a first most similar disparity feature block of the first feature block within a first disparity search range in the second previous frame decoded feature map according to a first disparity matching rule; Determine the first disparity vector according to the relative displacement between the first feature block and the first most approximate disparity feature block; Performing disparity compensation on the second previously decoded frame feature map according to the first disparity vector to generate a first current frame disparity prediction feature map; The first disparity residual is determined according to the first current frame feature map and the first current frame disparity prediction feature map.
7. The binocular image processing method according to claim 2, characterized in that: The binocular image data also includes a second current frame to be compressed; After decoding the first coded data to generate a first current decoded frame, the method further includes: determining a second motion vector and a second motion residual according to the second previously decoded frame and the second current frame; Determine a second disparity vector and a second disparity residual according to the first current decoded frame and the second current frame; The second motion vector, the second motion residual, the second disparity vector and the second disparity residual are encoded to generate second encoded data of the second current frame.
8. A binocular image processing device, characterized in that: include: A binocular image data acquisition module is used to acquire binocular image data; the binocular image data includes a first previously decoded frame, a second previously decoded frame and a first current frame to be compressed; a motion vector residual module, configured to determine a first motion vector and a first motion residual according to the first previously decoded frame and the first current frame; a disparity vector residual module, configured to determine a first disparity vector and a first disparity residual according to the second previously decoded frame and the first current frame; The first encoding module is used to encode the first motion vector, the first motion residual, the first disparity vector and the first disparity residual to generate first encoded data of the first current frame.
9. A computer device comprising a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, characterized in that: When the processor executes the computer-readable instructions, the binocular image processing method according to any one of claims 1 to 7 is implemented.
10. One or more readable storage media storing computer-readable instructions, wherein when the computer-readable instructions are executed by one or more processors, the one or more processors execute the binocular image processing method according to any one of claims 1 to 7.