Binocular image encoding method and apparatus, storage medium, computer program product

Abstract

This application discloses a binocular image encoding method, apparatus, storage medium, and computer program product. The method includes: acquiring first environmental parameters of a first region where a first binocular camera is located within a current time period; determining first acquisition parameters of the first binocular camera within the current time period based on the first environmental parameters; acquiring a binocular image of the first region acquired by the first binocular camera according to the first acquisition parameters; extracting image texture features from the binocular image; and determining an image encoding strategy for the binocular image based on the image texture features, so as to encode the binocular image according to the image encoding strategy. This application solves the technical problems of related technologies where setting the acquisition parameters of the binocular camera solely based on human experience leads to poor image quality, and encoding the acquired image according to fixed encoding parameters results in low encoding efficiency and poor image integrity after encoding.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and more specifically, to a binocular image encoding method and apparatus, storage medium, and computer program product. Background Technology

[0002] A binocular camera is a camera system that mimics the principle of human binocular vision. It consists of two cameras positioned side-by-side or separated by a certain baseline distance. These two cameras simultaneously capture the same scene, but due to differences in their viewing angles, the captured images exhibit parallax. Utilizing this parallax information, combined with the principle of triangulation, a binocular camera system can calculate the depth information of the object being photographed, thereby generating image data with three-dimensional spatial perception, enabling more accurate and reliable target detection and tracking.

[0003] However, binocular cameras also face some challenges in practical applications. For example, during video acquisition, cameras often lack effective adaptive adjustment capabilities when faced with environmental factors such as changes in lighting, color temperature differences, and noise interference, leading to problems such as overexposure, color distortion, or excessive noise in the acquired video. During video transmission, traditional video coding algorithms often use uniform coding parameters, which cannot adapt to the texture characteristics of different objects in the scene, potentially resulting in low coding efficiency in certain areas and an inability to balance video quality and bandwidth usage.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This application provides a binocular image encoding method and apparatus, storage medium, and computer program product to at least solve the technical problems of related technologies, which result in poor image quality due to setting the acquisition parameters of binocular cameras solely based on human experience, and low encoding efficiency and poor image integrity due to encoding the acquired images according to fixed encoding parameters.

[0006] According to one aspect of the embodiments of this application, a binocular image encoding method is provided, comprising: acquiring first environmental parameters of a first region where a first binocular camera is located within a current time period; determining first acquisition parameters of the first binocular camera within the current time period based on the first environmental parameters; acquiring binocular images of the first region acquired by the first binocular camera according to the first acquisition parameters; extracting image texture features of the binocular images, and determining an image encoding strategy for the binocular images based on the image texture features, so as to encode the binocular images according to the image encoding strategy.

[0007] Optionally, determining the first acquisition parameters of the first binocular camera within the current time period based on the first environmental parameters includes: analyzing the first environmental parameters using a pre-trained machine learning model to obtain the first acquisition parameters of the first binocular camera within the current time period; wherein the machine learning model is obtained by iteratively training a preset initial learning model using a training sample set and a sample label set, wherein the training sample set includes multiple training samples, and each training sample is a second environmental parameter of the second area where the second binocular camera is located within a historical time period, and the sample label set includes a sample label corresponding to each training sample, and the sample label is a second acquisition parameter optimized from the initial acquisition parameters of the second binocular camera within a historical time period.

[0008] Optionally, the second environmental parameters include at least one of the following: light intensity, color temperature, distance between the second binocular camera and the second area, motion state of objects within the second area, and background complexity of the second area; the second acquisition parameters include at least one of the following: exposure time, sensitivity, white balance, aperture size, frame rate, and exposure.

[0009] Optionally, the binocular image includes: a left-eye image and a right-eye image. Extracting image texture features from the binocular image includes: preprocessing the binocular image, wherein the preprocessing includes at least one of the following: distortion correction, epipolar correction, geometric transformation, image enhancement, and wavelet thresholding denoising; detecting multiple feature points within the binocular image using a pre-defined feature point detection algorithm, wherein the feature point detection algorithm includes at least one of the following: scale-invariant feature transform matching algorithm, directional fast feature and rotational binary improved feature algorithm, and accelerated robust feature algorithm; determining the feature vector of each feature point; and determining the image texture features of the binocular image based on the feature vectors of multiple feature points within the binocular image.

[0010] Optionally, determining the feature vector of the feature point includes: determining a local image centered on the feature point and with a preset distance as the radius; extracting image features from the local image and using the obtained local image feature vector as the feature vector of the feature point.

[0011] Optionally, the multiple feature points include: multiple first feature points in the left-eye image and multiple second feature points in the right-eye image. The determination of image texture features of the binocular image based on the feature vectors of the multiple feature points in the binocular image includes: determining the similarity between the feature vectors of the multiple first feature points and the feature vectors of the multiple second feature points; when the similarity between the feature vectors of the first target feature points and the feature vectors of the second target feature points within the multiple first feature points is greater than a preset similarity threshold, a system of linear equations is constructed using the first and second target feature points, and the essential matrix is ​​estimated by solving the system of linear equations. The essential matrix is ​​used to describe the geometric relationship between the two cameras of the first binocular camera. Based on the essential matrix, the first and second target feature points, a three-dimensional image of the binocular image is reconstructed using triangulation, and the image texture features of the three-dimensional image are extracted using a preset feature extraction algorithm.

[0012] Optionally, determining the image coding strategy for the stereo image based on image texture features includes: determining the texture complexity of the 3D image of the stereo image based on image texture features, and judging the relationship between the texture complexity and a preset complexity threshold; if the texture complexity is not higher than the preset complexity threshold, determining the image coding strategy as directly encoding the 3D image of the stereo image using inter-frame coding prediction technology; if the texture complexity is higher than the preset complexity threshold, dividing the 3D image of the stereo image into multiple 3D sub-images, and determining the image coding strategy as encoding each 3D sub-image using intra-frame coding prediction technology.

[0013] According to another aspect of the embodiments of this application, a binocular image encoding device is also provided, comprising: a first acquisition module, configured to acquire first environmental parameters of a first region where a first binocular camera is located within a current time period; a determination module, configured to determine first acquisition parameters of the first binocular camera within the current time period based on the first environmental parameters; a second acquisition module, configured to acquire binocular images obtained by the first binocular camera from the first region according to the first acquisition parameters; and an encoding module, configured to extract image texture features of the binocular images and determine an image encoding strategy for the binocular images based on the image texture features, so as to encode the binocular images according to the image encoding strategy.

[0014] According to another aspect of the embodiments of this application, a non-volatile storage medium is also provided, the non-volatile storage medium including a stored computer program, wherein the device where the non-volatile storage medium is located executes the above-described binocular image encoding method by running the computer program.

[0015] According to another aspect of the embodiments of this application, a computer program product is also provided, the computer program product including a stored computer program, wherein the computer program implements the above-described binocular image encoding method when executed by a processor.

[0016] In this embodiment, the first environmental parameters of the first area where the first binocular camera is located are obtained within the current time period; the first acquisition parameters of the first binocular camera within the current time period are determined based on the first environmental parameters to ensure the stability of the acquired image quality; high-quality binocular images of the first area are acquired by the first binocular camera according to the first acquisition parameters; image texture features of the binocular images are extracted, and an image encoding strategy for the binocular images is determined based on the image texture features, so as to encode the binocular images according to the image encoding strategy. This not only allows for the use of lower quantization parameters to preserve image details, but also effectively reduces the amount of encoded data, reducing transmission and storage costs. Especially in scenarios with limited network resources, the adoption of this adaptive encoding strategy can significantly improve the efficiency and performance of the video surveillance system, thereby solving the technical problems of related technologies that set the acquisition parameters of the binocular camera based solely on human experience, resulting in poor quality of the acquired images, and that encoding the acquired images according to fixed encoding parameters, resulting in low encoding efficiency and poor integrity of the encoded images. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0018] Figure 1 This is a hardware structure block diagram of an optional computer terminal (or mobile device) for implementing a binocular image encoding method according to an embodiment of this application;

[0019] Figure 2 This is a flowchart illustrating an optional binocular image encoding method according to an embodiment of this application;

[0020] Figure 3 This is a schematic diagram of the structure of an optional binocular image encoding device according to an embodiment of this application;

[0021] Figure 4 This is a schematic diagram of the structure of an optional electronic device according to an embodiment of this application. Detailed Implementation

[0022] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0023] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0024] Furthermore, all information and data (including but not limited to user device information, user personal information, etc.) involved in this application are information and data authorized by the user or fully authorized by all parties. For example, this system has an interface with the relevant user or organization. Before obtaining relevant information, it needs to send an acquisition request to the aforementioned user or organization through the interface, and obtain the relevant information after receiving consent from the aforementioned user or organization.

[0025] Example 1

[0026] According to an embodiment of this application, an embodiment of a binocular image encoding method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0027] The methods and embodiments provided in this application can be executed on mobile terminals, computer terminals, or similar computing devices. Figure 1 A hardware block diagram of a computer terminal (or mobile device) for implementing a binocular image encoding method is shown. Figure 1As shown, the computer terminal 10 (or mobile device 10) may include one or more processors 102 (shown as 102a, 102b, ..., 102n in the figure) 102 (processor 102 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0028] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0029] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the binocular image encoding method in this embodiment. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby implementing the aforementioned binocular image encoding method of the application. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0030] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0031] The display can be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 10 (or mobile device).

[0032] Under the above operating environment, Figure 2 This is a flowchart illustrating an optional binocular image encoding method according to an embodiment of this application, as shown below. Figure 2 As shown, the method includes at least steps S202-S208, wherein:

[0033] Step S202: Obtain the first environmental parameters of the first area where the first binocular camera is located within the current time period.

[0034] In the technical solution provided in step S202, the aforementioned first environmental parameter refers to real-time data collected about the environment surrounding the first binocular camera. This data includes, but is not limited to: light intensity, color temperature, distance between the first binocular camera and the first area, motion state of objects within the first area, and background complexity of the first area. These parameters can be detected and analyzed using sensors built into the camera, image processing algorithms, or external sensors. Specifically:

[0035] Illumination intensity is an indicator of the strength of ambient light, which can be detected by an ambient light sensor deployed on the first binocular camera.

[0036] Color temperature is used to indicate the color tendency of ambient light sources, and it can be detected by XYZ color sensors or other color sensors.

[0037] The distance between the first binocular camera and the first area, i.e. the distance between the first binocular camera and the object being photographed, can be determined by triangulation.

[0038] The motion state of objects within the first region includes their speed and direction. This can be achieved by continuously acquiring image sequences within the first region using a monocular camera and identifying moving objects by comparing pixel differences between consecutive image frames. Common methods include frame difference and optical flow. Frame difference compares pixel changes between consecutive frames to identify motion, while optical flow estimates the direction and speed of pixel movement, making it suitable for motion detection in dynamic backgrounds. Next, moving objects are separated from the image. Common segmentation methods include threshold-based segmentation, region-based segmentation, and edge-based segmentation. Then, feature points, such as corners and edges, are detected on the moving object and matched within consecutive frames to estimate its speed and direction. Feature point matching algorithms include SIFT (Scale Invariant Feature Transform), SURF (Speed-Up Robust Feature Transform), and ORB (Directional Fast Feature Transform and Rotational Binary Improved Feature Transform). Finally, the motion state of the object is classified based on its speed, direction, and acceleration. For example, motion states can be categorized as stationary, slow movement, fast movement, and acceleration or deceleration.

[0039] The background complexity of the first region can be evaluated by acquiring an image of the first region and calculating the image entropy, or by using a deep learning model.

[0040] Step S204: Determine the first acquisition parameters of the first binocular camera in the current time period based on the first environmental parameters.

[0041] As an optional implementation, in the technical solution provided in step S204 above, the method may include: using a pre-trained machine learning model to analyze the first environmental parameters to obtain the first acquisition parameters of the first binocular camera in the current time period, so as to optimize the image quality.

[0042] The first acquisition parameters mentioned above include, but are not limited to: exposure time, ISO, white balance, aperture size, frame rate, and exposure.

[0043] Specifically, light intensity directly affects the camera's adjustment of exposure time and ISO (International Organization for Standardization value, sensitivity) to ensure that images maintain good clarity and brightness under different lighting conditions; color temperature usually directly affects the camera's white balance to make image colors closer to reality and avoid color cast issues; the distance between the first binocular camera and the first area affects the camera's focal length to ensure that objects in the first area are clearly visible; the motion state of objects in the first area usually directly affects the camera's frame rate to ensure the clarity of object motion details; the background complexity of the first area usually affects the exposure of the first binocular camera to ensure the details of subsequently acquired images.

[0044] Optionally, the aforementioned machine learning model is obtained by iteratively training a preset initial learning model using a training sample set and a sample label set, in order to identify the relationship between different environmental conditions and camera acquisition parameters.

[0045] The training sample set and sample label set are determined based on historical data. The training sample set includes multiple training samples, and each training sample represents the second environmental parameters of the second area where the second binocular camera is located within a historical time period. The sample label set includes the sample label corresponding to each training sample, and the sample label represents the second acquisition parameters after optimizing the initial acquisition parameters of the second binocular camera within the historical time period. That is, the sample label represents the second acquisition parameters adjusted manually or automatically based on the second environmental parameters and video content analysis of the second binocular camera within the historical time period, and the knowledge gained from continuous experimentation and experience accumulation in actual operation. The aim is to provide the best or better image quality. Therefore, the second acquisition parameters are the optimal camera settings under the second environmental parameters.

[0046] By training the machine learning model using the aforementioned training samples and sample labels, the model can learn the optimal mapping relationship between environmental parameters and camera acquisition parameters. This allows the model to automatically predict and dynamically adjust camera parameters in new environments to achieve the best video acquisition results.

[0047] In step S206, the machine learning model can be loaded into memory. For example, the raw data of the machine learning model can be loaded from non-volatile memory into volatile memory so that the processor can run the machine learning model. The raw data of the machine learning model refers to unprocessed data, typically including the model's parameters and structural data. The structural data can be the computational relationships based on the parameters, such as the forward propagation computational relationships between intermediate layers or neurons. Specifically, the structural data can include structure-related code of the machine learning model, such as code used to perform related calculations between intermediate layers or neurons. Alternatively, a region can be partitioned in memory for loading the machine learning model, which can include a structural data storage area and a parameter storage area. The structural data storage area stores the structure-related code, and the parameters referenced by it can be pointed to by pointers to the addresses of specific parameters in the parameter storage area. During the training process of the machine learning model, it may be necessary to frequently update the parameters; in this case, the parameter values ​​in the parameter storage area can be updated.

[0048] Step S206: Obtain the binocular image of the first region obtained by the first binocular camera according to the first acquisition parameters.

[0049] In the technical solution provided in step S206, the parameters of the first binocular camera are set according to the first acquisition parameters determined in step S204 above, to ensure that these parameters can adapt to the current environmental conditions, thereby improving image quality and effect. After the adjustment is completed, the first binocular camera begins to synchronously acquire binocular images (including left-eye and right-eye images) in the first area. Synchronization means that the two cameras of the first binocular camera simultaneously or almost simultaneously capture images of the first area at the same moment, ensuring that the acquired images have temporal consistency, so as to facilitate subsequent stereo matching.

[0050] Step S208: Extract image texture features from the stereo image and determine the image encoding strategy for the stereo image based on the image texture features, so as to encode the stereo image according to the image encoding strategy.

[0051] As an optional implementation, in the technical solution provided in step S208 above, the method may include:

[0052] Step S2081: Preprocess the binocular images.

[0053] Because the acquired binocular images may contain noise or distortion, preprocessing is necessary. This preprocessing includes, but is not limited to, distortion removal, epipolar line correction, geometric transformation, image enhancement, and wavelet threshold denoising to improve image quality. Specifically:

[0054] Distortion correction is a process designed to eliminate image distortions caused by lens physical characteristics or the imaging system, such as radial or pincushion distortion and tangential distortion. It typically involves using the camera's distortion coefficients and intrinsic parameters (such as focal length and principal point position) to correct the image, making straight lines appear straighter and eliminating barrel or pincushion effects.

[0055] Epipolar correction is used to project the images from the left and right cameras onto a common image plane, so that the corresponding feature points are located on the same horizontal line.

[0056] Geometric transformations, including rotation, translation, scaling, and perspective transformations, are used to adjust the viewpoint of an image or correct it to suit specific application requirements. For example, an image can be rotated to the correct orientation, a tilted image can be corrected to a front view, or the image size can be adjusted to fit different display or processing needs.

[0057] Image enhancement improves the visual appeal of an image or highlights certain features by adjusting its contrast, brightness, saturation, and other parameters. Common image enhancement techniques include histogram equalization, contrast stretching, and sharpening filtering, used to improve image readability and analytical performance.

[0058] Wavelet thresholding denoising transforms the image into the wavelet domain, eliminates noise in the wavelet coefficients by setting a threshold, and then transforms the processed coefficients back into the image domain to obtain the denoised image, which can better preserve the edge and texture information of the image.

[0059] Step S2082: Detect multiple feature points in the binocular image using a preset feature point detection algorithm.

[0060] The aforementioned feature points refer to images or targets that can be represented in an invariant form, either identically or very similarly, in other similar images containing the same scene or target. In other words, for the same object or scene, if multiple images are captured from different angles and identical areas can be identified, these points or blocks are called feature points, such as corner points, edge points, or other significant image structural points. For binocular images, the corresponding multiple feature points include: multiple first feature points in the left-eye image and multiple second feature points in the right-eye image. Furthermore, the aforementioned feature point detection algorithms include, but are not limited to: Scale Invariant Feature Transform Matching (SIFT), Directional Fast Feature Algorithm (ORB) and Rotation-Based Binary Improved Feature Algorithm (SURF), and Accelerated Robust Feature Algorithm (SURF).

[0061] Step S2083: For each feature point, determine the feature vector of the feature point.

[0062] Optionally, the technical solution provided in step S2083 above can be implemented through the following steps: determining a local image centered on the feature point and with a preset distance as the radius; extracting image features from the local image, and using the obtained local image feature vector as the feature vector of the feature point (also known as the descriptor of the feature point).

[0063] This can be understood as follows: For each detected feature point, firstly, a circular or square region with a predetermined radius centered on the feature point is defined. The choice of this predetermined distance needs to consider the scale of the feature point and the image resolution to ensure sufficient image details are captured while avoiding computationally overly complex regions. Then, within the defined local image region, an image feature extraction algorithm is applied to extract features. This algorithm includes, but is not limited to, Gray-Level Co-occurrence Matrix (GLCM), Local Binary Pattern (LBP), Histogram of Oriented Gradients (HOG), Scale-Invariant Feature Transform (SIFT) descriptors, or other local region-based feature description algorithms. Finally, the features extracted from the local region are combined into a feature vector. This vector contains multidimensional data describing the image characteristics surrounding the feature point. It should be noted that the dimension and content of the feature vector depend on the chosen feature extraction algorithm. For example, the SIFT descriptor is typically a 128-dimensional feature vector containing scale-invariant features of the local image.

[0064] Step S2084: Determine the image texture features of the stereo image based on the feature vectors of multiple feature points within the stereo image.

[0065] Optionally, the technical solution provided in step S2084 above can be implemented through the following steps:

[0066] Step 1: Determine the similarity between the feature vectors of multiple first feature points and the feature vectors of multiple second feature points.

[0067] The similarity mentioned above can be calculated using metrics such as Euclidean distance, Hamming distance, or cosine similarity. The goal of similarity calculation is to find the best match between two feature points, that is, to find the pair of feature points describing the same physical point in the left and right images.

[0068] The second step is to construct a system of linear equations using the first and second target feature points when the similarity between the feature vectors of the first target feature points and the feature vectors of the second target feature points within multiple first feature points is greater than a preset similarity threshold. Then, the essential matrix is ​​estimated by solving the system of linear equations.

[0069] Specifically, when the similarity between the feature vectors of the first target feature point within multiple first feature points and the feature vectors of the second target feature point within multiple second feature points is greater than a preset similarity threshold, this pair of feature points can be considered as projections of the same physical point from different viewpoints. Subsequently, using the matched feature point pairs, the essential matrix is ​​estimated using an eight-point or five-point method, where the essential matrix describes the geometric relationship between the two cameras of the first binocular camera.

[0070] Taking the eight-point method for estimating the essential matrix as an example, firstly, given 8 pairs of matching feature points, for each pair of matching points (x1, y1) and (x2, y2), the following equation is constructed:

[0071] x1x2e 11 +x1y2e 12 +x1e 13 +y1x2e 21 +y1y2e 22 +y1e 23 +x2e 31 +y2e 32 +e 33 =0

[0072] The above equation can also be written in matrix form as follows:

[0073] Qe = 0

[0074] Here, Q is an 8x9 matrix, and e is the column vector of the essential matrix E containing 9 unknowns. Solving Q using singular decomposition yields the least-squares solution, i.e., performing singular decomposition on Q, resulting in Q = U∑V T We then take the last column of V as the least-squares solution of E. Since the singular values ​​of the essential matrix E should be of the form σ,σ,0, we need to constrain the obtained E to ensure that its singular values ​​satisfy this condition.

[0075] Step 3: Based on the essential matrix, the first target feature point, and the second target feature point, reconstruct the 3D image of the binocular image using triangulation, and extract the image texture features of the 3D image using a preset feature extraction algorithm.

[0076] Specifically, since the essential matrix is ​​calculated based on the relative rotation and translation between the two cameras of the first stereo camera, as well as the camera's intrinsic parameters (such as focal length and principal point position), it contains the linear constraints that all matching feature point pairs between the two cameras must satisfy. Therefore, a series of linear equations are set up using the essential matrix and the matching feature point pairs to solve for the relative rotation and translation between the two cameras of the first stereo camera. Subsequently, triangulation is performed using these solved parameters to calculate the three-dimensional coordinates of each matching feature point pair in the first region. The basic principle of triangulation is that, assuming the relative position and orientation of the two cameras are known (obtained through the essential matrix and camera intrinsic parameters), and the pixel coordinates of the feature points in the left and right images, the actual position of the feature points in three-dimensional space can be calculated through geometric relationships (such as similar triangles or least squares method). Furthermore, these feature points in three-dimensional space are geometrically processed to form a three-dimensional point cloud or mesh model, which can then generate a three-dimensional visualization of the entire scene.

[0077] Furthermore, after learning the image texture features of the 3D image reconstructed using stereo images, the image encoding strategy for stereo images can be determined as follows:

[0078] First, the texture complexity of the 3D image of the binocular image is determined based on the image texture features, and the relationship between the texture complexity and a preset complexity threshold is judged. Specifically, the texture complexity can be evaluated by analyzing the variance of pixels within the coding block, edge direction, or other texture features; the relationship between the texture complexity of the coding block and the preset complexity threshold is then determined.

[0079] If the texture complexity is not higher than a preset complexity threshold, the current size of the 3D image can be used as the encoding size due to its low texture complexity. Therefore, the image encoding strategy is to directly encode the 3D image of the stereo image using inter-frame coding prediction technology.

[0080] When the texture complexity exceeds a preset complexity threshold, the 3D image of the stereo image can be divided into multiple 3D sub-images due to its high texture complexity, allowing for the capture of details using smaller images. Therefore, the image coding strategy involves encoding each 3D sub-image using intra-frame coding prediction technology.

[0081] Inter-frame coding prediction technology utilizes the spatiotemporal redundancy between frames in a video sequence. It uses pixels from neighboring encoded images to predict pixels in the current image, thus removing redundant information in the time-frequency domain. Since video sequences typically contain strong temporal correlations, their prediction residuals are usually close to zero. The calculated residual signals are then transformed, quantized, and entropy-coded to achieve efficient video compression. The specific implementation process includes the following steps:

[0082] S1, Select one or more reference frames from the previously encoded frames (i.e., images). The selection of reference frames is based on their similarity to the current frame in terms of content and prediction performance. Typically, the most recent frames are selected from the decoding buffer as candidates.

[0083] S2, for each coded block in the current frame (divided into multiple coded blocks according to the complexity of texture features in different regions of the 3D image of the stereo image), search for the region in the reference frame that best matches it. Specifically, this search is usually performed within the search window of the reference frame, and the best matching position is determined by comparing the differences in pixel values ​​between blocks (such as using mean square error (MSE), absolute difference (SAD), etc. as matching metrics). The difference between the best matching position and the position of the current coded block is the motion vector, representing the displacement of the current block in time.

[0084] S3, the matching block in the reference frame is shifted pixel-level according to the motion vector to obtain the predicted frame. This process is called motion compensation, which can compensate for pixel changes caused by object motion and reduce temporal redundancy;

[0085] S4 calculates the pixel-level difference between the block in the current frame and the corresponding block in the motion-compensated prediction frame to obtain the prediction residual. The prediction residual represents the difference information between the current frame and the prediction frame, and needs further encoding and transmission.

[0086] S5. The obtained residual values ​​are transformed (e.g., DCT, wavelet transform), quantized, and entropy encoded to generate a bitstream. Among these processes, the transformation converts the spatial domain residuals to the frequency domain, making it easier to remove redundancy; quantization reduces the number of bits represented, thereby reducing the amount of data transmitted; and entropy encoding (e.g., Huffman coding, arithmetic coding) further compresses the data and improves coding efficiency.

[0087] Intra-frame coding prediction technology utilizes the pixel correlation within a single frame in video coding to reduce spatial redundancy and improve coding efficiency. Its specific implementation process includes the following steps:

[0088] S1, for each coded block in the current frame (divided into multiple coded blocks according to the texture feature complexity of different regions within the 3D image of the stereo image), the pixel value of the current block is predicted using the pixel values ​​of the already coded neighboring blocks. The prediction method can be based on the left neighboring block, the upper neighboring block, or a combination of both.

[0089] S2, determine the pattern with the smallest prediction error as the prediction pattern of the current block, wherein the prediction pattern includes but is not limited to: horizontal prediction, vertical prediction, DC prediction (average prediction), corner prediction and diagonal prediction, etc., and the selection of the prediction pattern is based on the similarity between the current block and these neighboring blocks.

[0090] S3 calculates the residual between the current coded block and the predicted block, i.e., the difference in pixel values. Similar to inter-frame prediction, this residual value needs to be further encoded and transmitted, but in intra-frame coded prediction, this residual value is relative to the values ​​of adjacent pixels within the same frame;

[0091] S4 transforms the prediction residual (such as Discrete Cosine Transform (DCT) or Integer Transform), quantizes it, and entropy codes it to generate a bit stream.

[0092] Based on the scheme defined in steps S202 to S208 above, it can be understood that in this embodiment, by real-time monitoring of the first environmental parameters of the first area where the first binocular camera is located within the current time period, the first acquisition parameters of the first binocular camera are intelligently adjusted to adapt to environmental changes, ensuring the stability of video quality and optimization of visual effects. Subsequently, based on the optimized acquisition parameters, a high-quality binocular image of the area is acquired. Furthermore, by extracting the image texture features of the binocular image and determining the image encoding strategy based on these features, the encoding parameters are adjusted according to different areas of the image. While ensuring video quality, the amount of encoded data is effectively reduced, and transmission and storage costs are reduced. Especially in scenarios with limited network resources, the adoption of this adaptive encoding strategy can significantly improve the efficiency and performance of the video surveillance system. In summary, this application embodiment, through real-time monitoring and adaptive adjustment of environmental parameters, combined with image texture feature analysis and encoding strategy optimization, achieves the technical effects of high-quality video acquisition and efficient video encoding in dynamic environments, meeting the requirements of high precision and low latency for video surveillance and remote vision applications.

[0093] Example 2

[0094] Based on Embodiment 1 of this application, an embodiment of a binocular image encoding device is also provided, which executes the binocular image encoding method described in the above embodiment during operation. Wherein, Figure 3 This is a schematic diagram of an optional binocular image encoding device according to an embodiment of this application, as shown below. Figure 3As shown, the binocular image encoding device includes at least a first acquisition module 32, a determination module 34, a second acquisition module 36, and an encoding module 38, wherein:

[0095] The first acquisition module 32 is used to acquire the first environmental parameters of the first area where the first binocular camera is located within the current time period;

[0096] The determination module 34 is used to determine the first acquisition parameters of the first binocular camera in the current time period based on the first environmental parameters;

[0097] The second acquisition module 36 is used to acquire the binocular image of the first binocular camera acquired in the first region according to the first acquisition parameters;

[0098] The encoding module 38 is used to extract the image texture features of the stereo image and determine the image encoding strategy of the stereo image based on the image texture features, so as to encode the stereo image according to the image encoding strategy.

[0099] It should be noted that each module in the above-mentioned binocular image encoding device can be a program module (for example, a set of program instructions to implement a certain function) or a hardware module. For the latter, it can be manifested in the following forms, but is not limited to them: each of the above modules is manifested as a processor, or the functions of each of the above modules are implemented by a processor.

[0100] Example 3

[0101] According to an embodiment of this application, a non-volatile storage medium is also provided, which stores a program, wherein when the program runs, it controls the device where the non-volatile storage medium is located to execute the binocular image encoding method in Embodiment 1.

[0102] Optionally, the device containing the non-volatile storage medium executes the following steps by running the program: obtaining the first environmental parameters of the first area where the first binocular camera is located within the current time period; determining the first acquisition parameters of the first binocular camera within the current time period based on the first environmental parameters; obtaining the binocular image of the first area acquired by the first binocular camera according to the first acquisition parameters; extracting the image texture features of the binocular image, and determining the image encoding strategy of the binocular image based on the image texture features, so as to encode the binocular image according to the image encoding strategy.

[0103] According to an embodiment of this application, a computer program product is also provided, which includes a stored computer program, wherein the computer program implements the binocular image encoding method in Embodiment 1 when executed by a processor.

[0104] Optionally, the computer program performs the following steps: obtaining first environmental parameters of the first area where the first binocular camera is located within the current time period; determining first acquisition parameters of the first binocular camera within the current time period based on the first environmental parameters; obtaining binocular images of the first area acquired by the first binocular camera according to the first acquisition parameters; extracting image texture features of the binocular images, and determining an image encoding strategy for the binocular images based on the image texture features, so as to encode the binocular images according to the image encoding strategy.

[0105] According to an embodiment of this application, a processor is also provided for running a program, wherein the program executes the binocular image encoding method in embodiment 1 during runtime.

[0106] Optionally, the program executes the following steps during runtime: obtaining the first environmental parameters of the first area where the first binocular camera is located within the current time period; determining the first acquisition parameters of the first binocular camera within the current time period based on the first environmental parameters; obtaining the binocular image of the first area acquired by the first binocular camera according to the first acquisition parameters; extracting the image texture features of the binocular image, and determining the image encoding strategy of the binocular image based on the image texture features, so as to encode the binocular image according to the image encoding strategy.

[0107] According to an embodiment of this application, an electronic device is also provided, wherein, Figure 4 This is a schematic diagram of the structure of an optional electronic device according to an embodiment of this application, such as... Figure 4 As shown, the electronic device includes one or more processors; a memory for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to run the programs, wherein the programs are configured to execute the binocular image encoding method in Embodiment 1 above during runtime.

[0108] Optionally, the processor is configured to execute the following steps via a computer program: acquiring first environmental parameters of the first area where the first binocular camera is located within the current time period; determining first acquisition parameters of the first binocular camera within the current time period based on the first environmental parameters; acquiring binocular images of the first area acquired by the first binocular camera according to the first acquisition parameters; extracting image texture features of the binocular images, and determining an image encoding strategy for the binocular images based on the image texture features, so as to encode the binocular images according to the image encoding strategy.

[0109] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0110] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0111] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between units or modules may be electrical or other forms.

[0112] The units described as separate components may or may not be physically separate. Similarly, the components shown as units may or may not be physical units; they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0113] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0114] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to related technologies, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0115] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A binocular image encoding method, characterized in that, include: Obtain the first environmental parameters of the first area where the first binocular camera is located within the current time period; The first acquisition parameters of the first binocular camera in the current time period are determined based on the first environmental parameters; Acquire binocular images of the first region using the first binocular camera according to the first acquisition parameters; Extract the image texture features of the stereo image, and determine the image encoding strategy of the stereo image based on the image texture features, so as to encode the stereo image according to the image encoding strategy; The determination of the first acquisition parameters of the first binocular camera in the current time period based on the first environmental parameters includes: analyzing the first environmental parameters using a pre-trained machine learning model to obtain the first acquisition parameters of the first binocular camera in the current time period. The machine learning model is obtained by iteratively training a preset initial learning model using a training sample set and a sample label set. The training sample set includes multiple training samples, and each training sample represents the second environmental parameters of the second region where the second binocular camera is located within a historical time period. The second environmental parameters include at least one of the following: light intensity, color temperature, distance between the second binocular camera and the second region, motion state of objects within the second region, and background complexity of the second region. The sample label set includes sample labels corresponding to each training sample, and the sample labels are the second acquisition parameters optimized from the initial acquisition parameters of the second binocular camera within a historical time period. The second acquisition parameters include at least one of the following: exposure time, ISO, white balance, aperture size, frame rate, and exposure. The method of determining the image encoding strategy for the stereo image based on the image texture features includes: determining the texture complexity of the 3D image of the stereo image based on the image texture features, and determining the relationship between the texture complexity and a preset complexity threshold; if the texture complexity is not higher than the preset complexity threshold, determining that the image encoding strategy is to directly encode the 3D image of the stereo image using inter-frame coding prediction technology; if the texture complexity is higher than the preset complexity threshold, dividing the 3D image of the stereo image into multiple 3D sub-images, and determining that the image encoding strategy is to encode each of the 3D sub-images using intra-frame coding prediction technology.

2. The method according to claim 1, characterized in that, The binocular images include: a left-eye image and a right-eye image, wherein extracting the image texture features of the binocular images includes: The binocular image is preprocessed, wherein the preprocessing includes at least one of the following: distortion correction, epipolar correction, geometric transformation, image enhancement, and wavelet thresholding denoising; Multiple feature points in the binocular image are detected using a preset feature point detection algorithm, wherein the feature point detection algorithm includes at least one of the following: scale-invariant feature transformation matching algorithm, directional fast feature and rotation binary improved feature algorithm, and accelerated robust feature algorithm; For each feature point, determine the feature vector of that feature point; The image texture features of the stereo image are determined based on the feature vectors of multiple feature points within the stereo image.

3. The method according to claim 2, characterized in that, Determining the feature vector of the feature point includes: Determine a local image centered on the feature point and with a preset distance as the radius; The image features of the local image are extracted, and the resulting local image feature vector is used as the feature vector of the feature point.

4. The method according to claim 2, characterized in that, The plurality of feature points include: a plurality of first feature points in the left-eye image and a plurality of second feature points in the right-eye image, wherein determining the image texture features of the binocular image based on the feature vectors of the plurality of feature points in the binocular image includes: Determine the similarity between the feature vectors of the plurality of first feature points and the feature vectors of the plurality of second feature points; When the similarity between the feature vector of the first target feature point within the plurality of first feature points and the feature vector of the second target feature point within the plurality of second feature points is greater than a preset similarity threshold, a system of linear equations is constructed using the first target feature points and the second target feature points, and the essential matrix is ​​estimated by solving the system of linear equations, wherein the essential matrix is ​​used to describe the geometric relationship between the two cameras of the first binocular camera. Based on the essential matrix, the first target feature point, and the second target feature point, the three-dimensional image of the binocular image is reconstructed using triangulation, and the image texture features of the three-dimensional image are extracted using a preset feature extraction algorithm.

5. A binocular image encoding device, characterized in that, include: The first acquisition module is used to acquire the first environmental parameters of the first area where the first binocular camera is located within the current time period. The determination module is used to determine the first acquisition parameters of the first binocular camera in the current time period based on the first environmental parameters; The second acquisition module is used to acquire the binocular image of the first region captured by the first binocular camera according to the first acquisition parameters; An encoding module is used to extract image texture features of the stereo image and determine an image encoding strategy for the stereo image based on the image texture features, so as to encode the stereo image according to the image encoding strategy; The determination of the first acquisition parameters of the first binocular camera in the current time period based on the first environmental parameters includes: analyzing the first environmental parameters using a pre-trained machine learning model to obtain the first acquisition parameters of the first binocular camera in the current time period. The machine learning model is obtained by iteratively training a preset initial learning model using a training sample set and a sample label set. The training sample set includes multiple training samples, and each training sample represents the second environmental parameters of the second region where the second binocular camera is located within a historical time period. The second environmental parameters include at least one of the following: light intensity, color temperature, distance between the second binocular camera and the second region, motion state of objects within the second region, and background complexity of the second region. The sample label set includes sample labels corresponding to each training sample, and the sample labels are the second acquisition parameters optimized from the initial acquisition parameters of the second binocular camera within a historical time period. The second acquisition parameters include at least one of the following: exposure time, ISO, white balance, aperture size, frame rate, and exposure. The method of determining the image encoding strategy for the stereo image based on the image texture features includes: determining the texture complexity of the 3D image of the stereo image based on the image texture features, and determining the relationship between the texture complexity and a preset complexity threshold; if the texture complexity is not higher than the preset complexity threshold, determining that the image encoding strategy is to directly encode the 3D image of the stereo image using inter-frame coding prediction technology; if the texture complexity is higher than the preset complexity threshold, dividing the 3D image of the stereo image into multiple 3D sub-images, and determining that the image encoding strategy is to encode each of the 3D sub-images using intra-frame coding prediction technology.

6. A non-volatile storage medium, characterized in that, The non-volatile storage medium stores a computer program, wherein the device containing the non-volatile storage medium executes the binocular image encoding method according to any one of claims 1 to 4 by running the computer program.

7. A computer program product, characterized in that, include: A computer program, wherein when executed by a processor, the computer program implements the binocular image encoding method according to any one of claims 1 to 4.

Images