Binocular image coding method and device, storage medium and computer program product

Abstract

The invention discloses a binocular image coding method and device, a storage medium and a computer program product. The method comprises the following steps: acquiring a first environment parameter of a first area where a first binocular camera is located in a current time period; determining a first acquisition parameter of the first binocular camera in the current time period according to the first environment parameter; acquiring a binocular image obtained by acquiring the first area by the first binocular camera according to the first acquisition parameter; and extracting image texture features of the binocular image, and determining an image coding strategy of the binocular image based on the image texture features so as to perform coding processing on the binocular image according to the image coding strategy. The technical problems that the quality of the collected image is poor due to the fact that the collection parameters of the binocular camera are set only according to human experience in the prior art, and the coding efficiency is low and the integrity of the coded image is poor due to the fact that the collected image is coded according to the fixed coding parameters are solved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a binocular image encoding method and device, a storage medium, and a computer program product. Background Art

[0002] A binocular camera is a camera system that mimics the principle of human binocular vision. It consists of two cameras placed side by side or separated by a certain baseline distance. The two cameras shoot the same scene at the same time, but due to the difference in viewing angles, the captured images will have parallax. Using this parallax information and combining it with the principle of triangulation, the binocular camera system can calculate the depth information of the object being photographed, thereby generating image data with three-dimensional spatial perception, achieving more accurate and reliable target detection and tracking.

[0003] However, binocular cameras also face some challenges in practical applications. For example, in the video acquisition stage, the camera often lacks effective adaptive adjustment capabilities when facing environmental factors such as lighting changes, color temperature differences, and noise interference, resulting in problems such as overexposure, color distortion, or excessive noise in the captured video quality. In the video transmission stage, traditional video encoding algorithms often use unified encoding parameters, which cannot adapt to the texture characteristics of different objects in the scene, which may lead to low encoding efficiency in some areas and fail to balance video quality and bandwidth usage.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present application provide a binocular image encoding method and device, a storage medium, and a computer program product to at least solve the technical problem that the related art sets the acquisition parameters of the binocular camera only according to human experience, resulting in poor quality of the acquired image, and encodes the acquired image according to fixed encoding parameters, resulting in low encoding efficiency and poor integrity of the encoded image.

[0006] According to one aspect of an embodiment of the present application, a binocular image encoding method is provided, including: obtaining a first environmental parameter of a first area where a first binocular camera is located within a current time period; determining a first acquisition parameter of the first binocular camera within the current time period based on the first environmental parameter; obtaining a binocular image of the first area acquired by the first binocular camera according to the first acquisition parameter; extracting image texture features of the binocular image, and determining an image encoding strategy for the binocular image based on the image texture features, so as to encode the binocular image according to the image encoding strategy.

[0007] Optionally, determining the first acquisition parameter of the first binocular camera in the current time period based on the first environmental parameter includes: analyzing the first environmental parameter using a pre-trained machine learning model to obtain the first acquisition parameter of the first binocular camera in the current time period; wherein the machine learning model is obtained by iteratively training a preset initial learning model using a training sample set and a sample label set, wherein the training sample set includes multiple training samples, and each training sample is a second environmental parameter of a second area where the second binocular camera is located in a historical time period, and the sample label set includes a sample label corresponding to each training sample, and the sample label is a second acquisition parameter after optimizing the initial acquisition parameter of the second binocular camera in the historical time period.

[0008] Optionally, the second environmental parameters include at least one of the following: light intensity, color temperature, the distance between the second binocular camera and the second area, the motion state of objects in the second area, and the background complexity of the second area; the second acquisition parameters include at least one of the following: exposure time, sensitivity, white balance, aperture size, frame rate, and exposure.

[0009] Optionally, the binocular image includes: a left-eye image and a right-eye image, wherein extracting image texture features of the binocular image includes: preprocessing the binocular image, wherein the preprocessing includes at least one of the following: de-distortion, epipolar correction, geometric change, image enhancement, and wavelet threshold denoising; using a preset feature point detection algorithm to detect multiple feature points in the binocular image, wherein the feature point detection algorithm includes at least one of the following: a scale-invariant feature transformation matching algorithm, a directional fast feature and a rotational binary improved feature algorithm, and an accelerated robust feature algorithm; for each feature point, determining a feature vector of the feature point; and determining the image texture features of the binocular image based on the feature vectors of the multiple feature points in the binocular image.

[0010] Optionally, determining a feature vector of a feature point includes: determining a local image with the feature point as the center and a preset distance as the radius; extracting image features of the local image, and using the obtained local image feature vector as the feature vector of the feature point.

[0011] Optionally, the multiple feature points include: multiple first feature points in the left eye image and multiple second feature points in the right eye image, wherein the image texture features of the binocular image are determined based on the feature vectors of the multiple feature points in the binocular image, including: determining the similarity between the feature vectors of the multiple first feature points and the feature vectors of the multiple second feature points; when the similarity between the feature vectors of the first target feature points in the multiple first feature points and the feature vectors of the second target feature points in the multiple second feature points is greater than a preset similarity threshold, constructing a linear equation group using the first target feature points and the second target feature points, and estimating the intrinsic matrix by solving the linear equation group, wherein the intrinsic matrix is ​​used to describe the geometric relationship between the two cameras of the first binocular camera; based on the intrinsic matrix, the first target feature points and the second target feature points, reconstructing the three-dimensional image of the binocular image by triangulation, and extracting the image texture features of the three-dimensional image by using a preset feature extraction algorithm.

[0012] Optionally, determining an image coding strategy for a binocular image based on image texture features includes: determining a texture complexity of a three-dimensional image of the binocular image based on the image texture features, and judging a relationship between the texture complexity and a preset complexity threshold; when the texture complexity is not higher than a preset complexity threshold, determining an image coding strategy to directly encode the three-dimensional image of the binocular image using an inter-frame coding prediction technique; when the texture complexity is higher than a preset complexity threshold, dividing the three-dimensional image of the binocular image into a plurality of three-dimensional sub-images, and determining an image coding strategy to encode each three-dimensional sub-image using an intra-frame coding prediction technique.

[0013] According to another aspect of an embodiment of the present application, a binocular image encoding device is also provided, including: a first acquisition module, used to obtain a first environmental parameter of a first area where a first binocular camera is located within a current time period; a determination module, used to determine a first acquisition parameter of the first binocular camera within the current time period based on the first environmental parameter; a second acquisition module, used to obtain a binocular image obtained by the first binocular camera acquiring the first area according to the first acquisition parameter; an encoding module, used to extract image texture features of the binocular image, and determine an image encoding strategy of the binocular image based on the image texture features, so as to encode the binocular image according to the image encoding strategy.

[0014] According to another aspect of an embodiment of the present application, a non-volatile storage medium is further provided, which includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the above-mentioned binocular image encoding method by running the computer program.

[0015] According to another aspect of an embodiment of the present application, a computer program product is further provided. The computer program product includes a stored computer program, wherein the computer program implements the above-mentioned binocular image encoding method when executed by a processor.

[0016] In an embodiment of the present application, a first environmental parameter of a first area where a first binocular camera is located in a current time period is obtained; a first acquisition parameter of the first binocular camera in the current time period is determined based on the first environmental parameter to ensure the stability of the quality of the acquired image; a high-quality binocular image of the first area acquired by the first binocular camera according to the first acquisition parameter is obtained; image texture features of the binocular image are extracted, and an image coding strategy of the binocular image is determined based on the image texture features, so that the binocular image is encoded according to the image coding strategy, so that not only a lower quantization parameter can be used to retain image details, but also the amount of encoded data can be effectively reduced, and the transmission and storage costs can be reduced. In particular, in scenarios where network resources are limited, the adoption of such an adaptive coding strategy can significantly improve the efficiency and performance of the video surveillance system, thereby solving the technical problem that the related technology only sets the acquisition parameters of the binocular camera based on human experience, resulting in poor quality of the acquired image, and encodes the acquired image according to fixed coding parameters, resulting in low coding efficiency and poor integrity of the encoded image. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0018] Figure 1 It is a hardware structure block diagram of an optional computer terminal (or mobile device) for implementing a binocular image encoding method according to an embodiment of the present application;

[0019] Figure 2 is a flow chart of an optional binocular image encoding method according to an embodiment of the present application;

[0020] Figure 3 is a schematic structural diagram of an optional binocular image encoding device according to an embodiment of the present application;

[0021] Figure 4 It is a schematic diagram of the structure of an optional electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0023] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0024] In addition, the relevant information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for display, data for analysis, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. For example, an interface is set up between this system and the relevant user or organization. Before obtaining relevant information, it is necessary to send an acquisition request to the aforementioned user or organization through the interface, and obtain relevant information after receiving the consent information fed back by the aforementioned user or organization.

[0025] Example 1

[0026] According to an embodiment of the present application, an embodiment of a binocular image encoding method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0027] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 FIG. 1 shows a hardware structure block diagram of a computer terminal (or mobile device) for implementing a binocular image encoding method. Figure 1As shown, the computer terminal 10 (or mobile device 10) may include one or more (102a, 102b, ..., 102n are used to illustrate) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0028] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10 (or mobile device). As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0029] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the binocular image encoding method in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, the binocular image encoding method of the above-mentioned application is realized. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0030] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0031] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 (or mobile device).

[0032] In the above operating environment, Figure 2 is a flow chart of an optional binocular image encoding method according to an embodiment of the present application, such as Figure 2 As shown, the method at least includes steps S202-S208, wherein:

[0033] Step S202: Acquire first environmental parameters of a first area where a first binocular camera is located in a current time period.

[0034] In the technical solution provided in step S202, the first environmental parameter refers to real-time data collected from the environment around the first binocular camera, including but not limited to: light intensity, color temperature, the distance between the first binocular camera and the first area, the motion state of objects in the first area, the background complexity of the first area, etc. These parameters can be detected and analyzed by the built-in sensor of the camera, image processing algorithm or external sensor. Specifically:

[0035] Light intensity is an indicator to measure the strength of ambient light, which can be detected by an ambient light sensor deployed on the first binocular camera.

[0036] Color temperature is used to indicate the color tendency of ambient light, which can be detected by an XYZ color sensor or other color sensors.

[0037] The distance between the first binocular camera and the first area, that is, the distance between the first binocular camera and the photographed object, can be determined by triangulation.

[0038] The motion state of the object in the first area includes the moving speed and direction of the object. The monocular camera can first continuously collect the image sequence in the first area, and identify the moving object by comparing the pixel differences between consecutive image frames. Common methods include frame difference and optical flow, among which frame difference compares the pixel changes between consecutive frames to identify the motion; and the optical flow method is realized by estimating the moving direction and speed of the pixel points, which is suitable for object motion detection under dynamic background; then, the moving object is separated from the image, among which common segmentation methods include threshold-based segmentation, region-based segmentation, edge-based segmentation, etc.; then, feature points such as corner points and edges are detected on the moving object, and matched in consecutive frames to estimate the moving speed and direction of the object, among which feature point matching algorithms such as SIFT (Scale Invariant Feature Transform), SURF (Speeded Up Robust Features), ORB (Directional Fast Features and Rotational Binary Improved Features), etc.; finally, the moving state of the object is classified according to the information such as the moving speed, direction and acceleration of the object. For example, the motion state can be divided into stationary, slow moving, fast moving, accelerated or decelerated moving, etc.

[0039] The background complexity of the first area can be evaluated by capturing an image of the first area and calculating the entropy of the image or by using a deep learning model.

[0040] Step S204: determining a first acquisition parameter of the first binocular camera in a current time period according to the first environmental parameter.

[0041] As an optional implementation, in the technical solution provided in the above step S204, the method may include: using a pre-trained machine learning model to analyze the first environmental parameters to obtain the first acquisition parameters of the first binocular camera in the current time period to achieve optimization of image quality.

[0042] The first acquisition parameters include but are not limited to: exposure time, sensitivity, white balance, aperture size, frame rate, and exposure.

[0043] Specifically, light intensity will directly affect the camera's adjustment of exposure time and ISO (International Organization for Standardization value, sensitivity) value to ensure that the image maintains good clarity and brightness under different lighting conditions; color temperature will usually directly affect the camera's white balance to make the image color closer to reality and avoid color cast problems; the distance between the first binocular camera and the first area will affect the camera's focal length to ensure that objects in the first area are clearly visible; the movement state of objects in the first area will usually directly affect the camera's acquisition frame rate to ensure the clarity of the object's motion details; the background complexity of the first area will usually affect the exposure of the first binocular camera to ensure the details of the subsequently acquired images.

[0044] Optionally, the above-mentioned machine learning model is obtained by iteratively training a preset initial learning model using a training sample set and a sample label set to identify the relationship between different environmental conditions and camera acquisition parameters.

[0045] Among them, the above-mentioned training sample set and sample label set are determined based on historical data, the training sample set includes multiple training samples, and each training sample is the second environmental parameter of the second area where the second binocular camera is located in the historical time period, the sample label set includes a sample label corresponding to each training sample, and the sample label is the second acquisition parameter after optimizing the initial acquisition parameter of the second binocular camera in the historical time period, that is, the sample label is based on the second environmental parameter and video content analysis of the second binocular camera in the historical time period, according to the knowledge gained from continuous experiments and experience accumulation in actual operation, and the second acquisition parameter is adjusted manually or automatically, aiming to provide the best or better image quality. Therefore, the second acquisition parameter is the optimal camera setting parameter under the second environmental parameter.

[0046] By training the machine learning model with the above training samples and sample labels, the model can learn the optimal mapping relationship between environmental parameters and camera acquisition parameters, so that it can automatically predict and dynamically adjust the camera parameters in a new environment to achieve the best video acquisition effect.

[0047] In step S206, the machine learning model can be loaded into the memory. For example, the raw data of the machine learning model can be loaded from the non-volatile memory into the volatile memory so that the processor runs the machine learning model. The raw data of the machine learning model refers to unprocessed data, which usually includes parameters and structural data of the machine learning model. The structural data can be a calculation relationship based on the parameters, such as the forward propagation calculation relationship between the intermediate layers and the neurons. Specifically, the structural data can include code related to the structure of the machine learning model, such as code for performing related calculations between the intermediate layers and the neurons. It is also possible to divide an area in the memory for loading the machine learning model, which may include a structural data storage area and a parameter storage area. The structural data storage area is used to store structure-related codes, and the parameters referenced by it can point to the address of specific parameters in the parameter storage area through a pointer. During the training process of the machine learning model, it may be necessary to update the parameters frequently, and the parameter values ​​in the parameter storage area can be updated.

[0048] Step S206, obtaining a binocular image of the first area acquired by the first binocular camera according to the first acquisition parameter.

[0049] In the technical solution provided in step S206, the first binocular camera is set according to the first acquisition parameters determined in step S204 to ensure that these parameters can adapt to the current environmental conditions, thereby improving the image quality and effect. After the adjustment is completed, the first binocular camera starts to synchronously acquire binocular images (including left-eye images and right-eye images) in the first area, wherein synchronization means that the two cameras of the first binocular camera capture the first area image at the same time or almost at the same time, ensuring that the acquired images have temporal consistency for subsequent stereo matching.

[0050] Step S208 , extracting image texture features of the binocular image, and determining an image coding strategy for the binocular image based on the image texture features, so as to perform coding processing on the binocular image according to the image coding strategy.

[0051] As an optional implementation, in the technical solution provided in the above step S208, the method may include:

[0052] Step S2081, pre-processing the binocular image.

[0053] Since the acquired binocular images may contain noise or distortion, they need to be preprocessed, including but not limited to: distortion removal, epipolar line rectification, geometric transformation, image enhancement, wavelet threshold denoising, etc., to improve the image quality. Specifically:

[0054] The de-distortion process is to eliminate image deformation caused by the physical properties of the lens or the imaging system, such as radial distortion (Barrel or Pincushion Distortion) and tangential distortion (Tangential Distortion). It usually involves using the distortion coefficients and internal parameters of the camera (such as focal length and principal point position) to correct the image so that straight lines appear more straight in the image and eliminate barrel or pincushion effects.

[0055] Epipolar correction is used to project the images from the left and right cameras onto a common image plane so that the corresponding feature points are on the same horizontal line.

[0056] Geometric changes include rotation, translation, scaling, and perspective transformation, which are used to adjust the viewpoint of an image or correct the image to suit specific application requirements. For example, an image can be rotated to the correct direction, a tilted image can be corrected to a front view, or the image can be resized to suit different display or processing requirements.

[0057] Image enhancement is to enhance the visual effect of an image or highlight certain features by adjusting the contrast, brightness, saturation, etc. Commonly used image enhancement techniques include histogram equalization, contrast stretching, sharpening filtering, etc., which are used to improve the readability and analysis performance of the image.

[0058] Wavelet threshold denoising is to convert the image into the wavelet domain, eliminate the noise part in the wavelet coefficients by setting the threshold, and then transform the processed coefficients back to the image domain to obtain the denoised image, which can better preserve the edge and texture information of the image.

[0059] Step S2082: Detect multiple feature points in the binocular image using a preset feature point detection algorithm.

[0060] Among them, the above-mentioned feature points refer to the ability to represent images or targets in the same or at least very similar invariant form in other similar images containing the same scene or target. That is, for the same object or scene, multiple pictures are collected from different angles. If the same places can be identified as the same, then these points or blocks are called feature points, such as corner points, edge points or other significant image structure points. For binocular images, the corresponding multiple feature points include: multiple first feature points in the left eye image and multiple second feature points in the right eye image. In addition, the above-mentioned feature point detection algorithms include but are not limited to: scale-invariant feature transform matching algorithm SIFT, directional fast feature and rotation binary improved feature algorithm ORB, accelerated robust feature algorithm SURF.

[0061] Step S2083: for each feature point, determine the feature vector of the feature point.

[0062] Optionally, the technical solution provided in the above step S2083 can be implemented through the following steps: determine a local image with the feature point as the center and a preset distance as the radius; extract image features of the local image, and use the obtained local image feature vector as the feature vector of the feature point (also called the descriptor of the feature point).

[0063] It can be understood as follows: for each detected feature point, first, a circular or square area with the feature point as the center and a preset distance as the radius is determined, wherein the selection of the preset distance needs to consider the scale of the feature point and the resolution of the image to ensure that sufficient image details can be captured while avoiding the area being too large and the calculation being too complicated. Then, within the defined local image area, an image feature extraction algorithm is applied to extract features from the local area, wherein the image feature extraction algorithm includes but is not limited to: gray level co-occurrence matrix (GLCM), local binary pattern (LBP), histogram of oriented gradients (HOG), scale-invariant feature transform (SIFT) descriptor or other feature description algorithms based on local areas. Finally, the features extracted from the local area are combined into a feature vector, which contains multi-dimensional data describing the image characteristics around the feature point. It should be noted that the dimension and content of the feature vector depend on the selected feature extraction algorithm. For example, the SIFT descriptor is usually a 128-dimensional feature vector that contains the scale-invariant features of the local image.

[0064] Step S2084: determining the image texture features of the binocular image based on the feature vectors of the multiple feature points in the binocular image.

[0065] Optionally, the technical solution provided in the above step S2084 can be implemented by the following steps:

[0066] Step 1: Determine the similarity between the feature vectors of the first feature points and the feature vectors of the second feature points.

[0067] The similarity can be calculated by Euclidean distance, Hamming distance or cosine similarity. The goal of similarity calculation is to find the best match between two feature points, that is, to find a pair of feature points that describe the same physical point in the left and right images.

[0068] Step 2: When the similarity between the feature vector of the first target feature point among the multiple first feature points and the feature vector of the second target feature point among the multiple second feature points is greater than a preset similarity threshold, a linear equation group is constructed using the first target feature points and the second target feature points, and the essential matrix is ​​estimated by solving the linear equation group.

[0069] Specifically, when the similarity between the feature vector of the first target feature point in the plurality of first feature points and the feature vector of the second target feature point in the plurality of second feature points is greater than a preset similarity threshold, it can be considered that the pair of feature points can be considered as projections of the same physical point under different viewing angles. Subsequently, the essential matrix is ​​estimated using the eight-point method or the five-point method using the matched feature point pairs, wherein the essential matrix is ​​used to describe the geometric relationship between the two cameras of the first binocular camera.

[0070] Taking the eight-point method to estimate the essential matrix as an example, firstly, 8 pairs of matching feature points are given, and for each pair of matching points (x 1 ,y 1 ) and (x 2 ,y 2 ), construct the following equation:

[0071] x 1 x 2 e 11 +x 1 y 2 e 12 +x 1 e 13 +y 1 x 2 e 21 +y 1 y 2 e 22 +y 1 e 23 +x 2 e 31 +y 2 e 32 +e 33 =0

[0072] The above equation can also be written in the following matrix form:

[0073] Qe=0

[0074] Among them, Q is an 8*9 matrix, and e is the column vector of the essential matrix E of 9 unknown numbers. The least squares solution of Q is solved by singular decomposition, that is, Q is decomposed singularly, and Q = U∑V T , and take the last column of V as the least squares solution of E. Since the singular values ​​of the essential matrix E should be in the form of σ,σ,0, it is necessary to constrain the solved E to ensure that its singular values ​​meet this condition.

[0075] Step 3: Based on the essential matrix, the first target feature point and the second target feature point, the three-dimensional image of the binocular image is reconstructed by triangulation, and the image texture features of the three-dimensional image are extracted by using a preset feature extraction algorithm.

[0076] Specifically, since the essential matrix is ​​calculated based on the relative rotation and translation between the two cameras of the first binocular camera, and the intrinsic parameters of the camera (such as focal length and principal point position), that is, it contains the linear constraints that all matching feature point pairs between the two cameras must satisfy. Therefore, a series of linear equations are set by the essential matrix and the matching feature point pairs to solve the relative rotation and translation between the two cameras of the first binocular camera; then, triangulation is performed using these solved parameters to calculate the three-dimensional coordinates of each matching feature point pair in the first area. Among them, the basic principle of triangulation is that, assuming that the relative position and direction of the two cameras are known (obtained by the essential matrix and the intrinsic parameters of the camera), as well as the pixel coordinates of the feature points in the left and right images, the actual position of the feature points in the three-dimensional space can be calculated through geometric relationships (such as similar triangles or least squares method). Then, these feature points in the three-dimensional space are geometrically transformed to form a three-dimensional point cloud or mesh model, and then a three-dimensional visualization of the entire scene can be generated.

[0077] Furthermore, after the image texture features of the three-dimensional image reconstructed using the binocular image are known, the image encoding strategy of the binocular image can be determined according to the following method, including:

[0078] First, the texture complexity of the three-dimensional image of the binocular image is determined based on the image texture features, and the relationship between the texture complexity and the preset complexity threshold is determined. The texture complexity can be evaluated by analyzing the variance, edge direction or other texture features of the pixels in the coding block; and the relationship between the texture complexity of the coding block and the preset complexity threshold is determined.

[0079] When the texture complexity is not higher than the preset complexity threshold, the current size of the 3D image can be used as the encoding size because of its low texture complexity. Therefore, the image encoding strategy is to directly encode the 3D image of the binocular image using the inter-frame coding prediction technology.

[0080] When the texture complexity is higher than the preset complexity threshold, the 3D image of the binocular image can be divided into multiple 3D sub-images due to its high texture complexity, so as to capture details with smaller images. Therefore, the image coding strategy is to encode each 3D sub-image using intra-frame coding prediction technology.

[0081] Among them, the inter-frame coding prediction technology utilizes the spatiotemporal redundancy between frames in the video sequence, and uses the pixels of the adjacent encoded images to predict the pixels of the current image to achieve the purpose of removing redundant information in the time and frequency domains. Since the video sequence usually includes strong temporal correlation, its prediction residual is usually close to "0". The calculated residual signal is subsequently transformed, quantized and entropy encoded to achieve efficient compression of the video. Its specific implementation process includes the following steps:

[0082] S1, select one or more previously encoded frames (i.e., images) as reference frames, where the reference frames are selected based on their content similarity and prediction performance with the current frame, and usually the most recent frames are selected from the decoding buffer as candidates;

[0083] S2, for each coding block in the current frame (divided according to the complexity of texture features of different regions in the three-dimensional image of the binocular image, multiple coding blocks are obtained), search the area in the reference frame that best matches it. Specifically, this search is usually performed within the search window of the reference frame, and the best matching position is determined by comparing the difference in pixel values ​​between blocks (such as using mean square error MSE, absolute difference and SAD as matching metrics), where the difference between the best matching position and the current coding block position is the motion vector, which represents the displacement of the current block in time;

[0084] S3, according to the motion vector, the matching block in the reference frame is displaced at the pixel level to obtain the predicted frame. This process is called motion compensation, which can compensate for the pixel changes caused by the movement of the object and reduce the temporal redundancy;

[0085] S4, performing pixel-level difference calculation between the block in the current frame and the corresponding block in the predicted frame after motion compensation to obtain a prediction residual, wherein the prediction residual represents the difference information between the current frame and the predicted frame, and needs to be further encoded and transmitted.

[0086] S5, transform (such as DCT, wavelet transform, etc.), quantize and entropy encode the residual value to generate a bit stream. Among them, the transformation can convert the residual in the spatial domain to the frequency domain, making it easier to remove redundancy; quantization reduces the bit representation, thereby reducing the amount of transmitted data; entropy coding (such as Huffman coding, arithmetic coding, etc.) will further compress the data and improve the coding efficiency.

[0087] Intra-frame coding prediction technology uses the pixel correlation within a single frame in video coding to reduce spatial redundancy and improve coding efficiency. Its specific implementation process includes the following steps:

[0088] S1, for each coding block in the current frame (divided according to the complexity of texture features in different regions of the three-dimensional image of the binocular image, multiple coding blocks are obtained), the pixel values ​​of the encoded adjacent blocks are used to predict the pixel values ​​of the current block. The prediction method can be based on the left adjacent block, the upper adjacent block, or a combination of the two;

[0089] S2, determining a mode with the smallest prediction error as a prediction mode of the current block, wherein the prediction modes include but are not limited to: horizontal prediction, vertical prediction, DC prediction (average prediction), corner prediction and diagonal prediction, etc., and the selection of the prediction mode is based on the similarity between the current block and these neighboring blocks;

[0090] S3, calculate the residual between the current coding block and the predicted block, that is, the difference in pixel values. Similar to inter-frame prediction, this residual value needs to be further encoded and transmitted, but in intra-frame coding prediction, this residual value is relative to the adjacent pixel values ​​in the same frame;

[0091] S4, transforming (such as discrete cosine transform DCT, integer transform), quantizing and entropy encoding the prediction residual to generate a bit stream.

[0092] Based on the scheme defined by the above steps S202 to S208, it can be known that in an embodiment, by real-time monitoring of the first environmental parameters of the first area where the first binocular camera is located in the current time period, the first acquisition parameters of the first binocular camera are intelligently adjusted to adapt to environmental changes and ensure the stability of video quality and optimization of visual effects. Subsequently, a high-quality binocular image of the area is obtained according to the optimized acquisition parameters. Furthermore, by extracting the image texture features of the binocular image and determining the image coding strategy based on these features to adjust the coding parameters according to different areas of the image, while ensuring the video quality, the amount of encoded data is effectively reduced, and the transmission and storage costs are reduced. In particular, in scenarios where network resources are limited, the adoption of this adaptive coding strategy can significantly improve the efficiency and performance of the video surveillance system. In summary, the embodiment of the present application achieves the technical effects of high-quality video acquisition and efficient video coding in a dynamic environment through real-time monitoring and adaptive adjustment of environmental parameters, combined with image texture feature analysis and coding strategy optimization, and meets the requirements of video surveillance and remote vision applications for high precision and low latency.

[0093] Example 2

[0094] Based on Embodiment 1 of the present application, an embodiment of a binocular image encoding device is also provided, and the binocular image encoding method of the above embodiment is executed when the device is running. Figure 3 is a schematic structural diagram of an optional binocular image encoding device according to an embodiment of the present application, such as Figure 3 As shown, the binocular image encoding device at least includes a first acquisition module 32, a determination module 34, a second acquisition module 36 and an encoding module 38, wherein:

[0095] A first acquisition module 32, used to acquire a first environmental parameter of a first area where a first binocular camera is located in a current time period;

[0096] A determination module 34, configured to determine a first acquisition parameter of the first binocular camera in a current time period according to the first environmental parameter;

[0097] A second acquisition module 36 is used to acquire a binocular image of the first area acquired by the first binocular camera according to the first acquisition parameter;

[0098] The encoding module 38 is used to extract image texture features of the binocular image and determine an image encoding strategy for the binocular image based on the image texture features, so as to perform encoding processing on the binocular image according to the image encoding strategy.

[0099] It should be noted that the various modules in the above-mentioned binocular image encoding device can be program modules (for example, a set of program instructions that implement a certain specific function) or hardware modules. For the latter, it can be expressed in the following forms, but is not limited to this: the expression form of each of the above-mentioned modules is a processor, or the functions of each of the above-mentioned modules are implemented by a processor.

[0100] Example 3

[0101] According to an embodiment of the present application, a non-volatile storage medium is further provided, in which a program is stored, wherein when the program is running, the device where the non-volatile storage medium is located is controlled to execute the binocular image encoding method in Example 1.

[0102] Optionally, the device where the non-volatile storage medium is located implements the following steps by running the program: obtaining a first environmental parameter of a first area where a first binocular camera is located within a current time period; determining a first acquisition parameter of the first binocular camera within the current time period based on the first environmental parameter; obtaining a binocular image of the first area acquired by the first binocular camera according to the first acquisition parameter; extracting image texture features of the binocular image, and determining an image coding strategy of the binocular image based on the image texture features, so as to encode the binocular image according to the image coding strategy.

[0103] According to an embodiment of the present application, a computer program product is further provided. The computer program product includes a stored computer program, wherein when the computer program is executed by a processor, the binocular image encoding method in Embodiment 1 is implemented.

[0104] Optionally, the computer program executes the following steps: obtaining a first environmental parameter of a first area where a first binocular camera is located within a current time period; determining a first acquisition parameter of the first binocular camera within the current time period based on the first environmental parameter; obtaining a binocular image of the first area acquired by the first binocular camera according to the first acquisition parameter; extracting image texture features of the binocular image, and determining an image coding strategy of the binocular image based on the image texture features, so as to encode the binocular image according to the image coding strategy.

[0105] According to an embodiment of the present application, a processor is also provided, which is used to run a program, wherein the binocular image encoding method in Example 1 is executed when the program is run.

[0106] Optionally, when the program is running, the following steps are executed: obtaining a first environmental parameter of a first area where a first binocular camera is located within a current time period; determining a first acquisition parameter of the first binocular camera within the current time period based on the first environmental parameter; obtaining a binocular image of the first area acquired by the first binocular camera according to the first acquisition parameter; extracting image texture features of the binocular image, and determining an image coding strategy for the binocular image based on the image texture features, so as to encode the binocular image according to the image coding strategy.

[0107] According to an embodiment of the present application, an electronic device is also provided, wherein: Figure 4 is a schematic diagram of the structure of an optional electronic device according to an embodiment of the present application, such as Figure 4 As shown, the electronic device includes one or more processors; a memory for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors are implemented to run the programs, wherein the programs are configured to execute the binocular image encoding method in the above-mentioned embodiment 1 when running.

[0108] Optionally, the processor is configured to execute the following steps through a computer program: obtaining a first environmental parameter of a first area where a first binocular camera is located within a current time period; determining a first acquisition parameter of the first binocular camera within the current time period based on the first environmental parameter; obtaining a binocular image of the first area acquired by the first binocular camera according to the first acquisition parameter; extracting image texture features of the binocular image, and determining an image coding strategy of the binocular image based on the image texture features, so as to encode the binocular image according to the image coding strategy.

[0109] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0110] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0111] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0112] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed over multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0113] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0114] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the relevant technology or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk, etc. Various media that can store program codes.

[0115] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A binocular image encoding method, characterized in that: include: Obtaining first environmental parameters of a first area where a first binocular camera is located within a current time period; Determining a first acquisition parameter of the first binocular camera in a current time period according to the first environmental parameter; Acquire a binocular image of the first area captured by the first binocular camera according to the first acquisition parameter; The image texture features of the binocular image are extracted, and an image coding strategy of the binocular image is determined based on the image texture features, so as to perform coding processing on the binocular image according to the image coding strategy.

2. The method according to claim 1, characterized in that Determining a first acquisition parameter of the first binocular camera in a current time period according to the first environmental parameter includes: Analyze the first environmental parameter using a pre-trained machine learning model to obtain a first acquisition parameter of the first binocular camera in a current time period; In which, the machine learning model is obtained by iteratively training a preset initial learning model using a training sample set and a sample label set, wherein the training sample set includes multiple training samples, and each of the training samples is a second environmental parameter of a second area where a second binocular camera is located within a historical time period, and the sample label set includes a sample label corresponding to each training sample, and the sample label is a second acquisition parameter after optimizing the initial acquisition parameters of the second binocular camera within the historical time period.

3. The method according to claim 2, characterized in that The second environmental parameter includes at least one of the following: light intensity, color temperature, the distance between the second binocular camera and the second area, the motion state of objects in the second area, and the background complexity of the second area; The second acquisition parameter includes at least one of the following: exposure time, sensitivity, white balance, aperture size, frame rate, and exposure.

4. The method according to claim 1, characterized in that: The binocular image includes: a left-eye image and a right-eye image, wherein extracting image texture features of the binocular image includes: Preprocessing the binocular image, wherein the preprocessing includes at least one of the following: de-distortion, epipolar correction, geometric change, image enhancement, and wavelet threshold denoising; Detecting multiple feature points in the binocular image using a preset feature point detection algorithm, wherein the feature point detection algorithm includes at least one of the following: a scale-invariant feature transformation matching algorithm, a directional fast feature and rotation binary improved feature algorithm, and an accelerated robust feature algorithm; For each of the feature points, determining a feature vector of the feature point; The image texture features of the binocular image are determined based on feature vectors of a plurality of feature points in the binocular image.

5. The method according to claim 4, characterized in that Determining the feature vector of the feature point includes: Determine a local image with the feature point as the center and a preset distance as the radius; The image features of the local image are extracted, and the obtained local image feature vector is used as the feature vector of the feature point.

6. The method according to claim 4, characterized in that The plurality of feature points include: a plurality of first feature points in the left-eye image and a plurality of second feature points in the right-eye image, wherein determining the image texture feature of the binocular image based on the feature vectors of the plurality of feature points in the binocular image includes: Determining similarities between the feature vectors of the plurality of first feature points and the feature vectors of the plurality of second feature points; When the similarity between a feature vector of a first target feature point in the plurality of first feature points and a feature vector of a second target feature point in the plurality of second feature points is greater than a preset similarity threshold, a linear equation system is constructed using the first target feature points and the second target feature points, and an essential matrix is ​​estimated by solving the linear equation system, wherein the essential matrix is ​​used to describe a geometric relationship between two cameras of the first binocular camera; Based on the intrinsic matrix, the first target feature points and the second target feature points, a three-dimensional image of the binocular image is reconstructed by triangulation, and image texture features of the three-dimensional image are extracted by using a preset feature extraction algorithm.

7. The method according to claim 1, characterized in that Determining an image encoding strategy for the binocular image based on the image texture feature includes: Determining the texture complexity of the three-dimensional image of the binocular image according to the image texture feature, and judging the magnitude relationship between the texture complexity and a preset complexity threshold; When the texture complexity is not higher than a preset complexity threshold, determining that the image coding strategy is to directly encode the three-dimensional image of the binocular image using an inter-frame coding prediction technology; When the texture complexity is higher than a preset complexity threshold, the three-dimensional image of the binocular image is divided into a plurality of three-dimensional sub-images, and the image encoding strategy is determined to encode each of the three-dimensional sub-images using intra-frame coding prediction technology.

8. A binocular image encoding device, characterized in that: include: A first acquisition module, used to acquire a first environmental parameter of a first area where a first binocular camera is located within a current time period; A determination module, configured to determine a first acquisition parameter of the first binocular camera in a current time period according to the first environmental parameter; A second acquisition module is used to acquire a binocular image of the first area captured by the first binocular camera according to the first acquisition parameter; The encoding module is used to extract the image texture features of the binocular image and determine the image encoding strategy of the binocular image based on the image texture features, so as to encode the binocular image according to the image encoding strategy.

9. A non-volatile storage medium, characterized in that: The non-volatile storage medium stores a computer program, wherein the device where the non-volatile storage medium is located executes the binocular image encoding method according to any one of claims 1 to 7 by running the computer program.

10. A computer program product, characterized in that include: A computer program, wherein when the computer program is executed by a processor, the binocular image encoding method according to any one of claims 1 to 7 is implemented.

Images