Model training method, device and data processing equipment based on binocular images

Abstract

The present application provides a model training method, device and data processing equipment based on binocular images, and the image matching model includes a teacher model and a student model. In the method provided in the present application, two groups of sample images acquired at different time points by a binocular image acquisition device are first obtained. Then, for any two sample images in the two groups of sample images, the teacher model is used to perform optical flow estimation according to the preset geometric constraints between the two sample images to obtain a more accurate high-confidence optical flow estimation result, and the preset geometric constraints are geometric constraints based on binocular images. Finally, the high-confidence optical flow estimation result is used as annotation information, and the two sample images are used to perform machine learning training for image element matching on the student model. In this way, self-supervised training using unlabeled images can be achieved, and the model obtained by training has a high recognition accuracy.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a binocular image-based model training method, device and data processing equipment. Background Art

[0002] In the field of computer vision recognition, how to identify and match the same object in different images is a widely studied computer vision task, among which obtaining a Convolutional Neural Network (CNN) model that can accurately perform optical flow estimation or stereo matching is a hot research project.

[0003] In order to obtain an accurate image matching model, it is necessary to perform machine learning training on the image matching model. Usually, the training methods include supervised training methods and unsupervised training methods. Supervised training methods require a large number of labeled training image samples. If labeled real images are used as training samples, the training cost is very high. If simulated labeled images are used as training samples, the obtained model has poor accuracy in recognizing real images. Some unsupervised training methods use the optical flow estimation obtained by the teacher model as a label to guide the training of the student model. However, the optical flow estimation based on the teacher model is not accurate enough, which may greatly affect the recognition ability of the student model. Summary of the invention

[0004] In order to overcome at least one of the deficiencies in the prior art, one of the purposes of the present application is to provide a model training method based on binocular images, which is applied to train an image matching model, wherein the image matching model includes a teacher model and a student model, and the method includes:

[0005] Acquire two groups of sample images acquired at different time points by a binocular image acquisition device;

[0006] For any two sample images in the two groups of sample images, the teacher model is used to perform optical flow estimation according to a preset geometric constraint between the two sample images to obtain an optical flow estimation result, wherein the preset geometric constraint is a geometric constraint based on a binocular image;

[0007] The optical flow estimation result is used as annotation information, and the two sample images are used to perform machine learning training of image element matching on the student model. The image element matching process is to identify image elements belonging to the same object in the two sample images.

[0008] Another object of the present application is to provide a model training device based on binocular images, which is used to train an image matching model, wherein the image matching model includes a teacher model and a student model, and the device includes:

[0009] An image acquisition module, used to acquire two sets of sample images acquired at different time points by a binocular image acquisition device;

[0010] A first training module is used for performing optical flow estimation on any two sample images in the two groups of sample images according to a preset geometric constraint between the two sample images through the teacher model to obtain an optical flow estimation result, wherein the preset geometric constraint is a geometric constraint based on a binocular image;

[0011] The second training module is used to use the optical flow estimation result as annotation information and use the two sample images to perform machine learning training on image element matching on the student model. The image element matching process is to identify image elements belonging to the same object in the two sample images.

[0012] Another object of the present application is to provide a data processing device, including a machine-readable storage medium and a processor, wherein the machine-readable storage medium stores machine-executable instructions, and when the machine-executable instructions are executed by the processor, the model training method based on binocular images provided in this embodiment is implemented.

[0013] Compared with the prior art, this application has at least the following beneficial effects:

[0014] The model training method, device and data processing equipment based on binocular images provided in this application use binocular images as training samples and combine the inherent geometric constraints of binocular images to make the teacher model output high-confidence optical flow estimation results to guide the image matching learning of the student model. In this way, self-supervised training using unlabeled images can be achieved, and the model obtained by training has high recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0016] Figure 1 A schematic diagram of a data processing device provided in an embodiment of the present application;

[0017] Figure 2A schematic diagram of a flow chart of a binocular image-based model training method provided in an embodiment of the present application;

[0018] Figure 3 A schematic diagram of the correlation between binocular stereo matching and optical flow provided in an embodiment of the present application;

[0019] Figure 4 A schematic diagram of the model training principle based on binocular images provided in an embodiment of the present application;

[0020] Figure 5 A schematic diagram of obtaining an optical flow map provided in an embodiment of the present application;

[0021] Figure 6 A schematic diagram of the geometric constraints of the optical flow graph provided in the embodiment of the present application;

[0022] Figure 7 This is a schematic diagram of the optical flow estimation test results;

[0023] Figure 8 This is a schematic diagram of binocular stereo matching test results;

[0024] Fig. 9 A schematic diagram of a binocular image-based model training device provided in an embodiment of the present application.

[0025] Icons: 100 - data processing equipment; 110 - model training device based on binocular images; 111 - image acquisition module; 112 - first training module; 113 - second training module; 120 - machine-readable storage medium; 130 - processor. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0027] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0028] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0029] In some unsupervised training methods, an optical flow estimation model is used as a teacher model, and the annotation results obtained by performing optical flow estimation on the training samples are then used to guide another optical flow estimation model as a student model to perform optical flow estimation training. Among them, inaccurate optical flow estimation of the teacher model will directly lead to poor optical flow estimation accuracy of the trained student model.

[0030] Based on the discovery of the above problems, this embodiment provides a solution that uses binocular images as training samples and uses the fixed geometric constraints of binocular images to estimate optical flow, so that the teacher model can obtain more accurate optical flow estimation results, thereby effectively improving the image matching accuracy of the student model. The solution provided in this embodiment is elaborated in detail below.

[0031] Please refer to Figure 1 , Figure 1 This is the structural intention of the data processing device 100 provided in the embodiment of the present application. The data processing device 100 includes a model training device 110 based on binocular images, a machine-readable storage medium 120 and a processor 130.

[0032] The components of the machine-readable storage medium 120 and the processor 130 are electrically connected to each other directly or indirectly to realize data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines. The model training device 110 based on binocular images includes at least one software function module that can be stored in the machine-readable storage medium 120 in the form of software or firmware or solidified in the operating system (OS) of the data processing device 100. The processor 130 is used to execute the executable modules stored in the machine-readable storage medium 120, such as the software function modules and computer programs included in the model training device 110 based on binocular images.

[0033] The machine-readable storage medium 120 may be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), etc. The machine-readable storage medium 120 is used to store a program, and the processor 130 executes the program after receiving an execution instruction.

[0034] Please refer to Figure 2 , Figure 2 For application Figure 1 The data processing device 100 shown is a flowchart of a model training method based on binocular images. The method including various steps will be described in detail below.

[0035] Step S210, obtaining two groups of sample images acquired at different time points by a binocular image acquisition device.

[0036] Step S220, for any two sample images in the two groups of sample images, perform optical flow estimation according to a preset geometric constraint between the two sample images through the teacher model to obtain an optical flow estimation result, wherein the preset geometric constraint is a geometric constraint based on a binocular image.

[0037] The preset geometric constraint is to determine the geometric limit of the optical flow between sample images by using the 3D spatial geometric characteristics that the binocular images are images taken by left and right cameras at the same time and on the same horizontal line from different angles.

[0038] Step S230, using the optical flow estimation result as annotation information, and using the two sample images to perform machine learning training of image element matching on the student model, wherein the image element matching process is to identify image elements belonging to the same object in the two sample images.

[0039] Optical flow is a technology that determines the movement of the same object in different frames based on brightness, based on the assumption that the brightness of the same target in different images taken within a short period of time will not change and that the object will not undergo significant position changes within a short period of time.

[0040] Binocular stereo matching is a computer vision task to identify the same object from images taken at different angles at the same time.

[0041] The inventors have found that the two images in the binocular image can be regarded as two images obtained by moving the camera to another angle and taking the image after taking the image at one angle. Therefore, binocular image matching can be regarded as a special case of optical flow estimation. And for binocular images with corrected horizontal epipolar lines, there is an inherent geometric constraint relationship between the images. Therefore, in step S210, the images collected by the binocular image acquisition device are used as training samples, and the inherent geometric constraints of the binocular images can be used to enable the teacher model to obtain accurate optical flow estimates.

[0042] For details, please refer to Figure 3 , which shows the geometric relationship between optical flow and stereo disparity in 3D space. l and O r are the corrected center points of the left and right cameras in the binocular image acquisition device, B is the distance between the centers of the two cameras, P(X, Y, Z) is the point in the 3D space at time t, and P l and P r are the projection positions of point P on the images captured by the left and right cameras respectively.

[0043] Point P moves to P+ΔP at time t+1, where the displacement ΔP = (ΔX, ΔY, ΔZ). Optical flow w l and w r They represent the optical flow obtained in the left and right camera capture images before and after the point P moves, respectively, while the stereo disparity represents the displacement of the matching points between the two binocular images recorded simultaneously. Despite the different definitions, optical flow estimation and binocular stereo matching can be regarded as the same type of problem, that is, matching corresponding pixels.

[0044] In binocular stereo matching, matching pixels should be located on the epipolar lines between the binocular image pairs, while optical flow is not constrained by this structure. Therefore, in this embodiment, binocular stereo matching can be regarded as a special case of optical flow. In other words, the displacement between binocular images can be regarded as a one-dimensional "motion". For the corrected binocular images, the epipolar lines are horizontal, that is, binocular stereo matching becomes searching for matching pixels along the horizontal direction. Due to the inherent geometric constraints of binocular images, using binocular images for optical flow estimation can obtain more accurate optical flow estimation results.

[0045] In addition, since the occluded object does not conform to the assumption of constant luminosity in the optical flow estimation, it will greatly affect the accuracy of the output result of the teacher model. In order to enable the teacher model to obtain a more accurate optical flow estimation, in step S220 of this embodiment, the teacher model can perform optical flow estimation according to the preset geometric constraints and the confidence map determined by the unoccluded areas in the two sample images to obtain the optical flow estimation result after excluding the occluded area.

[0046] In this way, the teacher model combines the confidence map obtained based on the photometric difference to analyze the occluded area to obtain a confidence map. By combining the confidence map, a high-confidence optical flow map can be obtained, which can guide the student model to learn image matching more accurately.

[0047] For details, please refer to Figure 4 , Figure 4 The schematic diagram of the principle of the binocular image-based model training method provided in this embodiment. In the two groups of sample images obtained by step S210, each group of sample images may include two sample images. Figure 5 , assuming that at time t, the left and right cameras in the binocular image acquisition device capture images I1 and I2 respectively, and at time t+1, the left and right cameras in the binocular image acquisition device capture images I3 and I4 respectively.

[0048] In step S220, two sample images may be selected from the above four images, and initial optical flow maps of the two sample images may be obtained by calculation according to the preset geometric constraints. Figure 5 As shown, the four sample images obtained in step S210 can obtain 12 optical flow maps from each other. In this embodiment, image I i To I j The optical flow map is denoted as w i→j .

[0049] Then, the initial optical flow map can be subjected to forward-backward brightness detection, and pixels whose brightness difference exceeds a preset range are regarded as obscured pixels, and the confidence of the obscured pixels is set to 0; pixels whose brightness difference does not exceed the preset range are regarded as unobstructed pixels, and the confidence of the unobstructed pixels is set to 1. Since the confidence of the obscured pixels is set to 0 in the confidence map, the obscured pixels are excluded after the optical flow map is multiplied by the confidence map, so that the obtained optical flow map only includes the unobstructed areas with high confidence.

[0050] When performing forward-backward detection, the image I in the two samples can be obtained first. i To Image I j The forward optical flow w of pixel p on the initial optical flow map i→j (p), and obtain image I j To Image I i Backward optical flow in,

[0051] Then detect the forward optical flow w i→j (p) and backward optical flow Are the following conditions met: Among them, α=0.01 and β=0.5.

[0052] If it is satisfied, it means that the luminosity difference of the pixel p is within the preset range, that is, the pixel p is located in an unobstructed area, and the confidence of the pixel p is set to 1.

[0053] If it is not satisfied, it means that the luminosity difference of the pixel p exceeds the preset range, that is, the pixel p is located in the blocked area, and the confidence of the pixel p is set to 0.

[0054] After obtaining the confidence map, optical flow estimation can be performed on the two sample images according to the preset geometric constraints and the confidence map to obtain the optical flow estimation result.

[0055] In this embodiment, the preset geometric constraints include triangle constraints and quadrilateral constraints, which can be obtained by the photometric loss function L p , the quadrilateral loss function L determined by the quadrilateral constraint q , the triangle loss function L determined by the triangle constraint t and the confidence map to perform optical flow estimation on the two sample images.

[0056] Specifically, according to the inherent characteristics of binocular images, the four images obtained in step S210 have several fixed constraints. is a pixel in image I1, and are the pixels in images I2, I3, and I4 respectively. Figure 6 , taking image I1 as the benchmark, we can choose w 1→2 、w 3→4 represents stereo disparity, choose w 1→3 、w 2→4 Represents the optical flow at different time points, choose w 1→4 represents the cross-parallax optical flow. Then,

[0057]

[0058] Since the movement of an object from the position in image I1 to the position in image I4 is equivalent to moving from the position in image I1 to the position in image I2 and then from the position in image I2 to the position in image I4, then,

[0059]

[0060] Accordingly, according to the object moving from the position in image I1 to the position in image I3, and then from the position in image I3 to the position in image I4, we can get:

[0061]

[0062] According to the above two formulas, we can get:

[0063]

[0064] Since the matching pixels are on the same epipolar line when processing the binocular stereo matching task, and the epipolar line in the corrected binocular image is horizontal, combined with the above equation, we can get:

[0065]

[0066] Among them, u i→j For image I i To Image I j The horizontal optical flow, v i→j For image I i To Image I j The vertical optical flow.

[0067] For pixel p, the photometric loss function L p for:

[0068]

[0069] in, According to the image I in the two samples i To Image I j The optical flow w i→j Image I j Warp to Image I i The distorted image obtained, M i→j For image I i To Image I j Confidence map of ψ(x)=(|x|+s) q , s=0.01, q=0.4.

[0070] The quadrilateral constraint is used to define the geometric relationship between the optical flow and the stereo disparity. In this embodiment, the quadrilateral constraint is only used for pixels with high confidence. q =L qu +L qv , L qu is the quadrilateral loss function L q The component in the horizontal direction, L qv is the quadrilateral loss function L q Component in the vertical direction, where:

[0071]

[0072]

[0073] and are the pixels at the same position on images I1, I2, I3 and I4, I1 and I2 are binocular images collected at time t, I3 and I4 are binocular images collected at time t+1, M q =M 1→2 (p)⊙M 1→3 (p)⊙M 1→4 (p).

[0074] The triangle constraint is used to define the relationship between optical flow, stereo disparity and cross-view optical flow. Similar to the quadrilateral constraint loss, in this embodiment, the triangle constraint is only used for high-confidence pixels. The triangle loss function L t for:

[0075]

[0076] in, are the pixels at the same position in images I1 and I2, respectively, 1→4 is the optical flow from image I1 to image I4, w 2→4 is the optical flow from image I2 to image I4, w 1→2 is the optical flow from image I1 to image I2, I1 and I2 are binocular images collected at time t, and I3 and I4 are binocular images collected at time t+1.

[0077] After obtaining an optical flow estimation result with high confidence through step S220, the optical flow estimation result can be used as annotation information in step S230, and the student model can be trained using the two sample images in step S220.

[0078] The preset self-supervised loss function L is used during the training process of the student model. s For the loss student model, the proxy optical flow in the optical flow estimation result with high confidence obtained in step S220 can be recorded as and the agent confidence graph is denoted as Then there is,

[0079]

[0080] Among them, w i→j is the optical flow derived from the student model.

[0081] It should be noted that, in this embodiment, unlike the training of the teacher model, the occluded area and the unoccluded area are no longer distinguished in the self-supervised training of the student model, so that the student model can estimate the optical flow of the occluded area.

[0082] Using the method provided in this embodiment, during the training process, the teacher model is used to obtain the optical flow of some high-confidence pixels from the input sample image as labeling information, and the student model performs optical flow estimation training for all pixels on the image based on the labeling information obtained by the teacher model.

[0083] Therefore, in this embodiment, after the image matching model is trained, the student model can be used to perform optical flow estimation or binocular image matching. During use, two images to be processed can be obtained, and then the two images to be processed can be input into the trained student model to obtain the image matching results output by the student model for the two images to be processed.

[0084] When the trained student model is used for optical flow estimation, two images captured at different time points can be input into the student model, and the student model outputs an optical flow map between the two images. When the trained student model is used for binocular image matching, the images captured by the left and right cameras in the binocular image can be input into the student model, and the student model outputs a stereo disparity map of the two images.

[0085] Optionally, in order to improve the recognition ability of the student model, in this embodiment, the two sample images can be randomly cropped in the same manner, and the student model can be trained by machine learning for image element matching using the cropped two sample images. Furthermore, in this embodiment, when training the student model, the two sample images can also be randomly scaled and rotated in the same manner, so as to avoid overfitting in the training process.

[0086] In this embodiment, the image matching model can be constructed using the TensorFlow system with the Adam optimizer. For the teacher model, the batch parameter can be set to 1, because there are 12 optical flow estimates for 4 images. For the student model, the batch parameter can be set to 4, and some data enhancement strategies can be adopted. During training, images with a resolution of 320*896 can be set as input. During testing, the resolution of the image can be adjusted to 384*1280.

[0087] Figure 7 The test results of optical flow estimation on KITTI2012 and KITTI2015 datasets using some existing models and the image matching model trained in this embodiment are shown, where 'fg' and 'bg' represent the results of the foreground color area and the background color area, respectively. Figure 7 In “Ours+L p +L q +L t+Self-supervision" is the optical flow estimation test data of the image matching model trained in this embodiment. It can be seen that the recognition ability of the image matching model is significantly higher than Figure 7 Other models among them.

[0088] Figure 8 The test results of binocular stereo matching of some existing models and the image matching model trained by this embodiment on the KITTI2012 and KITTI2015 datasets are shown. Figure 8 In “Ours+L p +L q +L t +Self-supervision" is the binocular stereo matching test data of the image matching model trained in this embodiment. It can be seen that the recognition ability of the image matching model is significantly higher than Figure 7 Other models among them.

[0089] Please refer to Fig. 9 This embodiment also provides a model training device 110 based on binocular images, which may include an image acquisition module 111, a first training module 112 and a second training module 113.

[0090] The image acquisition module 111 is used to acquire two groups of sample images acquired at different time points by a binocular image acquisition device.

[0091] In this embodiment, the image acquisition module 111 can be used to perform Figure 2 As shown in step S210, for a detailed description of the image acquisition module 111, please refer to the description of step S210.

[0092] The first training module 112 is used to perform optical flow estimation on any two sample images in the two groups of sample images according to a preset geometric constraint between the two sample images through the teacher model to obtain an optical flow estimation result, wherein the preset geometric constraint is a geometric constraint based on a binocular image.

[0093] In this embodiment, the first training module 112 can be used to perform Figure 2 As shown in step S220, for a detailed description of the first training module 112, please refer to the description of step S220.

[0094] The second training module 113 is used to use the optical flow estimation result as annotation information, and use the two sample images to perform machine learning training on image element matching on the student model. The image element matching process is to identify image elements belonging to the same object in the two sample images.

[0095] In this embodiment, the second training module 113 can be used to perform Figure 2 As shown in step S230, for a detailed description of the second training module 113, please refer to the description of step S230.

[0096] In summary, the model training method, device and data processing equipment based on binocular images provided by this application use binocular images as training samples, and combine the inherent geometric constraints of binocular images to make the teacher model output high-confidence optical flow estimation results to guide the image matching learning of the student model. In this way, self-supervised training using unlabeled images can be achieved, and the model obtained by training has high recognition accuracy.

[0097] The above are only various implementations of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A model training method based on binocular images, characterized in that: Applied to training an image matching model, the image matching model includes a teacher model and a student model, and the method includes: Acquire two groups of sample images acquired at different time points by a binocular image acquisition device; For any two sample images in the two groups of sample images, the teacher model performs optical flow estimation according to a preset geometric constraint between the two sample images to obtain an optical flow estimation result, wherein the preset geometric constraint is a geometric constraint based on a binocular image; for the student model, the optical flow estimation result includes a proxy optical flow output by the teacher model and the agent confidence map The optical flow estimation result is used as the annotation information, and the two sample images are used to perform machine learning training of image element matching on the student model. The image element matching process is to identify the image elements belonging to the same object in the two sample images, including: Using the two sample images, according to the self-supervised loss function L s The student model is subjected to machine learning training for image element matching, wherein: p is the image I in the two samples i To Image I j The pixel point on W i→j is the optical flow derived from the student model, ψ(x)=(|x|+s) q , s=0.01, q=0.

4.

2. The method according to claim 1, characterized in that The method further comprises: Get two images to be processed; The two images to be processed are input into the trained student model to obtain image matching results output by the student model for the two images to be processed.

3. The method according to claim 1, characterized in that The performing optical flow estimation according to the preset geometric constraints between the two sample images by the teacher model includes: The teacher model performs optical flow estimation according to the preset geometric constraints and the confidence map determined by the unobstructed areas in the two sample images to obtain the optical flow estimation result after excluding the obstructed areas.

4. The method according to claim 3, characterized in that The performing optical flow estimation according to the preset geometric constraint and the confidence map determined by the unobstructed areas in the two sample images includes: Obtaining initial optical flow maps of the two sample images by calculation according to the preset geometric constraints; Performing forward-backward brightness detection on the initial optical flow map, taking pixels whose brightness difference exceeds a preset range as obscured pixels, and setting the confidence of the obscured pixels to 0; taking pixels whose brightness difference does not exceed the preset range as unobstructed pixels, and setting the confidence of the unobstructed pixels to 1; The optical flow estimation is performed on the two sample images according to the preset geometric constraint and the confidence map to obtain the optical flow estimation result.

5. The method according to claim 4, characterized in that The performing forward-backward brightness detection according to the initial optical flow map comprises: Get the image I in the two samples i To Image I j The forward optical flow w of pixel p on the initial optical flow map i→j (p), and obtain image I j To Image I i Backward Optical Flow in, Detect the forward optical flow w i→j (p) and backward optical flow Are the following conditions met: Where, α=0.01, β=0.5, If satisfied, the confidence of pixel p is set to 1; If not, the confidence of pixel p is set to 0.

6. The method according to claim 3, characterized in that The preset geometric constraints include triangle constraints and quadrilateral constraints; and performing optical flow estimation according to the preset geometric constraints and a confidence map determined by unobstructed areas in the two sample images includes: Through the photometric loss function L p , the quadrilateral loss function L determined according to the quadrilateral constraint q , the triangle loss function L determined according to the triangle constraint t and the confidence map to perform optical flow estimation on the two sample images; For pixel p, the photometric loss function L p for: in, According to the image I in the two samples i To Image I j The optical flow w i→j Image I j Warp to Image I i The distorted image obtained, M i→j For image I i To Image I j Confidence map of ψ(x)=(|x|+s) q , s = 0.01, q = 0.4; The quadrilateral loss function L q =L qu +L qv , L qu is the quadrilateral loss function L q The component in the horizontal direction, L qv is the quadrilateral loss function L q Component in the vertical direction, where: and are the pixels at the same position on images I1, I2, I3 and I4, I1 and I2 are binocular images collected at time t, I3 and I4 are binocular images collected at time t+1, u is the horizontal optical flow, v is the vertical optical flow, ψ(x)=(|x|+s) q , s=0.01,q=0.4,M q =M 1→2 (p)⊙M 1→3 (p)⊙M 1→4 (p), M i→j For image I i To Image I j Confidence map of ; The triangle loss function L t for: in, are the pixels at the same position in images I1 and I2, respectively, 1→4 is the optical flow from image I1 to image I4, w 2→4 is the optical flow from image I2 to image I4, w 1→2 is the optical flow from image I1 to image I2, I1 and I2 are binocular images collected at time t, I3 and I4 are binocular images collected at time t+1, M i→j For image I i To Image I j Confidence map of ψ(x)=(|x|+s) q , s=0.01, q=0.

4.

7. The method according to claim 1, characterized in that The step of using the optical flow estimation result as annotation information and performing machine learning training of image element matching on the student model using the two sample images comprises: Performing the same random cropping on the two sample images; The optical flow estimation result is used as annotation information, and the two cropped sample images are used to perform machine learning training on image element matching on the student model.

8. A model training device based on binocular images, characterized in that: Applied to training an image matching model, the image matching model includes a teacher model and a student model, and the device includes: An image acquisition module, used to acquire two sets of sample images acquired at different time points by a binocular image acquisition device; The first training module is used to estimate the optical flow of any two sample images in the two groups of sample images through the teacher model according to the preset geometric constraints between the two sample images to obtain the optical flow estimation result, wherein the preset geometric constraints are geometric constraints based on the binocular image; for the student model, the optical flow estimation result includes the proxy optical flow output by the teacher model and the agent confidence map The second training module is used to use the optical flow estimation result as annotation information and use the two sample images to perform machine learning training of image element matching on the student model. The image element matching process is to identify image elements belonging to the same object in the two sample images, including: Using the two sample images, according to the self-supervised loss function L s The student model is subjected to machine learning training for image element matching, wherein: p is the image I in the two samples i To Image I j The pixel point on w i→j is the optical flow derived from the student model, ψ(x)=(|x|+s) q , s=0.01, q=0.

4.

9. A data processing device, characterized in that: The method comprises a machine-readable storage medium and a processor, wherein the machine-readable storage medium stores machine-executable instructions, and when the machine-executable instructions are executed by the processor, the method according to any one of claims 1 to 7 is implemented.

Images