Measurement method, device, system and equipment based on binocular camera

Abstract

The invention relates to a measurement method, device, system and equipment based on a binocular camera. The method comprises the steps that a to-be-measured left view image and a to-be-measured right view image containing a target object are acquired, and each of the to-be-measured left view image and the to-be-measured right view image comprises two identification points located on the target object; respectively carrying out identification point detection on the to-be-measured left view image and the to-be-measured right view image to obtain two identification point coordinates in the to-be-measured left view image and two corresponding identification point coordinates in the to-be-measured right view image; respectively converting the identification point coordinates in the to-be-measured left view image and the to-be-measured right view image into two identification point three-dimensional coordinates; and calculating length information between the two identification points based on the three-dimensional coordinates of the identification points. By adopting the method, measurement can be carried out without additional equipment, and potential safety hazards are avoided on the basis of reducing the measurement cost.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a measurement method, device, system and equipment based on a binocular camera. Background Art

[0002] With the rapid development of technologies such as artificial intelligence, the technology of using machines to replace human eyes to perform some measurements and judgments is becoming more and more widely used. Binocular stereo vision is an important branch of machine vision. In some dangerous working environments that are not suitable for construction operations or where artificial vision cannot meet the requirements, binocular stereo vision is used to directly simulate human eyes to process the objects to be measured. This is simpler and faster, greatly improving production efficiency and automation.

[0003] In the field of medical imaging, the importance of binocular stereo vision is self-evident. It can help doctors diagnose diseases and perform surgeries more quickly and accurately. Among them, the three-dimensional measurement technology based on machine vision can achieve high-precision measurement of the three-dimensional shape and size of objects by analyzing the collected images. This simple structure, non-contact and high efficiency have broad prospects in the medical field and other fields.

[0004] As for the measurement of shapes and sizes in images, the existing technology usually requires the use of additional external devices such as rulers, beam shapers or ranging modules, and these additional devices usually increase the complexity and cost of the measuring equipment. In addition, when measuring data related to internal lesions in the human body, sending additional objects into the human body also has an impact on the surgical environment.

[0005] At present, the existing medical measurement technology requires the use of additional equipment to measure targets, which leads to high costs and certain safety hazards. No effective solution has been proposed yet. Summary of the invention

[0006] Based on this, it is necessary to provide a measurement method, device, system and equipment based on a binocular camera to address the above technical problems.

[0007] In a first aspect, the present application provides a measurement method based on a binocular camera. The method comprises:

[0008] Acquire a left view image to be measured and a right view image to be measured that contain the target object, wherein both the left view image to be measured and the right view image to be measured include two identification points located on the target object;

[0009] Performing marker point detection on the left view image to be measured and the right view image to be measured respectively, obtaining coordinates of two marker points in the left view image to be measured and corresponding coordinates of two marker points in the right view image to be measured;

[0010] Convert the coordinates of the marking points in the left-view image to be measured and the right-view image to be measured into the three-dimensional coordinates of the two marking points respectively;

[0011] The length information between two marking points is calculated based on the three-dimensional coordinates of the marking points.

[0012] In one embodiment, the coordinates of the marking points in the left view image to be measured and the right view image to be measured are respectively converted into two three-dimensional coordinates of the marking points, including:

[0013] The left view image to be measured and the right view image to be measured are detected, and if it is detected that the number of identification points in the left view image to be measured and / or the right view image to be measured is not equal to 2, an error processing is performed;

[0014] When it is detected that the number of the marking points in the left view image to be measured and the right view image to be measured is equal to 2, the coordinates of the two marking points in the left view image to be measured and the right view image to be measured are converted into two three-dimensional coordinates of the marking points.

[0015] In one embodiment, performing marker point detection on the left view image to be measured and the right view image to be measured respectively includes:

[0016] The left view image to be measured and the right view image to be measured are detected, and two auxiliary instruments are respectively detected and obtained, wherein the two auxiliary instruments are respectively placed at two marking points;

[0017] The marking point is determined by the front end point of the auxiliary instrument, and the coordinates of the marking point in the left-view image to be measured and the coordinates of the marking point in the right-view image to be measured are obtained.

[0018] In one embodiment, performing marker point detection on the left view image to be measured and the right view image to be measured respectively includes:

[0019] Acquire a first left-view image to be measured and a corresponding first right-view image to be measured containing the target object at the same time; wherein both the first left-view image to be measured and the first right-view image to be measured include an auxiliary device placed at a marking point;

[0020] After obtaining the first left-view image to be measured and the first right-view image to be measured, obtaining a second left-view image to be measured and a second right-view image to be measured containing the target object, wherein the second left-view image to be measured and the second right-view image to be measured both include the auxiliary device placed at another marking point; the left-view image to be measured includes the first left-view image to be measured and the second left-view image to be measured, and the right-view image to be measured includes the first right-view image to be measured and the second right-view image to be measured;

[0021] The first left-view image to be measured, the first right-view image to be measured, the second left-view image to be measured, and the second right-view image to be measured are detected respectively to obtain the coordinates of the marking points.

[0022] In one embodiment, the left view image to be measured and the right view image to be measured are detected, and two auxiliary devices are respectively detected and obtained, including:

[0023] Inputting the left view image to be measured and the right view image to be measured into a preset well-trained detection neural network, respectively obtaining the left view position information of the two auxiliary instruments in the left view image to be measured, and the right view position information of the two auxiliary instruments in the right view image to be measured;

[0024] Based on the left view position information and the right view position information, the coordinates of the two marking points in the left view image to be measured and the right view image to be measured are aligned to obtain the aligned coordinates of the marking points.

[0025] In one embodiment, the coordinates of the marking points in the left view image to be measured and the right view image to be measured are respectively converted into two three-dimensional coordinates of the marking points, including:

[0026] Obtaining disparity information of coordinates of the marking points in the left-view image to be measured and the right-view image to be measured through registration processing, and determining depth information of the coordinates of each marking point based on the disparity information;

[0027] Based on the depth information of the coordinates of each marking point, each marking point coordinate is converted into a corresponding three-dimensional coordinate of the marking point.

[0028] In one embodiment, the method further comprises:

[0029] Acquire a preset endoscopic view training set, wherein the endoscopic view training set carries a position information label for an auxiliary instrument and a coordinate label of an identification point of the auxiliary instrument;

[0030] The endoscopic view training set is input into the preset initial detection neural network for training to obtain the training view prediction results. The loss function result is calculated based on the training view prediction results, the position information label and the identification point coordinate label, and the gradient of the loss function result is reversely transmitted to the initial detection neural network for iterative training to generate a fully trained detection neural network.

[0031] In one embodiment, calculating the length information between two marking points based on the three-dimensional coordinates of the marking points includes:

[0032] The length information is obtained by calculating the Euclidean distance between the three-dimensional coordinates of the marking points in the three-dimensional space, and the three-dimensional coordinates of the two marking points are connected by a straight line.

[0033] In a second aspect, the present application also provides a measuring device. The device comprises:

[0034] An acquisition module is used to acquire a left view image to be measured and a right view image to be measured containing the target object, wherein the left view image to be measured and the right view image to be measured both include two identification points located on the target object;

[0035] A calculation module is used to perform identification point detection on the left view image to be measured and the right view image to be measured, respectively, to obtain the coordinates of two identification points in the left view image to be measured and the corresponding two identification point coordinates in the right view image to be measured; and to convert the identification point coordinates in the left view image to be measured and the right view image to be measured into two three-dimensional coordinates of the identification points respectively;

[0036] A generation module is used to calculate the length information between two marking points based on the three-dimensional coordinates of the marking points.

[0037] In a third aspect, the present application further provides an endoscope system, the system comprising a binocular camera and a measuring device; the binocular camera is connected to the measuring device;

[0038] The binocular camera is used to obtain a left view image to be measured and a right view image to be measured for a preset target object, and send the left view image to be measured and the right view image to be measured to a measuring device, wherein the measuring device is used to perform the functions described above.

[0039] The above-mentioned binocular camera-based measurement method, device, system and equipment first obtains the left view image to be measured and the right view image to be measured containing the target object, respectively performs identification point detection on the left view image to be measured and the right view image to be measured, and obtains the coordinates of two identification points in the left view image to be measured and the corresponding two identification point coordinates in the right view image to be measured; and converts the identification point coordinates in the left view image to be measured and the right view image to be measured into two three-dimensional coordinates of the identification points; finally, the length information between the two identification points is calculated based on the three-dimensional coordinates of the identification points. Through this application, the measurement of the line segment to be measured can be completed by identifying the identification points without the need for additional measurement equipment, which effectively reduces the measurement cost in the endoscope scene and avoids potential safety hazards. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is a schematic flow chart of a measuring method in one embodiment;

[0041] Figure 2 A schematic flow chart of a measuring method in a preferred embodiment;

[0042] Figure 3 is a structural block diagram of a measuring device in one embodiment;

[0043] Figure 4FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0045] In one embodiment, Figure 1 As shown, a measurement method based on a binocular camera is provided, comprising the following steps:

[0046] Step S110, obtaining a left view image to be measured and a right view image to be measured that contain the target object, wherein both the left view image to be measured and the right view image to be measured include two identification points located on the target object.

[0047] Specifically, the present embodiment is preferably applied to the scenario of measuring the shape, size and other information of the lesion tissue or normal tissue in the human body based on endoscope. The left-view image to be measured and the right-view image to be measured of the target object are obtained by taking pictures with a preset binocular camera, wherein the target object is the lesion or normal tissue to be measured. It should be noted that the present embodiment is mainly used for the measurement of straight line segments, rather than curves. The above-mentioned identification points are the points at the two end points of the line segments, straight-line distances, etc. in the human body that need to be measured in length. Furthermore, the identification points can be actual points or virtual auxiliary points for auxiliary measurement. The identification points can be located at the edge of the target object or inside the area where the target object is located. The relative position between the identification points and the target object is not excessively limited in this application.

[0048] Furthermore, the method for obtaining the above-mentioned identification points includes, but is not limited to, the doctor places surgical instruments at both ends of the line segment to be measured, and the position of the front end of the surgical instruments is the position of the identification points; or according to a preset neural network, automatically identifying the identification points located at both ends of the line segment to be measured on the target object in the image, etc. In some preferred embodiments, the obtained left-view image to be measured and the obtained right-view image to be measured are obtained at the same time, avoiding the problem of inconsistent coordinates caused by image jitter caused by patient breathing and other reasons.

[0049] Step S120 , performing marker point detection on the left view image to be measured and the right view image to be measured respectively, and obtaining coordinates of two marker points in the left view image to be measured and corresponding coordinates of two marker points in the right view image to be measured.

[0050] Specifically, the marker points in the left view image to be measured and the right view image to be measured are detected respectively, and each image includes only two marker points. Through these two marker points, the line segment, length, distance and other targets to be measured can be measured. In practical applications, the method for detecting marker points includes, but is not limited to, identifying the two marker points through a well-trained neural network, and marking the positions and coordinates of the two marker points. It should be noted that the coordinates of the above two marker points are two-dimensional coordinates.

[0051] Step S130 , converting the coordinates of the marking points in the left view image to be measured and the right view image to be measured into two three-dimensional coordinates of the marking points respectively.

[0052] Specifically, the two-dimensional coordinates of the marking points in the left-view image to be measured and the right-view image to be measured are converted to obtain the three-dimensional coordinates of the marking points. It can be understood that the three-dimensional coordinates of the marking points in the three-dimensional form have an additional depth coordinate axis compared to the two-dimensional coordinates of the marking points. The length information between the two marking points is calculated based on the three-dimensional coordinates of the marking points.

[0053] Through step S110 to step S130, the left-view image to be measured and the right-view image to be measured at the same time are first obtained, thereby avoiding the problem of inconsistent coordinates caused by image jitter due to patient breathing and other reasons; further, in this embodiment, the coordinates of two identification points in the left-view image to be measured and the right-view image to be measured can be detected without the need for additional measurement tools, which effectively reduces the measurement cost in the endoscope scenario and avoids potential safety hazards.

[0054] In some embodiments, the coordinates of the marking points in the left-view image to be measured and the right-view image to be measured are respectively converted into two three-dimensional coordinates of the marking points, including:

[0055] The left view image to be measured and the right view image to be measured are detected, and if it is detected that the number of identification points in the left view image to be measured and / or the right view image to be measured is not equal to 2, an error processing is performed;

[0056] When it is detected that the number of the marking points in the left view image to be measured and the right view image to be measured is equal to 2, the coordinates of the two marking points in the left view image to be measured and the right view image to be measured are converted into two three-dimensional coordinates of the marking points.

[0057] Specifically, in this embodiment, the identification points in the left-view image to be measured and the right-view image to be measured need to be further detected. If it is detected that the number of identification points in any of the left-view image to be measured and the right-view image to be measured is not equal to 2, the measurement is suspended and an error processing is performed. It can be understood that a unique line segment can be determined by two points, and if the number of identification points is not equal to 2, the target to be measured cannot be determined. Therefore, if it is detected that the number of identification points in any image is not equal to 2, an error processing is performed.

[0058] Similarly, if it is detected that the number of marking points in the left-view image to be measured and the right-view image to be measured is equal to 2, the coordinates of the marking points in the image are further converted into three-dimensional coordinates of the marking points in three-dimensional form, so as to calculate the length information between the two marking points. Through this embodiment, the marking points in the left-view image to be measured and the right-view image to be measured can be detected before the coordinate conversion, thereby further ensuring the accuracy of the measurement result and improving the detection accuracy.

[0059] In some embodiments, performing marker point detection on the left view image to be measured and the right view image to be measured respectively includes:

[0060] The left view image to be measured and the right view image to be measured are detected, and two auxiliary instruments are respectively detected and obtained, wherein the two auxiliary instruments are respectively placed at two marking points;

[0061] The marking point is determined by the front end point of the auxiliary instrument, and the coordinates of the marking point in the left-view image to be measured and the coordinates of the marking point in the right-view image to be measured are obtained.

[0062] Specifically, the present embodiment provides a step of detecting the marking point, firstly detecting the auxiliary instrument in the left view image to be measured and the right view image to be measured, wherein the auxiliary instrument is an instrument that the doctor needs to use when performing surgery or medical examination, such as scissors, tweezers, etc. There are two auxiliary instruments, and the two auxiliary instruments are placed at two marking points respectively. After the front ends of the two auxiliary instruments are placed at the two marking points, they are photographed by a binocular camera, so that the left view image to be measured and the right view image to be measured can be obtained, and the marking point can be determined by the front end of the auxiliary instrument, that is, the front end of the auxiliary instrument is the marking point, wherein the coordinates of the front end point of the auxiliary instrument photographed in the left view image to be measured and the right view image to be measured are the coordinates of the marking point. The present embodiment can be implemented based on existing surgical equipment, reducing the cost of use, and having almost no other impact on the surgical cavity or lesion area; and the marking points are marked by the doctor using the existing auxiliary instruments, which is more flexible, can accurately measure the non-edge points to be measured, and greatly improves the measurement accuracy in the weak texture area.

[0063] In some embodiments, performing marker point detection on the left view image to be measured and the right view image to be measured respectively includes:

[0064] Acquire a first left-view image to be measured and a corresponding first right-view image to be measured containing the target object at the same time; wherein both the first left-view image to be measured and the first right-view image to be measured include an auxiliary device placed at a marking point;

[0065] After obtaining the first left-view image to be measured and the first right-view image to be measured, obtaining a second left-view image to be measured and a second right-view image to be measured containing the target object, wherein the second left-view image to be measured and the second right-view image to be measured both include the auxiliary device placed at another marking point; the left-view image to be measured includes the first left-view image to be measured and the second left-view image to be measured, and the right-view image to be measured includes the first right-view image to be measured and the second right-view image to be measured;

[0066] The first left-view image to be measured, the first right-view image to be measured, the second left-view image to be measured, and the second right-view image to be measured are detected respectively to obtain the coordinates of the marking points.

[0067] Specifically, at the same time, a binocular camera is used to photograph the target object and an auxiliary device placed at a marking point on the target object, and a pair of first left-view images to be measured and a first right-view image to be measured containing the target object are obtained. After the first left-view image to be measured and the first right-view image to be measured are photographed, the position of the auxiliary device is moved to another marking point, and then the target object and the auxiliary device placed at another marking point on the target object are photographed by the binocular camera to obtain the second left-view image to be measured and the second right-view image to be measured. Through this embodiment, an auxiliary device is placed at two marking points in succession and photographed, and the coordinates of the two marking points can be collected based on one auxiliary device, which reduces the use requirements of the present application and expands the scope of application.

[0068] Furthermore, after obtaining the identification points included in the first left-view image to be measured and the first right-view image to be measured, as well as obtaining another identification point included in the second left-view image to be measured and the second right-view image to be measured through this embodiment, considering that in actual applications, the picture generally changes due to two image captures, the images can be further aligned. For example, the surf algorithm can be used to complete the image alignment, and the identification points included in the first left-view image to be measured and the first right-view image to be measured are mapped to the second left-view image to be measured and the second right-view image to be measured, thereby unifying the spatial coordinate systems of the two identification points captured twice, which facilitates the subsequent more accurate calculation of the length information between the two identification points.

[0069] In some embodiments, the left view image to be measured and the right view image to be measured are detected, and two auxiliary devices are respectively detected and obtained, including:

[0070] Inputting the left view image to be measured and the right view image to be measured into a preset well-trained detection neural network, respectively obtaining the left view position information of the two auxiliary instruments in the left view image to be measured, and the right view position information of the two auxiliary instruments in the right view image to be measured;

[0071] Based on the left view position information and the right view position information, the coordinates of the two marking points in the left view image to be measured and the right view image to be measured are aligned to obtain the aligned coordinates of the marking points.

[0072] Specifically, the collected left view image to be measured and the right view image to be measured are input into a well-trained detection neural network, and the left view position information of the area where the two auxiliary devices are located in the left view image to be measured and the right view position information of the area where the two auxiliary devices are located in the right view image to be measured can be obtained. It can be understood from the present application that the above-mentioned left view position information and right view position information both include the area range information of the area where the auxiliary devices are located. In actual applications, the front end point of the above-mentioned auxiliary device is generally located at the front end of the area range of the auxiliary device. It can be understood that the front end point coordinate information is the above-mentioned identification point coordinate.

[0073] Further, according to the above-mentioned left view position information and right view position information, the position information of the two auxiliary instruments can be matched first, so as to determine the registration of the coordinates of the identification points of the two auxiliary instruments in the left view image to be measured and the right view image to be measured. It can be understood that the number of auxiliary instruments in the present application is two, and the two auxiliary instruments are respectively located at two identification points, and have a clear spatial position relationship. Therefore, the matching of the position area where the two auxiliary instruments are located in the left view position information of this embodiment and the position area where the two auxiliary instruments are located in the right view position information is relatively simple. In this embodiment, since there are only two auxiliary instruments, the solution of directly obtaining identification points from the binocular view in this embodiment does not require complex image matching operations, and due to the auxiliary detection of the front end point of the auxiliary instrument, it effectively solves the problem that the image matching is easy to mismatch in the weak texture area, resulting in inaccurate measurement results and time-consuming, and also improves the accuracy of identifying identification points in the weak texture area.

[0074] In some embodiments, the coordinates of the marking points in the left-view image to be measured and the right-view image to be measured are respectively converted into two three-dimensional coordinates of the marking points, including:

[0075] Obtaining disparity information of coordinates of the marking points in the left-view image to be measured and the right-view image to be measured through registration processing, and determining depth information of the coordinates of each marking point based on the disparity information;

[0076] Based on the depth information of the coordinates of each marking point, each marking point coordinate is converted into a corresponding three-dimensional coordinate of the marking point.

[0077] Specifically, in practical applications, binocular cameras are generally calibrated, so there is only lateral parallax in the left view image to be measured and the right view image to be measured, that is, the above disparity information is the lateral disparity information of the coordinates of the marking points in the left view and the right view. Based on the above disparity information, the depth information of the coordinates of each marking point can be determined, and the calculation formula of the depth information is:

[0078]

[0079] Among them, f is the focal length of the camera, b is the baseline distance, d is the parallax information, and z is the depth value of the coordinate. It can be understood that after the camera is calibrated, the focal length and baseline distance of the camera are known parameters. Therefore, according to the depth information of the coordinates of each marker point calculated, the coordinates of each marker point are converted into the corresponding three-dimensional coordinates of the marker point.

[0080] In some embodiments, the method further comprises:

[0081] Acquire a preset endoscopic view training set, wherein the endoscopic view training set carries a position information label for an auxiliary instrument and a coordinate label of an identification point of the auxiliary instrument;

[0082] The endoscopic view training set is input into the preset initial detection neural network for training to obtain the training view prediction results. The loss function result is calculated based on the training view prediction results, the position information label and the identification point coordinate label, and the gradient of the loss function result is reversely transmitted to the initial detection neural network for iterative training to generate a fully trained detection neural network.

[0083] Specifically, relevant materials of the measurement scene can be collected first to establish the above-mentioned endoscope view training set. It can be understood that the collected materials should include auxiliary equipment, location information labels of the area where the auxiliary equipment is located, and identification point coordinate labels of the coordinates of the front end points of the auxiliary equipment.

[0084] The preset initial detection neural network is trained through the above-mentioned endoscopic view training set, wherein the loss function can be selected by relevant technical personnel, such as L1 loss function, L2 loss function, or IoU (Intersection over Union Loss) loss function, etc. After the training is completed, the above-mentioned fully trained detection neural network for target area detection and key point detection can be obtained, so that the position information of the auxiliary device in the view and the front end point coordinates of the auxiliary device, that is, the identification point coordinates, can be detected through the above-mentioned detection neural network.

[0085] In some embodiments, calculating the length information between two marking points based on the three-dimensional coordinates of the marking points includes:

[0086] The length information is obtained by calculating the Euclidean distance between the three-dimensional coordinates of the marking points in the three-dimensional space, and the three-dimensional coordinates of the two marking points are connected by a straight line.

[0087] Specifically, in three-dimensional space, the Euclidean distance between two known marker points can be calculated based on their three-dimensional coordinates to obtain the length information between the two marker points. Alternatively, a straight line can be connected between the two marker points and displayed in the image.

[0088] The present application also provides a preferred embodiment of a measurement method based on a binocular camera. Figure 2 FIG. 1 is a flow chart of a measuring method in an embodiment.

[0089] Step S210, training the preset initial detection neural network to obtain a fully trained detection neural network, wherein an endoscopic view training set of the measurement scene is collected, the endoscopic view training set includes surgical instruments of different types and numbers, i.e., the above-mentioned auxiliary instruments, and correspondingly, also includes position information labels of the auxiliary instruments and identification point coordinate labels of the auxiliary instruments. The endoscopic view training set is input into the initial detection neural network for training, and the above-mentioned fully trained detection neural network can be obtained after the iteration is completed.

[0090] In step S220, the acquired left-view image to be measured and the acquired right-view image to be measured are input into the detection neural network, so as to obtain the number of auxiliary instruments contained in each image, the position information of the auxiliary instruments and the coordinates of the front end points of the auxiliary instruments.

[0091] In step S230, the number of auxiliary instruments in the left view image to be measured and the right view image to be measured is judged respectively. If the number of auxiliary instruments in each image is two, jump to step S240. If the number of auxiliary instruments is not equal to two, end the measurement task and perform error processing.

[0092] Step S240, according to the position information of each auxiliary device in the image, the coordinates of the marking points of the two auxiliary devices in the left view image to be measured and the right view image to be measured are registered to obtain the parallax information of the coordinates of the marking points.

[0093] Step S250, calculating the depth information of each marker point coordinate based on the disparity information, and converting the two-dimensional marker point coordinates into three-dimensional coordinates of the marker point based on the depth information.

[0094] Step S260, obtain the length information between the two marking points by calculating the Euclidean distance between the three-dimensional coordinates of the marking points in the three-dimensional space, connect the three-dimensional coordinates of the two marking points with a straight line, and display them.

[0095] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0096] Based on the same inventive concept, the embodiment of the present application also provides a measuring device for implementing the above-mentioned binocular camera-based measuring method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above-mentioned method, so the specific limitations in one or more measuring device embodiments provided below can refer to the limitations of the binocular camera-based measuring method above, and will not be repeated here.

[0097] In one embodiment, Figure 3 As shown, a measuring device is provided, including: an acquisition module 31, a calculation module 32 and a generation module 33, wherein:

[0098] An acquisition module 31 is used to acquire a left view image to be measured and a right view image to be measured containing a target object, wherein both the left view image to be measured and the right view image to be measured include two identification points located on the target object;

[0099] The calculation module 32 is used to perform identification point detection on the left view image to be measured and the right view image to be measured, respectively, to obtain the coordinates of two identification points in the left view image to be measured and the corresponding two identification point coordinates in the right view image to be measured; and to convert the identification point coordinates in the left view image to be measured and the right view image to be measured into two three-dimensional coordinates of the identification points respectively;

[0100] The generating module 33 is used to calculate the length information between two marking points based on the three-dimensional coordinates of the marking points.

[0101] Specifically, the acquisition module 31 acquires a left-view image to be measured and a right-view image to be measured containing the target object, wherein the left-view image to be measured and the right-view image to be measured both include two identification points located on the target object, and sends the identification points to the calculation module 32. The calculation module 32 performs identification point detection on the left-view image to be measured and the right-view image to be measured, respectively, to obtain the coordinates of the identification points in the left-view image to be measured and the coordinates of the identification points in the right-view image to be measured, converts the coordinates of the identification points in the left-view image to be measured and the right-view image to be measured into three-dimensional coordinates of the two identification points respectively, and sends the three-dimensional coordinates of the identification points to the generation module 33. The generation module 33 calculates the length information between the two identification points based on the three-dimensional coordinates of the identification points.

[0102] Each module in the above-mentioned measuring device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module above.

[0103] The present application also provides an endoscope system, comprising a binocular camera and a measuring device as described above; the binocular camera is connected to the measuring device;

[0104] The binocular camera is used to obtain a left view image to be measured and a right view image to be measured for a preset target object, and send the left view image to be measured and the right view image to be measured to a measuring device, wherein the measuring device is used to perform the steps described above.

[0105] Specifically, this embodiment also provides an endoscope system, which includes but is not limited to a binocular camera and a measuring device. The calibrated binocular camera collects the left view image to be measured and the right view image to be measured, and sends them to the measuring device. The measuring device implements the measurement function of the distance between the two marking points mentioned above.

[0106] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 4 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store measurement algorithm related data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a measurement method based on a binocular camera is implemented.

[0107] Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0108] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0109] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0110] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0111] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A measurement method based on a binocular camera, characterized in that: The method comprises: Acquire a left view image to be measured and a right view image to be measured that contain a target object, wherein both the left view image to be measured and the right view image to be measured include two identification points located on the target object; Respectively perform identification point detection on the left view image to be measured and the right view image to be measured to obtain coordinates of two identification points in the left view image to be measured and corresponding coordinates of two identification points in the right view image to be measured; Converting the coordinates of the marking points in the left-view image to be measured and the right-view image to be measured into two three-dimensional coordinates of the marking points respectively; The length information between the two marking points is calculated based on the three-dimensional coordinates of the marking points.

2. The method according to claim 1, characterized in that The step of converting the coordinates of the marking points in the left-view image to be measured and the right-view image to be measured into two three-dimensional coordinates of the marking points respectively comprises: Detecting the left-view image to be measured and the right-view image to be measured, and if it is detected that the number of the identification points in the left-view image to be measured and / or the right-view image to be measured is not equal to 2, performing error processing; When it is detected that the number of the marking points in the left-view image to be measured and the right-view image to be measured is both equal to 2, the coordinates of the two marking points in the left-view image to be measured and the right-view image to be measured are converted into two three-dimensional coordinates of the marking points.

3. The method according to claim 1, characterized in that The detecting of the marking points on the left-view image to be measured and the right-view image to be measured respectively comprises: Detecting the left-view image to be measured and the right-view image to be measured, respectively detecting and obtaining two auxiliary instruments, wherein the two auxiliary instruments are respectively placed at the two marking points; The marking point is determined by the front end point of the auxiliary instrument, and the coordinates of the marking point in the left-view image to be measured and the coordinates of the marking point in the right-view image to be measured are obtained.

4. The method according to claim 1, characterized in that: The detecting of the marking points on the left-view image to be measured and the right-view image to be measured respectively comprises: Acquire a first left-view image to be measured and a corresponding first right-view image to be measured containing the target object at the same time; wherein both the first left-view image to be measured and the first right-view image to be measured include an auxiliary device placed at one of the marking points; After acquiring the first left-view image to be measured and the first right-view image to be measured, acquiring a second left-view image to be measured and a second right-view image to be measured containing the target object, wherein the second left-view image to be measured and the second right-view image to be measured both include the auxiliary device placed at another of the marking points; the left-view image to be measured includes the first left-view image to be measured and the second left-view image to be measured, and the right-view image to be measured includes the first right-view image to be measured and the second right-view image to be measured; The first left-view image to be measured, the first right-view image to be measured, the second left-view image to be measured, and the second right-view image to be measured are detected respectively to obtain the coordinates of the marking points.

5. The method according to claim 3, characterized in that: The detecting of the left-view image to be measured and the right-view image to be measured, respectively detecting and obtaining two auxiliary instruments, comprises: Inputting the left view image to be measured and the right view image to be measured into a preset well-trained detection neural network, respectively obtaining left view position information of the two auxiliary devices in the left view image to be measured, and right view position information of the two auxiliary devices in the right view image to be measured; Based on the left view position information and the right view position information, the coordinates of the two marking points in the left view image to be measured and the right view image to be measured are aligned to obtain the aligned coordinates of the marking points.

6. The method according to claim 5, characterized in that The step of converting the coordinates of the marking points in the left-view image to be measured and the right-view image to be measured into two three-dimensional coordinates of the marking points respectively comprises: Obtaining disparity information of the coordinates of the marking points in the left-view image to be measured and the right-view image to be measured through the registration process, and determining depth information of the coordinates of each marking point based on the disparity information; Based on the depth information of each of the marker point coordinates, each of the marker point coordinates is converted into a corresponding three-dimensional coordinate of the marker point.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: Acquire a preset endoscopic view training set, wherein the endoscopic view training set carries a location information label for an auxiliary instrument and a coordinate label of an identification point of the auxiliary instrument; The endoscopic view training set is input into a preset initial detection neural network for training to obtain a training view prediction result, and a loss function result is calculated based on the training view prediction result, the position information label and the identification point coordinate label, and the gradient of the loss function result is reversely transmitted to the initial detection neural network for iterative training to generate a fully trained detection neural network.

8. The method according to claim 1, characterized in that The calculating the length information between two marking points based on the three-dimensional coordinates of the marking points includes: The length information is obtained by calculating the Euclidean distance between the three-dimensional coordinates of the marking points in the three-dimensional space, and the three-dimensional coordinates of two marking points are connected by a straight line.

9. A measurement device based on a binocular camera, characterized in that: The device comprises: An acquisition module, used for acquiring a left view image to be measured and a right view image to be measured containing a target object, wherein the left view image to be measured and the right view image to be measured both include two identification points located on the target object; A calculation module is used to perform identification point detection on the left-view image to be measured and the right-view image to be measured, respectively, to obtain the coordinates of two identification points in the left-view image to be measured and the corresponding two identification point coordinates in the right-view image to be measured; and to convert the identification point coordinates in the left-view image to be measured and the right-view image to be measured into two three-dimensional coordinates of the identification points, respectively; A generation module is used to calculate the length information between two of the identification points based on the three-dimensional coordinates of the identification points.

10. An endoscope system, characterized in that: The system includes a binocular camera and a measuring device; the binocular camera is connected to the measuring device; The binocular camera is used to obtain a left view image to be measured and a right view image to be measured for a preset target object, and send the left view image to be measured and the right view image to be measured to the measuring device, wherein the measuring device is used to perform the function as described in claim 9.

Images