A three-dimensional reconstruction and measurement method and system using multi-eye stereo vision
By constructing a 3D fusion map using multi-view stereo vision technology, the distance measurement objects and hazard sources in the power field are identified and reconstructed, solving the problem of occlusion affecting distance measurement in specific parts of the power field, and realizing accurate distance prediction and early warning.
Patent Information
- Application Number
- CN202410512851.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-04-26
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-04-26
AI Technical Summary
In power field operations, distance measurements between the target object and the hazard source need to be accurate to a specific location. However, due to the complex environment, the distance is often obstructed, affecting the accuracy of the measurements.
Using multi-view stereo vision technology, images are acquired through multiple vision cameras to build a 3D fusion map, identify ranging objects and key parts, directly measure or predict the distance between specific parts and hazard sources, and use reconstruction history and feature extraction technology to perform adaptive model reconstruction.
It improves the applicability of detailed distance measurement scenarios in power fields, can accurately predict distances when there is obstruction in specific locations, and issue warnings when danger approaches.
Smart Images

Figure CN118447158B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of stereoscopic vision technology, and in particular to a three-dimensional reconstruction and measurement method and system using multi-view stereoscopic vision. Background Art
[0002] At present, stereo vision is mostly used to measure the distance between the distance measurement object and the danger source at the power site, to determine whether the distance measurement object will be threatened by the danger source, and to make necessary and timely safety warnings. However, in the power site, it is often necessary to carry out detailed distance measurement between the distance measurement object and the danger source. Specifically, the danger threat from the danger source generally means that a specific part of the distance measurement object has produced a specific action, such as: reaching out, probing, etc. The distance measurement between the distance measurement object and the danger source requires measuring the precise distance between the specific part and the danger source. However, the on-site environment in the power site is often complex, and a specific part may be blocked, affecting the precise distance measurement. Therefore, a solution is urgently needed. Summary of the invention
[0003] One of the purposes of the present invention is to provide a three-dimensional reconstruction and measurement method using multi-eye stereo vision. When the precise distance between a specific part and a hazardous source cannot be measured in detail, a second distance is predicted, thereby improving the applicability of the system in refining distance measurement scenarios in power sites.
[0004] An embodiment of the present invention provides a three-dimensional reconstruction and measurement method using multi-eye stereo vision, comprising:
[0005] activating multiple visual cameras within the target scene;
[0006] Acquire a visual image captured by each of the visual cameras;
[0007] Based on the visual image, construct a three-dimensional fusion image;
[0008] Obtaining the preset distance measurement object type, key part type and hazard source type corresponding to the target scene;
[0009] Attempting to identify a ranging object of the ranging object type from the three-dimensional fusion image;
[0010] When the attempt is successful, attempting to identify a key part of the key part type on the ranging object from the three-dimensional fusion image;
[0011] When the attempt is successful, a first distance between the key part and a hazard source of the hazard source type in the target scene is determined from the three-dimensional fusion image; otherwise, a second distance between the key part and the hazard source is predicted based on the three-dimensional fusion image.
[0012] Preferably, predicting the second distance between the key part and the danger source based on the three-dimensional fusion image includes:
[0013] Identify the visible part of the distance measurement object from the three-dimensional fusion image;
[0014] Determining whether the visible part meets the standard part condition;
[0015] When the answer is yes, a template is generated based on a preset reconstruction history condition, and a reconstruction history condition is generated according to the key part and the visible part;
[0016] Trying to search for a reconstruction history that meets the reconstruction history condition from a preset reconstruction history library;
[0017] When the attempt is successful, determining a first part posture relationship between the visible part and the key part based on the reconstruction history;
[0018] Reconstructing a first part model of the key part in the three-dimensional fusion image based on the first part posture relationship;
[0019] Determine the second distance between the first part model and the danger source from the three-dimensional fusion image;
[0020] When the attempt fails, extracting features from the visible part based on a preset feature extraction template to obtain multiple part features;
[0021] Generate a template based on a preset posture relationship condition, and generate a posture relationship condition according to the plurality of part features and a preset action probability table corresponding to the target scene;
[0022] Determine a second part posture relationship that meets the posture relationship condition from a preset part posture relationship library;
[0023] Reconstructing a second part model of the key part in the three-dimensional fusion image based on the posture relationship of the second part;
[0024] The second distance between the second part model and the danger source is determined from the three-dimensional fusion image.
[0025] Preferably, the standard site conditions include:
[0026] There is at least one corresponding association relationship between the visible part and the key part in a preset part association relationship library;
[0027] The positions where the operating actions of the personnel ranked top N in probability in the action probability table should occur match the visible position; N is a positive integer.
[0028] Preferably, the three-dimensional reconstruction and measurement method using multi-eye stereo vision also includes:
[0029] Obtaining a preset safety distance corresponding to the hazard source type;
[0030] When the first distance / the second distance is less than or equal to the safety distance, a preset warning message is output.
[0031] Preferably, the three-dimensional reconstruction and measurement method using multi-eye stereo vision also includes:
[0032] The third distance between the distance measuring object and the danger source is continuously tracked.
[0033] An embodiment of the present invention provides a three-dimensional reconstruction and measurement system using multi-eye stereo vision, comprising:
[0034] An activation module, used for activating multiple visual cameras within a target scene;
[0035] A first acquisition module, used to acquire the visual image captured by each of the visual cameras;
[0036] A building module, used for building a three-dimensional fusion image based on the visual image;
[0037] A second acquisition module is used to acquire a preset distance measurement object type, a key part type, and a hazard source type corresponding to the target scene;
[0038] A first identification module, configured to attempt to identify a ranging object of the ranging object type from the three-dimensional fusion image;
[0039] A second identification module, configured to attempt to identify a key part of the key part type on the ranging object from the three-dimensional fusion image when the attempt is successful;
[0040] A distance measurement module is used to determine, from the three-dimensional fusion image, a first distance between the key part and a hazard source of the hazard source type in the target scene when the attempt is successful; otherwise, based on the three-dimensional fusion image, predict a second distance between the key part and the hazard source.
[0041] The distance measurement module predicts a second distance between the key part and the danger source based on the three-dimensional fusion image, including:
[0042] Identify the visible part of the distance measurement object from the three-dimensional fusion image;
[0043] Determining whether the visible part meets the standard part condition;
[0044] When the answer is yes, a template is generated based on a preset reconstruction history condition, and a reconstruction history condition is generated according to the key part and the visible part;
[0045] Trying to search for a reconstruction history that meets the reconstruction history condition from a preset reconstruction history library;
[0046] When the attempt is successful, determining a first part posture relationship between the visible part and the key part based on the reconstruction history;
[0047] Reconstructing a first part model of the key part in the three-dimensional fusion image based on the first part posture relationship;
[0048] Determine the second distance between the first part model and the danger source from the three-dimensional fusion image;
[0049] When the attempt fails, extracting features from the visible part based on a preset feature extraction template to obtain multiple part features;
[0050] Generate a template based on a preset posture relationship condition, and generate a posture relationship condition according to the plurality of part features and a preset action probability table corresponding to the target scene;
[0051] Determine a second part posture relationship that meets the posture relationship condition from a preset part posture relationship library;
[0052] Reconstructing a second part model of the key part in the three-dimensional fusion image based on the posture relationship of the second part;
[0053] The second distance between the second part model and the danger source is determined from the three-dimensional fusion image.
[0054] The standard site conditions include:
[0055] There is at least one corresponding association relationship between the visible part and the key part in a preset part association relationship library;
[0056] The positions where the operating actions of the personnel ranked top N in probability in the action probability table should occur match the visible position; N is a positive integer.
[0057] The 3D reconstruction and measurement system using multi-view stereo vision also includes:
[0058] The early warning module includes:
[0059] Obtaining a preset safety distance corresponding to the hazard source type;
[0060] When the first distance / the second distance is less than or equal to the safety distance, a preset warning message is output.
[0061] The 3D reconstruction and measurement system using multi-view stereo vision also includes:
[0062] A tracking module is used to continuously track the third distance between the distance measuring object and the danger source.
[0063] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings.
[0064] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS
[0065] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:
[0066] Figure 1 A schematic diagram of a three-dimensional reconstruction and measurement method using multi-view stereo vision according to an embodiment of the present invention;
[0067] Figure 2 Schematic diagram of a three-dimensional reconstruction and measurement system using multi-view stereo vision in an embodiment of the present invention. DETAILED DESCRIPTION
[0068] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.
[0069] The embodiment of the present invention provides a three-dimensional reconstruction and measurement method using multi-eye stereo vision, such as Figure 1 As shown, including:
[0070] Step S1: activating multiple visual cameras within the target scene;
[0071] Step S2: Acquire the visual image captured by each of the visual cameras;
[0072] Step S3: constructing a three-dimensional fusion image based on the visual image;
[0073] Step S4: obtaining the preset distance measurement object type, key part type and hazard source type corresponding to the target scene;
[0074] Step S5: trying to identify the ranging object of the ranging object type from the three-dimensional fusion image;
[0075] Step S6: when the attempt is successful, attempting to identify a key part of the key part type on the distance measurement object from the three-dimensional fusion image;
[0076] Step S7: When the attempt is successful, determine the first distance between the key part and the hazard source of the hazard source type in the target scene from the three-dimensional fusion image; otherwise, predict the second distance between the key part and the hazard source based on the three-dimensional fusion image.
[0077] The target scene is a scene within the power site; when building a three-dimensional scene graph, based on the three-dimensional fusion technology, the visual images collected by each visual camera are three-dimensionally fused to obtain a three-dimensional fusion graph; the distance measurement object type is the type of object that needs to be warned of the distance to the hazard source in the target scene, such as: power construction personnel, power operation vehicles, etc.; the key part type is the specific part that needs to be measured when measuring the distance between the distance measurement object and the hazard source when issuing a hazard source distance warning in the target scene, such as: hands, head, etc.; the hazard source type is the type of hazard source existing in the target scene, such as: high-voltage equipment, etc.; when the distance measurement object is identified from the three-dimensional fusion graph, an attempt is made to identify the key part of the distance measurement object, and when it is identified, the first distance is directly determined, and when it cannot be determined, the second distance is predicted; when the precise distance between the specific part and the hazard source cannot be refined and measured, the application predicts the second distance, thereby improving the applicability of the system in refining the distance measurement scene in the power site.
[0078] In one embodiment, predicting the second distance between the key part and the danger source based on the three-dimensional fusion image includes:
[0079] Identify the visible part of the distance measuring object from the three-dimensional fused image; the visible part is the body part of the distance measuring object that can be identified in the three-dimensional fused image;
[0080] Determining whether the visible part meets the standard part condition;
[0081] When the answer is yes, a template is generated based on a preset reconstruction history condition, and a reconstruction history condition is generated according to the key part and the visible part; the reconstruction history condition includes: the key part and the visible part appear simultaneously in the historical three-dimensional fusion image in the reconstruction history;
[0082] Trying to search for a reconstruction history that meets the reconstruction history condition from a preset reconstruction history library;
[0083] When the attempt is successful, determining a first part posture relationship between the visible part and the key part based on the reconstruction history; the first part posture relationship is a relative posture relationship between the visible part and the key part in the historical three-dimensional fusion image in the reconstruction history;
[0084] Based on the first part posture relationship, reconstructing the first part model of the key part in the three-dimensional fusion image; a visible part appears in the three-dimensional fusion image, and therefore, based on the first part posture relationship, the first part model of the key part can be reconstructed in the three-dimensional fusion image;
[0085] Determine the second distance between the first part model and the danger source from the three-dimensional fusion image; directly determine the second distance after reconstructing the first part model;
[0086] When the attempt fails, feature extraction is performed on the visible part based on a preset feature extraction template to obtain multiple part features; the part features include: part posture, part size, etc.;
[0087] A template is generated based on a preset posture relationship condition, and a posture relationship condition is generated according to the preset action probability table corresponding to the plurality of part features and the target scene; the action probability table includes personnel operation actions sorted from large to small in probability of occurrence, and the greater the probability of occurrence, the more likely the corresponding personnel operation action is to occur in the target scene; the posture relationship condition includes: when a visible part having the part feature has a second part posture relationship with the key part, the visible part and the key part jointly generate the personnel operation action;
[0088] Determine a second part posture relationship that meets the posture relationship condition from a preset part posture relationship library; a large number of second part posture relationships corresponding to different posture relationship conditions are preset in the part posture relationship library, and can be set by technical personnel according to actual needs; the first part posture relationship is a relative posture relationship between a visible part and a key part;
[0089] Based on the posture relationship of the second part, reconstructing the second part model of the key part in the three-dimensional fusion image; similarly, a visible part appears in the three-dimensional fusion image, so based on the posture relationship of the second part, the second part model of the key part can be reconstructed in the three-dimensional fusion image;
[0090] The second distance between the second part model and the danger source is determined from the three-dimensional fusion image. After the second part model is reconstructed, the second distance is directly determined.
[0091] When predicting the second distance, the embodiment of the present invention can adaptively reconstruct the part model, thereby improving the prediction efficiency of the second distance.
[0092] In one embodiment, the standard site conditions include:
[0093] There is at least one corresponding relationship between the visible part and the key part in a preset part association relationship library; the association relationship includes: part connection relationship, etc.;
[0094] The parts of the human work actions that are ranked top N in probability in the action probability table are matched with the visible parts; N is a positive integer. The parts that are supposed to be produced are the parts of the body that the human should use when performing the human work action; the standard part conditions are set so that the screened visible parts are more conducive to the reconstruction of the first part model and the second part model. In one embodiment, the three-dimensional reconstruction and measurement method using multi-eye stereo vision also includes:
[0095] Obtaining a preset safety distance corresponding to the hazard source type;
[0096] When the first distance / the second distance is less than or equal to the safety distance, a preset warning message is output.
[0097] In one embodiment, the three-dimensional reconstruction and measurement method using multi-eye stereo vision further includes:
[0098] The third distance between the distance measuring object and the danger source is continuously tracked.
[0099] The embodiment of the present invention provides a three-dimensional reconstruction and measurement system using multi-eye stereo vision, such as Figure 2 As shown, including:
[0100] An activation module 1 is used to activate multiple visual cameras in a target scene;
[0101] A first acquisition module 2, used to acquire the visual image captured by each of the visual cameras;
[0102] A building module 3 is used to build a three-dimensional fusion image based on the visual image;
[0103] The second acquisition module 4 is used to acquire the preset distance measurement object type, key part type and hazard source type corresponding to the target scene;
[0104] A first recognition module 5, configured to attempt to recognize a ranging object of the ranging object type from the three-dimensional fusion image;
[0105] A second recognition module 6, configured to attempt to recognize a key part of the key part type on the distance measurement object from the three-dimensional fusion image when the attempt is successful;
[0106] The distance measuring module 7 is used to determine the first distance between the key part and the hazard source of the hazard source type in the target scene from the three-dimensional fusion image when the attempt is successful; otherwise, predict the second distance between the key part and the hazard source based on the three-dimensional fusion image.
[0107] The distance measuring module 7 predicts the second distance between the key part and the danger source based on the three-dimensional fusion image, including:
[0108] Identify the visible part of the distance measurement object from the three-dimensional fusion image;
[0109] Determining whether the visible part meets the standard part condition;
[0110] When the answer is yes, a template is generated based on a preset reconstruction history condition, and a reconstruction history condition is generated according to the key part and the visible part;
[0111] Trying to search for a reconstruction history that meets the reconstruction history condition from a preset reconstruction history library;
[0112] When the attempt is successful, determining a first part posture relationship between the visible part and the key part based on the reconstruction history;
[0113] Reconstructing a first part model of the key part in the three-dimensional fusion image based on the first part posture relationship;
[0114] Determine the second distance between the first part model and the danger source from the three-dimensional fusion image;
[0115] When the attempt fails, extracting features from the visible part based on a preset feature extraction template to obtain multiple part features;
[0116] Generate a template based on a preset posture relationship condition, and generate a posture relationship condition according to the plurality of part features and a preset action probability table corresponding to the target scene;
[0117] Determine a second part posture relationship that meets the posture relationship condition from a preset part posture relationship library;
[0118] Reconstructing a second part model of the key part in the three-dimensional fusion image based on the posture relationship of the second part;
[0119] The second distance between the second part model and the danger source is determined from the three-dimensional fusion image.
[0120] The standard site conditions include:
[0121] There is at least one corresponding association relationship between the visible part and the key part in a preset part association relationship library;
[0122] The positions where the operating actions of the personnel ranked top N in probability in the action probability table should occur match the visible position; N is a positive integer.
[0123] The 3D reconstruction and measurement system using multi-view stereo vision also includes:
[0124] The early warning module includes:
[0125] Obtaining a preset safety distance corresponding to the hazard source type;
[0126] When the first distance / the second distance is less than or equal to the safety distance, a preset warning message is output.
[0127] The 3D reconstruction and measurement system using multi-view stereo vision also includes:
[0128] A tracking module is used to continuously track the third distance between the distance measuring object and the danger source.
[0129] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.
Claims
1. A three-dimensional reconstruction and measurement method using multi-eye stereo vision, characterized in that: include: activating multiple visual cameras within the target scene; Acquire a visual image captured by each of the visual cameras; Based on the visual image, construct a three-dimensional fusion image; Obtaining the preset distance measurement object type, key part type and hazard source type corresponding to the target scene; Attempting to identify a ranging object of the ranging object type from the three-dimensional fusion image; When the attempt is successful, attempting to identify a key part of the key part type on the ranging object from the three-dimensional fusion image; When the attempt is successful, determining a first distance between the key part and a hazard source of the hazard source type in the target scene from the three-dimensional fusion image; Otherwise, predicting a second distance between the key part and the danger source based on the three-dimensional fusion image; The predicting, based on the three-dimensional fusion image, a second distance between the key part and the danger source comprises: Identify the visible part of the distance measurement object from the three-dimensional fusion image; Determining whether the visible part meets the standard part condition; When the answer is yes, a template is generated based on a preset reconstruction history condition, and a reconstruction history condition is generated according to the key part and the visible part; Trying to search for a reconstruction history that meets the reconstruction history condition from a preset reconstruction history library; When the attempt is successful, determining a first part posture relationship between the visible part and the key part based on the reconstruction history; Reconstructing a first part model of the key part in the three-dimensional fusion image based on the first part posture relationship; Determine the second distance between the first part model and the danger source from the three-dimensional fusion image; When the attempt fails, extracting features from the visible part based on a preset feature extraction template to obtain multiple part features; Generate a template based on a preset posture relationship condition, and generate a posture relationship condition according to the plurality of part features and a preset action probability table corresponding to the target scene; Determine a second part posture relationship that meets the posture relationship condition from a preset part posture relationship library; Reconstructing a second part model of the key part in the three-dimensional fusion image based on the posture relationship of the second part; The second distance between the second part model and the danger source is determined from the three-dimensional fusion image.
2. A three-dimensional reconstruction and measurement method using multi-eye stereo vision as claimed in claim 1, characterized in that: The standard site conditions include: There is at least one corresponding association relationship between the visible part and the key part in a preset part association relationship library; The positions where the operating actions of the personnel ranked top N in probability in the action probability table should occur match the visible position; N is a positive integer.
3. The three-dimensional reconstruction and measurement method using multi-eye stereo vision as claimed in claim 1, characterized in that: Also includes: Obtaining a preset safety distance corresponding to the hazard source type; When the first distance / the second distance is less than or equal to the safety distance, a preset warning message is output.
4. A three-dimensional reconstruction and measurement method using multi-eye stereo vision as claimed in claim 1, characterized in that: Also includes: The third distance between the distance measuring object and the danger source is continuously tracked.
5. A three-dimensional reconstruction and measurement system using multi-eye stereo vision, characterized in that: include: An activation module, used for activating multiple visual cameras within a target scene; A first acquisition module, used to acquire the visual image captured by each of the visual cameras; A building module, used for building a three-dimensional fusion image based on the visual image; A second acquisition module is used to acquire a preset distance measurement object type, a key part type, and a hazard source type corresponding to the target scene; A first identification module, configured to attempt to identify a ranging object of the ranging object type from the three-dimensional fusion image; A second identification module, configured to attempt to identify a key part of the key part type on the ranging object from the three-dimensional fusion image when the attempt is successful; a distance measurement module, for determining, from the three-dimensional fusion image, a first distance between the key part and a hazard source of the hazard source type in the target scene when the attempt is successful; Otherwise, predicting a second distance between the key part and the danger source based on the three-dimensional fusion image; The distance measurement module predicts a second distance between the key part and the danger source based on the three-dimensional fusion image, including: Identify the visible part of the distance measurement object from the three-dimensional fusion image; Determining whether the visible part meets the standard part condition; When the answer is yes, a template is generated based on a preset reconstruction history condition, and a reconstruction history condition is generated according to the key part and the visible part; Trying to search for a reconstruction history that meets the reconstruction history condition from a preset reconstruction history library; When the attempt is successful, determining a first part posture relationship between the visible part and the key part based on the reconstruction history; Reconstructing a first part model of the key part in the three-dimensional fusion image based on the first part posture relationship; Determine the second distance between the first part model and the danger source from the three-dimensional fusion image; When the attempt fails, extracting features from the visible part based on a preset feature extraction template to obtain multiple part features; Generate a template based on a preset posture relationship condition, and generate a posture relationship condition according to the plurality of part features and a preset action probability table corresponding to the target scene; Determine a second part posture relationship that meets the posture relationship condition from a preset part posture relationship library; Reconstructing a second part model of the key part in the three-dimensional fusion image based on the posture relationship of the second part; The second distance between the second part model and the danger source is determined from the three-dimensional fusion image.
6. A three-dimensional reconstruction and measurement system using multi-eye stereo vision as claimed in claim 5, characterized in that: The standard site conditions include: There is at least one corresponding association relationship between the visible part and the key part in a preset part association relationship library; The positions where the operating actions of the personnel ranked top N in probability in the action probability table should occur match the visible position; N is a positive integer.
7. A three-dimensional reconstruction and measurement system using multi-eye stereo vision as claimed in claim 5, characterized in that: Also includes: The early warning module includes: Obtaining a preset safety distance corresponding to the hazard source type; When the first distance / the second distance is less than or equal to the safety distance, a preset warning message is output.
8. The three-dimensional reconstruction and measurement system using multi-eye stereo vision as claimed in claim 5, characterized in that: Also includes: A tracking module is used to continuously track the third distance between the distance measuring object and the danger source.
Citation Information
Patent Citations
Electric power operation safety management and control system and method based on multi-dimensional information fusion
CN112465401A
Power equipment detection, distance measurement and early warning method based on artificial intelligence and monocular vision
CN115439741A
Power transmission line multi-dimensional environment monitoring method for three-dimensional distance measuring device
CN117671603A
Power transmission line hazard source intelligent identification method based on three-dimensional reconstruction
CN117809297A
Substation safety early warning method based on target three-dimensional distance measurement
CN117875719A
