Multi-view vision obstacle avoidance method and system using image sensor

Abstract

The application is suitable for the technical field of multi-vision obstacle avoidance, and provides a multi-vision obstacle avoidance method and system applying an image sensor. The application determines a target obstacle and records relevant position data by performing multi-vision monitoring and shooting of an environment; plans an obstacle avoidance selection area, selects multiple obstacle avoidance positions; calculates the obstacle avoidance expected value of the multiple obstacle avoidance positions, and judges whether the obstacle avoidance condition is met; when the obstacle avoidance condition is met, a final obstacle avoidance position is determined, and obstacle avoidance movement control is performed; when the obstacle avoidance condition is not met, the selection area is expanded, and the multiple obstacle avoidance positions are reselected. The obstacle avoidance selection area can be planned, the multiple obstacle avoidance positions can be selected, the corresponding obstacle avoidance expected value can be calculated, and then it is compared and judged whether the obstacle avoidance condition is met. When the obstacle avoidance condition is not met, the selection area is expanded, and the multiple obstacle avoidance positions are reselected. The selected final obstacle avoidance position can realize obstacle avoidance while having a smaller impact on the whole transportation process.

Description

Technical Field

[0001] This invention belongs to the field of multi-view visual obstacle avoidance technology, and particularly relates to a multi-view visual obstacle avoidance method and system using image sensors. Background Technology

[0002] Multi-view visual obstacle avoidance is an obstacle avoidance technology that uses two or more cameras to perform stereo perception and spatial modeling of the surrounding environment, thereby achieving automatic obstacle detection, distance calculation and path avoidance. It can acquire image data from different perspectives and use visual algorithms such as image matching, 3D reconstruction and depth estimation to generate depth maps or point cloud models of the environment to identify the position, shape and motion state of obstacles.

[0003] Compared to monocular vision, multi-view obstacle avoidance has advantages such as high depth perception accuracy, strong environmental adaptability, and comprehensive spatial understanding. It can accurately identify and avoid obstacles in complex environments and is widely used in fields such as autonomous driving, drone flight, and robot navigation. It is one of the core technologies for achieving high-precision environmental perception and autonomous decision-making.

[0004] Among multi-view obstacle avoidance, binocular visual obstacle avoidance has the widest application.

[0005] In existing technologies, binocular vision obstacle avoidance mainly focuses on the accurate identification and distance measurement of obstacles to achieve obstacle detection and localization. However, there is a lack of targeted technical solutions for the selection of obstacle avoidance positions. Usually, after completing the identification and localization of obstacles, a temporary position that can avoid obstacles is selected through simple geometric offset or local path adjustment, without comprehensively considering the influence of obstacle position and destination position. Although obstacle avoidance can be achieved, it is easy to cause a large deviation from the overall transportation path, significantly increasing the travel distance or time, which has a significant impact on the entire transportation process. Summary of the Invention

[0006] The purpose of this invention is to provide a multi-view visual obstacle avoidance method and system using image sensors, aiming to solve the technical problems existing in the prior art mentioned in the background.

[0007] The embodiments of the present invention are implemented as follows:

[0008] A multi-view visual obstacle avoidance method using an image sensor, the method specifically includes the following steps:

[0009] During the movement, multi-view monitoring and filming of the environment are carried out to acquire multi-view monitoring data, and the multi-view monitoring data is identified to determine target obstacles and record relevant location data.

[0010] Perform obstacle avoidance analysis on the relevant location data, plan the obstacle avoidance selection area, and select multiple potential obstacle avoidance locations from the obstacle avoidance selection area;

[0011] Based on the relevant location data, the expected obstacle avoidance value of multiple proposed obstacle avoidance locations is calculated and compared with a preset threshold to determine whether the obstacle avoidance conditions are met.

[0012] When the obstacle avoidance conditions are met, the final obstacle avoidance position is determined from the multiple proposed obstacle avoidance positions, and obstacle avoidance movement control is performed.

[0013] If the obstacle avoidance conditions are not met, the obstacle avoidance selection area is expanded, the expanded selection area is determined, and multiple proposed obstacle avoidance locations are reselected.

[0014] As a further limitation of the technical solution of this embodiment of the invention, the step of performing multi-view monitoring and filming of the environment during movement, acquiring multi-view monitoring data, identifying the target obstacle from the multi-view monitoring data, and recording relevant location data specifically includes the following steps:

[0015] During the movement, multi-view monitoring and photography of the environment are carried out to acquire multi-view monitoring data;

[0016] Import obstacle feature data;

[0017] Based on the obstacle feature data, the multi-view monitoring data is identified to determine the target obstacle;

[0018] The multi-view monitoring data is used for location analysis, and the relevant location data is recorded.

[0019] As a further limitation of the technical solution of this embodiment of the invention, the step of performing positioning analysis on the multi-view monitoring data and recording relevant location data specifically includes the following steps:

[0020] The multi-view monitoring data is used to perform obstacle localization analysis to determine the obstacle distance and direction of the target obstacle;

[0021] Import location feature data;

[0022] Based on the location feature data, the multi-view monitoring data is identified to determine multiple location feature entities and the distances between multiple entities are determined.

[0023] Obtain the entity positions of multiple of the aforementioned positioning feature entities;

[0024] Based on the multiple entity locations and multiple entity distances, perform current location analysis to determine the current location;

[0025] The location of the target obstacle is determined based on the current location, the obstacle distance, and the obstacle direction.

[0026] Get the destination location;

[0027] The current location, the obstacle location, and the destination location are organized and the relevant location data are recorded.

[0028] As a further limitation of the technical solution of this embodiment of the invention, the step of performing obstacle avoidance analysis on the relevant location data, planning the obstacle avoidance selection area, and selecting multiple proposed obstacle avoidance locations from the obstacle avoidance selection area specifically includes the following steps:

[0029] Determine the initial selection distance;

[0030] Based on the current location and the initial selection distance, plan the obstacle avoidance selection area;

[0031] Obtain the obstacle avoidance area map of the obstacle avoidance selection area;

[0032] Based on the obstacle avoidance area map, select multiple potential obstacle avoidance locations.

[0033] As a further limitation of the technical solution of this embodiment of the invention, the step of calculating the expected obstacle avoidance value of multiple proposed obstacle avoidance locations based on the relevant location data, and comparing it with a preset threshold to determine whether the obstacle avoidance conditions are met specifically includes the following steps:

[0034] Based on the relevant location data, calculate the obstacle distance and destination distance corresponding to multiple proposed obstacle avoidance locations;

[0035] Based on the multiple obstacle distances and multiple endpoint distances, an obstacle avoidance benefit analysis of potential energy impact is performed to calculate the expected obstacle avoidance value of multiple proposed obstacle avoidance positions;

[0036] The expected values ​​of obstacle avoidance are compared among the various values, and the highest expected value is selected.

[0037] The maximum expected value is compared with a preset threshold.

[0038] When the maximum expected value is greater than a preset threshold, it is determined that the obstacle avoidance condition is met;

[0039] If the maximum expected value is less than or equal to a preset threshold, the obstacle avoidance condition is determined not to be met.

[0040] As a further limitation of the technical solution of this embodiment of the invention, the calculation formula for the expected obstacle avoidance value of the plurality of proposed obstacle avoidance positions is as follows:

[0041] ;

[0042] ;

[0043] in, Representing the The expected value of obstacle avoidance at each proposed obstacle avoidance position. As a preset instant reward, The preset discount factor, As the preset maximum expected value, The preset attraction gain, Representing the The endpoint distance of each simulated obstacle avoidance position. The preset repulsive gain, Representing the The distance to the obstacle at the proposed obstacle avoidance position. This is the preset maximum distance at which an obstacle can have an impact.

[0044] As a further limitation of the technical solution of this invention embodiment, the step of determining the final obstacle avoidance position from the plurality of proposed obstacle avoidance positions and performing obstacle avoidance movement control specifically includes the following steps:

[0045] From the plurality of proposed obstacle avoidance positions, determine the final obstacle avoidance position corresponding to the maximum expected value;

[0046] Based on the current location and the final obstacle avoidance location, plan an obstacle avoidance movement route;

[0047] Obstacle avoidance movement control is performed according to the described obstacle avoidance movement route.

[0048] As a further limitation of the technical solution of this embodiment of the invention, the step of expanding the obstacle avoidance selection area, determining the expanded selection area, and reselecting multiple proposed obstacle avoidance locations specifically includes the following steps:

[0049] Determine the enlargement length;

[0050] The expansion distance is determined based on the expansion length and the initial selection distance;

[0051] The expanded area is determined based on the current location and the expanded distance;

[0052] From the expanded area, the obstacle avoidance selection area is hidden to obtain an expanded selection area;

[0053] Obtain the expanded area map of the expanded selection area;

[0054] Based on the expanded area map, multiple proposed obstacle avoidance locations were reselected.

[0055] A multi-view visual obstacle avoidance system utilizing image sensors, comprising a multi-view monitoring and imaging module, a region planning and processing module, a prediction calculation and comparison module, an obstacle avoidance movement control module, and a region expansion planning module, wherein:

[0056] The multi-view monitoring and shooting module is used to perform multi-view monitoring and shooting of the environment during movement, acquire multi-view monitoring data, identify the target obstacles from the multi-view monitoring data, and record the relevant location data.

[0057] The regional planning and processing module is used to perform obstacle avoidance analysis on the relevant location data, plan the obstacle avoidance selection area, and select multiple proposed obstacle avoidance locations from the obstacle avoidance selection area;

[0058] The expected calculation and comparison module is used to calculate the expected value of obstacle avoidance for multiple proposed obstacle avoidance locations based on the relevant location data, and compare it with a preset threshold to determine whether the obstacle avoidance conditions are met.

[0059] The obstacle avoidance movement control module is used to determine the final obstacle avoidance position from multiple proposed obstacle avoidance positions and perform obstacle avoidance movement control when the obstacle avoidance conditions are met.

[0060] The area expansion planning module is used to expand the obstacle avoidance selection area when the obstacle avoidance conditions are not met, determine the expanded selection area, and reselect multiple proposed obstacle avoidance locations.

[0061] As a further limitation of the technical solution of this embodiment of the invention, the multi-view monitoring and shooting module specifically includes:

[0062] Image sensors are used to capture multi-view images of the environment while in motion, acquiring multi-view monitoring data.

[0063] The data import unit is used to import obstacle feature data;

[0064] An obstacle recognition unit is used to identify the target obstacle based on the obstacle feature data and the multi-view monitoring data.

[0065] The positioning analysis unit is used to perform positioning analysis on the multi-view monitoring data and record relevant location data.

[0066] Compared with the prior art, the beneficial effects of the present invention are:

[0067] This invention identifies target obstacles and records their location data through multi-view environmental monitoring and photography; it plans an obstacle avoidance selection area and selects multiple potential obstacle avoidance locations; it calculates the expected obstacle avoidance value of these locations and determines whether the obstacle avoidance conditions are met; if the conditions are met, it determines the final obstacle avoidance location and performs obstacle avoidance movement control; if the conditions are not met, it expands the selection area and reselects multiple potential obstacle avoidance locations. This method allows for the planning of the obstacle avoidance selection area, the selection of multiple potential obstacle avoidance locations, the calculation of their corresponding expected obstacle avoidance values, and the comparison of these values ​​to determine if the obstacle avoidance conditions are met. Furthermore, it expands the selection area and reselects multiple potential obstacle avoidance locations when the conditions are not met, ensuring that the selected final obstacle avoidance location has minimal impact on the overall transportation process while achieving obstacle avoidance. Attached Figure Description

[0068] Figure 1 A flowchart of a multi-view visual obstacle avoidance method using an image sensor, provided in an embodiment of the present invention, is shown.

[0069] Figure 2 The following is an application architecture diagram of the multi-view visual obstacle avoidance system using an image sensor provided in an embodiment of the present invention. Detailed Implementation

[0070] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0071] Understandably, binocular visual obstacle avoidance has the widest application among multi-view obstacle avoidance technologies. Current binocular visual obstacle avoidance mainly focuses on the accurate identification and distance measurement of obstacles to achieve obstacle detection and localization. However, it lacks targeted technical solutions for selecting obstacle avoidance locations. Typically, after obstacle identification and localization, a temporary location is chosen to bypass the obstacle through simple geometric offsets or local path adjustments, without comprehensively considering the impact of both the obstacle location and the destination location. While obstacle avoidance can be achieved, it easily leads to significant deviations from the overall transportation path, greatly increasing travel distance or time, and significantly impacting the entire transportation process.

[0072] To address the aforementioned problems, the multi-view visual obstacle avoidance method and system using image sensors disclosed in this invention involves: acquiring multi-view monitoring data during movement by capturing images of the environment; identifying target obstacles based on the multi-view monitoring data and recording relevant location data; performing obstacle avoidance analysis on the relevant location data to plan an obstacle avoidance selection area and selecting multiple potential obstacle avoidance locations from the selection area; calculating the expected obstacle avoidance value of the multiple potential obstacle avoidance locations based on the relevant location data and comparing it with a preset threshold to determine whether the obstacle avoidance conditions are met; if the obstacle avoidance conditions are met, determining the final obstacle avoidance location from the multiple potential obstacle avoidance locations and performing obstacle avoidance movement control; if the obstacle avoidance conditions are not met, expanding the obstacle avoidance selection area to determine an expanded selection area and reselecting multiple potential obstacle avoidance locations. It can plan the obstacle avoidance selection area, select multiple potential obstacle avoidance locations, calculate the corresponding expected obstacle avoidance value, and then compare and judge whether the obstacle avoidance conditions are met. If the obstacle avoidance conditions are not met, the selection area is expanded and multiple potential obstacle avoidance locations are reselected. It can achieve obstacle avoidance while ensuring that the selected final obstacle avoidance location has a small impact on the entire transportation process.

[0073] Specifically, Figure 1 A flowchart of a multi-view visual obstacle avoidance method using an image sensor, as provided in an embodiment of the present invention, is shown.

[0074] In a preferred embodiment of the present invention, a multi-view visual obstacle avoidance method using an image sensor specifically includes the following steps:

[0075] Step S101: During the movement, perform multi-view monitoring and shooting of the environment, acquire multi-view monitoring data, identify the target obstacle by the multi-view monitoring data, and record the relevant location data.

[0076] In this embodiment of the invention, during the movement of the main body, multi-view monitoring and filming of the environment are performed to acquire multi-view monitoring data, and obstacle feature data is imported. Using the obstacle feature data as a standard, feature identification and matching analysis are performed on the multi-view monitoring data to determine whether there are obstacles on the main body's movement path. If obstacles are determined to be on the main body's movement path, the target obstacle is identified. Then, obstacle positioning analysis is performed on the multi-view monitoring data to determine the obstacle distance and obstacle direction of the target obstacle. At the same time, positioning feature data is imported, and using the positioning feature data as a standard, the multi-view monitoring data is identified to determine multiple positioning feature entities around the main body, and the entity distances of multiple positioning feature entities are determined. The entity positions of multiple positioning feature entities are extracted from the positioning feature data, and then the main body can be positioned by combining the multiple entity positions and multiple entity distances to determine the current positioning position of the main body. Then, based on the current positioning position, obstacle distance, and obstacle direction, the target obstacle can be positioned to determine the obstacle position of the target obstacle. At the same time, the endpoint position is obtained, and the current positioning position, obstacle position, and endpoint position are organized to obtain relevant position data.

[0077] It is understood that, in the embodiments of the present invention, multi-view monitoring and shooting can be binocular monitoring and shooting, and the acquired multi-view monitoring data is binocular monitoring data.

[0078] It is understandable that multiple location feature entities are physical objects used to assist visual positioning. Their positions are fixed and they have special feature markers. The location feature data not only records the feature data of the location feature entities, but also records the position data of the location feature entities.

[0079] Specifically, in another preferred embodiment provided by the present invention, the step of performing multi-view environmental monitoring and photography during movement, acquiring multi-view monitoring data, identifying the multi-view monitoring data, determining target obstacles, and recording relevant location data specifically includes the following steps:

[0080] During the movement, multi-view monitoring and photography of the environment are carried out to acquire multi-view monitoring data;

[0081] Import obstacle feature data;

[0082] Based on the obstacle feature data, the multi-view monitoring data is identified to determine the target obstacle;

[0083] The multi-view monitoring data is used for location analysis, and the relevant location data is recorded.

[0084] Specifically, in another preferred embodiment provided by the present invention, the step of performing location analysis on the multi-view monitoring data and recording relevant location data specifically includes the following steps:

[0085] The multi-view monitoring data is used to perform obstacle localization analysis to determine the obstacle distance and direction of the target obstacle;

[0086] Import location feature data;

[0087] Based on the location feature data, the multi-view monitoring data is identified to determine multiple location feature entities and the distances between multiple entities are determined.

[0088] Obtain the entity positions of multiple of the aforementioned positioning feature entities;

[0089] Based on the multiple entity locations and multiple entity distances, perform current location analysis to determine the current location;

[0090] The location of the target obstacle is determined based on the current location, the obstacle distance, and the obstacle direction.

[0091] Get the destination location;

[0092] The current location, the obstacle location, and the destination location are organized and the relevant location data are recorded.

[0093] Furthermore, the multi-view visual obstacle avoidance method using an image sensor also includes the following steps:

[0094] Step S102: Perform obstacle avoidance analysis on the relevant location data, plan the obstacle avoidance selection area, and select multiple proposed obstacle avoidance locations from the obstacle avoidance selection area.

[0095] In this embodiment of the invention, an initial selection distance is determined, and then, according to the current location and the initial selection distance, a corresponding obstacle avoidance selection area is planned around the body. Then, an obstacle avoidance area map corresponding to the obstacle avoidance selection area is obtained. By performing distribution analysis on the obstacle avoidance area map, multiple proposed obstacle avoidance locations are selected in the obstacle avoidance selection area.

[0096] Understandably, by analyzing the distribution of obstacle avoidance area maps, we can identify open areas within the obstacle avoidance selection area, and then select multiple potential obstacle avoidance locations from these open areas.

[0097] Specifically, in another preferred embodiment provided by the present invention, the step of performing obstacle avoidance analysis on the relevant location data, planning the obstacle avoidance selection area, and selecting multiple proposed obstacle avoidance locations from the obstacle avoidance selection area specifically includes the following steps:

[0098] Determine the initial selection distance;

[0099] Based on the current location and the initial selection distance, plan the obstacle avoidance selection area;

[0100] Obtain the obstacle avoidance area map of the obstacle avoidance selection area;

[0101] Based on the obstacle avoidance area map, select multiple potential obstacle avoidance locations.

[0102] Furthermore, the multi-view visual obstacle avoidance method using an image sensor also includes the following steps:

[0103] Step S103: Based on the relevant location data, calculate the expected obstacle avoidance value of multiple proposed obstacle avoidance locations, and compare it with a preset threshold to determine whether the obstacle avoidance conditions are met.

[0104] In this embodiment of the invention, based on relevant location data, the obstacle distances between multiple proposed obstacle avoidance positions and the target obstacle are calculated, and the endpoint distances between the multiple proposed obstacle avoidance positions and the endpoint position are also calculated. Based on the multiple obstacle distances and multiple endpoint distances, an obstacle avoidance benefit analysis is performed considering the potential energy influence of the repulsion between the obstacle and the target obstacle and the attraction between the obstacle and the endpoint position. The expected obstacle avoidance value of the multiple proposed obstacle avoidance positions is calculated, and then these multiple expected obstacle avoidance values ​​are compared. The maximum expected value is selected from the multiple expected obstacle avoidance values, and then compared with a preset threshold to determine whether the maximum expected value is greater than the preset threshold. If the maximum expected value is greater than the preset threshold, the obstacle avoidance condition is determined to be met; otherwise, if the maximum expected value is less than or equal to the preset threshold, the obstacle avoidance condition is determined not to be met. Specifically, the calculation formula for the expected obstacle avoidance value of the multiple proposed obstacle avoidance positions is as follows:

[0105] ;

[0106] ;

[0107] in, Representing the The expected value of obstacle avoidance at each proposed obstacle avoidance position. As a preset instant reward, The preset discount factor, As the preset maximum expected value, The preset attraction gain, Representing the The endpoint distance of each simulated obstacle avoidance position. The preset repulsive gain, Representing the The distance to the obstacle at the proposed obstacle avoidance position. This is the preset maximum distance at which an obstacle can have an impact.

[0108] Specifically, in another preferred embodiment provided by the present invention, the step of calculating the expected obstacle avoidance value of multiple proposed obstacle avoidance locations based on the relevant location data, and comparing it with a preset threshold to determine whether the obstacle avoidance conditions are met specifically includes the following steps:

[0109] Based on the relevant location data, calculate the obstacle distance and destination distance corresponding to multiple proposed obstacle avoidance locations;

[0110] Based on the multiple obstacle distances and multiple endpoint distances, an obstacle avoidance benefit analysis of potential energy impact is performed to calculate the expected obstacle avoidance value of multiple proposed obstacle avoidance positions;

[0111] The expected values ​​of obstacle avoidance are compared among the various values, and the highest expected value is selected.

[0112] The maximum expected value is compared with a preset threshold.

[0113] When the maximum expected value is greater than a preset threshold, it is determined that the obstacle avoidance condition is met;

[0114] If the maximum expected value is less than or equal to a preset threshold, the obstacle avoidance condition is determined not to be met.

[0115] Furthermore, the multi-view visual obstacle avoidance method using an image sensor also includes the following steps:

[0116] Step S104: When the obstacle avoidance conditions are met, determine the final obstacle avoidance position from the multiple proposed obstacle avoidance positions and perform obstacle avoidance movement control.

[0117] In this embodiment of the invention, when it is determined that the obstacle avoidance conditions are met, the final obstacle avoidance position corresponding to the maximum expected value is determined from multiple proposed obstacle avoidance positions. Then, based on the current positioning position and the final obstacle avoidance position, an obstacle avoidance movement route is planned, and the body is controlled to avoid obstacles according to the obstacle avoidance movement route.

[0118] Specifically, in another preferred embodiment provided by the present invention, determining the final obstacle avoidance position from the plurality of proposed obstacle avoidance positions and performing obstacle avoidance movement control specifically includes the following steps:

[0119] From the plurality of proposed obstacle avoidance positions, determine the final obstacle avoidance position corresponding to the maximum expected value;

[0120] Based on the current location and the final obstacle avoidance location, plan an obstacle avoidance movement route;

[0121] Obstacle avoidance movement control is performed according to the described obstacle avoidance movement route.

[0122] Furthermore, the multi-view visual obstacle avoidance method using an image sensor also includes the following steps:

[0123] Step S105: When the obstacle avoidance conditions are not met, expand the obstacle avoidance selection area, determine the expanded selection area, and reselect multiple proposed obstacle avoidance locations.

[0124] In this embodiment of the invention, when it is determined that the obstacle avoidance conditions are not met, the expansion length is determined, and then the expansion distance is determined based on the expansion length and the initial selection distance. Then, according to the current positioning position and the expansion distance, the corresponding expansion area is determined around the body. After that, the obstacle avoidance selection area is hidden in the expansion area to obtain the expansion selection area, and the expansion area map corresponding to the expansion selection area is obtained. By performing distribution analysis on the expansion area map, multiple proposed obstacle avoidance positions are reselected in the expansion selection area. Then, the expected value of obstacle avoidance is calculated and compared until it is determined that the obstacle avoidance conditions are met.

[0125] Specifically, in another preferred embodiment provided by the present invention, the step of expanding the obstacle avoidance selection area, determining the expanded selection area, and reselecting multiple proposed obstacle avoidance locations specifically includes the following steps:

[0126] Determine the enlargement length;

[0127] The expansion distance is determined based on the expansion length and the initial selection distance;

[0128] The expanded area is determined based on the current location and the expanded distance;

[0129] From the expanded area, the obstacle avoidance selection area is hidden to obtain an expanded selection area;

[0130] Obtain the expanded area map of the expanded selection area;

[0131] Based on the expanded area map, multiple proposed obstacle avoidance locations were reselected.

[0132] Furthermore, Figure 2 The following is an application architecture diagram of the multi-view visual obstacle avoidance system using an image sensor provided in an embodiment of the present invention.

[0133] Specifically, in another preferred embodiment provided by the present invention, a multi-view visual obstacle avoidance system using an image sensor includes:

[0134] The multi-view monitoring and shooting module 101 is used to perform multi-view monitoring and shooting of the environment during movement, acquire multi-view monitoring data, identify the target obstacle from the multi-view monitoring data, and record the relevant location data.

[0135] In this embodiment of the invention, during the movement of the main body, the multi-view monitoring and shooting module 101 performs multi-view monitoring and shooting of the environment to acquire multi-view monitoring data and imports obstacle feature data. Using the obstacle feature data as a standard, the multi-view monitoring data is used for feature identification and matching analysis to determine whether there are obstacles on the main body's movement path. If obstacles are found on the main body's movement path, the target obstacle is identified. Then, obstacle positioning analysis is performed on the multi-view monitoring data to determine the obstacle distance and obstacle direction of the target obstacle. At the same time, positioning feature data is imported and used as a standard to identify multiple positioning feature entities around the main body and determine the entity distance of multiple positioning feature entities. The entity positions of multiple positioning feature entities are extracted from the positioning feature data. Then, by combining the multiple entity positions and multiple entity distances, the main body can be positioned to determine its current positioning position. Then, based on the current positioning position, obstacle distance, and obstacle direction, the target obstacle can be positioned to determine its obstacle position. At the same time, the endpoint position is obtained. The current positioning position, obstacle position, and endpoint position are organized to obtain relevant position data.

[0136] Specifically, in another preferred embodiment provided by the present invention, the multi-view monitoring and imaging module 101 specifically includes:

[0137] Image sensors are used to capture multi-view images of the environment while in motion, acquiring multi-view monitoring data.

[0138] The data import unit is used to import obstacle feature data;

[0139] An obstacle recognition unit is used to identify the target obstacle based on the obstacle feature data and the multi-view monitoring data.

[0140] The positioning analysis unit is used to perform positioning analysis on the multi-view monitoring data and record relevant location data.

[0141] Furthermore, the multi-view visual obstacle avoidance system using image sensors also includes:

[0142] The regional planning and processing module 102 is used to perform obstacle avoidance analysis on the relevant location data, plan the obstacle avoidance selection area, and select multiple proposed obstacle avoidance locations from the obstacle avoidance selection area.

[0143] In this embodiment of the invention, the area planning processing module 102 determines the initial selection distance, and then plans the corresponding obstacle avoidance selection area around the body according to the current positioning position and the initial selection distance. Then, it obtains the obstacle avoidance area map corresponding to the obstacle avoidance selection area, and selects multiple proposed obstacle avoidance positions in the obstacle avoidance selection area by performing distribution analysis on the obstacle avoidance area map.

[0144] The expected calculation and comparison module 103 is used to calculate the expected obstacle avoidance value of multiple proposed obstacle avoidance locations based on the relevant location data, and compare it with a preset threshold to determine whether the obstacle avoidance conditions are met.

[0145] In this embodiment of the invention, the expected calculation and comparison module 103 calculates the obstacle distances between multiple proposed obstacle avoidance positions and the target obstacle based on relevant location data, and also calculates the endpoint distances between the multiple proposed obstacle avoidance positions and the endpoint position. Based on the multiple obstacle distances and multiple endpoint distances, it performs an obstacle avoidance benefit analysis of the potential energy influence between the target obstacle and the endpoint position, calculating the expected obstacle avoidance value of the multiple proposed obstacle avoidance positions. Then, it compares these multiple expected obstacle avoidance values, selects the maximum expected value, and compares it with a preset threshold. It determines whether the maximum expected value is greater than the preset threshold. If the maximum expected value is greater than the preset threshold, the obstacle avoidance condition is satisfied; otherwise, if the maximum expected value is less than or equal to the preset threshold, the obstacle avoidance condition is not satisfied. Specifically, the calculation formula for the expected obstacle avoidance value of the multiple proposed obstacle avoidance positions is as follows:

[0146] ;

[0147] ;

[0148] in, Representing the The expected value of obstacle avoidance at each proposed obstacle avoidance position. As a preset instant reward, The preset discount factor, As the preset maximum expected value, The preset attraction gain, Representing the The endpoint distance of each simulated obstacle avoidance position. The preset repulsive gain, Representing the The distance to the obstacle at the proposed obstacle avoidance position. This is the preset maximum distance at which an obstacle can have an impact.

[0149] The obstacle avoidance movement control module 104 is used to determine the final obstacle avoidance position from a plurality of proposed obstacle avoidance positions and perform obstacle avoidance movement control when the obstacle avoidance conditions are met.

[0150] In this embodiment of the invention, when it is determined that the obstacle avoidance conditions are met, the obstacle avoidance movement control module 104 determines the final obstacle avoidance position corresponding to the maximum expected value from multiple proposed obstacle avoidance positions, and then plans an obstacle avoidance movement route based on the current positioning position and the final obstacle avoidance position, and then performs obstacle avoidance movement control on the body according to the obstacle avoidance movement route.

[0151] The area expansion planning module 105 is used to expand the obstacle avoidance selection area when the obstacle avoidance conditions are not met, determine the expanded selection area, and reselect multiple proposed obstacle avoidance locations.

[0152] In this embodiment of the invention, when it is determined that the obstacle avoidance conditions are not met, the area expansion planning module 105 determines the expansion length, and then determines the expansion distance based on the expansion length and the initial selection distance. Then, according to the current positioning position and the expansion distance, it determines the corresponding expansion area around the body. After that, the obstacle avoidance selection area is hidden in the expansion area to obtain the expansion selection area, and the expansion area map corresponding to the expansion selection area is obtained. By performing distribution analysis on the expansion area map, multiple proposed obstacle avoidance positions are reselected in the expansion selection area. Then, the expected value of obstacle avoidance is calculated and compared until it is determined that the obstacle avoidance conditions are met.

[0153] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A multi-view visual obstacle avoidance method using an image sensor, characterized in that, The method specifically includes the following steps: During the movement, multi-view monitoring and filming of the environment are carried out to acquire multi-view monitoring data, and the multi-view monitoring data is identified to determine target obstacles and record relevant location data. Perform obstacle avoidance analysis on the relevant location data, plan the obstacle avoidance selection area, and select multiple potential obstacle avoidance locations from the obstacle avoidance selection area; Based on the relevant location data, the expected obstacle avoidance value of multiple proposed obstacle avoidance locations is calculated and compared with a preset threshold to determine whether the obstacle avoidance conditions are met. When the obstacle avoidance conditions are met, the final obstacle avoidance position is determined from the multiple proposed obstacle avoidance positions, and obstacle avoidance movement control is performed. When the obstacle avoidance conditions are not met, the obstacle avoidance selection area is expanded, the expanded selection area is determined, and multiple proposed obstacle avoidance locations are reselected. The step of calculating the expected obstacle avoidance value of multiple proposed obstacle avoidance locations based on the relevant location data and comparing it with a preset threshold to determine whether the obstacle avoidance conditions are met specifically includes the following steps: Based on the relevant location data, calculate the obstacle distance and destination distance corresponding to multiple proposed obstacle avoidance locations; Based on the multiple obstacle distances and multiple endpoint distances, an obstacle avoidance benefit analysis of potential energy impact is performed to calculate the expected obstacle avoidance value of multiple proposed obstacle avoidance positions; The expected values ​​of obstacle avoidance are compared among the various values, and the highest expected value is selected. The maximum expected value is compared with a preset threshold. When the maximum expected value is greater than a preset threshold, it is determined that the obstacle avoidance condition is met; When the maximum expected value is less than or equal to a preset threshold, it is determined that the obstacle avoidance condition is not met. The formula for calculating the expected obstacle avoidance value of the multiple proposed obstacle avoidance locations is as follows: ； ； in, Representing the The expected value of obstacle avoidance at each proposed obstacle avoidance position. As a preset instant reward, The preset discount factor, As the preset maximum expected value, The preset attraction gain, Representing the The endpoint distance of each simulated obstacle avoidance position. The preset repulsive gain, Representing the The distance to the obstacle at the proposed obstacle avoidance position. This is the preset maximum distance at which an obstacle can have an impact.

2. The multi-view visual obstacle avoidance method using an image sensor according to claim 1, characterized in that, The process of performing multi-view environmental monitoring and filming during movement, acquiring multi-view monitoring data, identifying target obstacles from the multi-view monitoring data, and recording relevant location data specifically includes the following steps: During the movement, multi-view monitoring and photography of the environment are carried out to acquire multi-view monitoring data; Import obstacle feature data; Based on the obstacle feature data, the multi-view monitoring data is identified to determine the target obstacle; The multi-view monitoring data is used for location analysis, and the relevant location data is recorded.

3. The multi-view visual obstacle avoidance method using an image sensor according to claim 2, characterized in that, The process of performing location analysis on the multi-view monitoring data and recording relevant location data specifically includes the following steps: The multi-view monitoring data is used to perform obstacle localization analysis to determine the obstacle distance and direction of the target obstacle; Import location feature data; Based on the location feature data, the multi-view monitoring data is identified to determine multiple location feature entities and the distances between multiple entities are determined. Obtain the entity positions of multiple of the aforementioned positioning feature entities; Based on the multiple entity locations and multiple entity distances, perform current location analysis to determine the current location; The location of the target obstacle is determined based on the current location, the obstacle distance, and the obstacle direction. Get the destination location; The current location, the obstacle location, and the destination location are organized and the relevant location data are recorded.

4. The multi-view visual obstacle avoidance method using an image sensor according to claim 3, characterized in that, The steps of performing obstacle avoidance analysis on the relevant location data, planning the obstacle avoidance selection area, and selecting multiple potential obstacle avoidance locations from the obstacle avoidance selection area specifically include the following steps: Determine the initial selection distance; Based on the current location and the initial selection distance, plan the obstacle avoidance selection area; Obtain the obstacle avoidance area map of the obstacle avoidance selection area; Based on the obstacle avoidance area map, select multiple potential obstacle avoidance locations.

5. The multi-view visual obstacle avoidance method using an image sensor according to claim 3, characterized in that, Determining the final obstacle avoidance position from multiple proposed obstacle avoidance positions and performing obstacle avoidance movement control specifically includes the following steps: From the plurality of proposed obstacle avoidance positions, determine the final obstacle avoidance position corresponding to the maximum expected value; Based on the current location and the final obstacle avoidance location, plan an obstacle avoidance movement route; Obstacle avoidance movement control is performed according to the described obstacle avoidance movement route.

6. The multi-view visual obstacle avoidance method using an image sensor according to claim 4, characterized in that, The process of expanding the obstacle avoidance selection area, determining the expanded selection area, and reselecting multiple proposed obstacle avoidance locations specifically includes the following steps: Determine the enlargement length; The expansion distance is determined based on the expansion length and the initial selection distance; The expanded area is determined based on the current location and the expanded distance; From the expanded area, the obstacle avoidance selection area is hidden to obtain an expanded selection area; Obtain the expanded area map of the expanded selection area; Based on the expanded area map, multiple proposed obstacle avoidance locations were reselected.

7. A multi-view visual obstacle avoidance system using an image sensor, characterized in that, The system includes a multi-view monitoring and shooting module, a region planning and processing module, a prediction calculation and comparison module, an obstacle avoidance and movement control module, and a region expansion planning module, wherein: The multi-view monitoring and shooting module is used to perform multi-view monitoring and shooting of the environment during movement, acquire multi-view monitoring data, identify the target obstacles from the multi-view monitoring data, and record the relevant location data. The regional planning and processing module is used to perform obstacle avoidance analysis on the relevant location data, plan the obstacle avoidance selection area, and select multiple proposed obstacle avoidance locations from the obstacle avoidance selection area; The expected calculation and comparison module is used to calculate the expected value of obstacle avoidance for multiple proposed obstacle avoidance locations based on the relevant location data, and compare it with a preset threshold to determine whether the obstacle avoidance conditions are met. The obstacle avoidance movement control module is used to determine the final obstacle avoidance position from multiple proposed obstacle avoidance positions and perform obstacle avoidance movement control when the obstacle avoidance conditions are met. The area expansion planning module is used to expand the obstacle avoidance selection area when the obstacle avoidance conditions are not met, determine the expanded selection area, and reselect multiple proposed obstacle avoidance locations. The step of calculating the expected obstacle avoidance value for multiple proposed obstacle avoidance locations based on the relevant location data and comparing it with a preset threshold to determine whether the obstacle avoidance condition is met specifically involves: Based on the relevant location data, calculate the obstacle distance and destination distance corresponding to multiple proposed obstacle avoidance locations; Based on the multiple obstacle distances and multiple endpoint distances, an obstacle avoidance benefit analysis of potential energy impact is performed to calculate the expected obstacle avoidance value of multiple proposed obstacle avoidance positions; The expected values ​​of obstacle avoidance are compared among the various values, and the highest expected value is selected. The maximum expected value is compared with a preset threshold. When the maximum expected value is greater than a preset threshold, it is determined that the obstacle avoidance condition is met; When the maximum expected value is less than or equal to a preset threshold, it is determined that the obstacle avoidance condition is not met. The formula for calculating the expected obstacle avoidance value of the multiple proposed obstacle avoidance locations is as follows: ； ； in, Representing the The expected value of obstacle avoidance at each proposed obstacle avoidance position. As a preset instant reward, The preset discount factor, As the preset maximum expected value, The preset attraction gain, Representing the The endpoint distance of each simulated obstacle avoidance position. The preset repulsive gain, Representing the The distance to the obstacle at the proposed obstacle avoidance position. This is the preset maximum distance at which an obstacle can have an impact.

8. The multi-view visual obstacle avoidance system using an image sensor according to claim 7, characterized in that, The multi-view monitoring and imaging module specifically includes: Image sensors are used to capture multi-view images of the environment while in motion, acquiring multi-view monitoring data. The data import unit is used to import obstacle feature data; An obstacle recognition unit is used to identify the target obstacle based on the obstacle feature data and the multi-view monitoring data. The positioning analysis unit is used to perform positioning analysis on the multi-view monitoring data and record relevant location data.

Images