Indoor positioning and object searching method for intelligent unmanned vehicle system

Abstract

An indoor positioning and object search method for an intelligent unmanned vehicle system is used to search for an object in an indoor area. First, a model of the indoor environment is established to locate the vehicle body. Then, an image input module captures real-time environmental images to generate environmental image information. Then, an image feature recognition module receives the environmental image information to generate feature recognition information. Then, based on the environmental image information and the feature recognition information, a region of interest (ROI) corresponding to the object in the area is established, and it is determined whether the region of interest has an image of the object that matches an object image data stored in a database. If so, the relative coordinate position and distance between the object and the vehicle body are measured and calculated.

Description

Technical Field

[0001] The present invention relates to an indoor positioning and object searching method for an intelligent unmanned vehicle system, and in particular to an object searching method that can perform positioning in an indoor area and establish an area of ​​interest for searching in the indoor area. Background Art

[0002] In the past, drones could be controlled and moved within the user's visual range through remote control. Since drones can be equipped with camera lenses, they can directly shoot the ground from a bird's-eye view in an open space outdoors. When ordinary people lose items in an area, they can use drones to search for them in the area in order to retrieve the lost items.

[0003] However, in general indoor environments, it is impossible to search through the above methods, mainly due to the influence of the indoor space structure corners or furniture layout, resulting in multiple blind spots in the user's operating field of view. If the field of view is judged directly through the drone's camera lens, it will also be limited by the camera lens's limited shooting angle and cannot easily avoid obstacles in the indoor environment such as beams or furniture.

[0004] Therefore, when an average person loses an item in an indoor environment, in order to find the lost item back, he or she can only try to recall where the lost item was last used or seen, or go back to a place he or she has been to. However, such a way of finding the item is often unreliable. For example, a pair of glasses placed on a windowsill is thrown away casually, and there are too many items around that block or confuse the vision, which will cause a blind spot in the visual field search, making it very difficult to find the lost item.

[0005] On the other hand, if the search is not just for an item but for a person, such as a child who is lost in a hypermarket, if the child is focused on a product in the store and does not show any emotional tension, and the child is small and obscured by the store's display, it will also cause difficulty in finding the child. Or, when the security guards are patrolling the store at night, if small animals such as cats, dogs or thieves invade and use the store's display as a hiding place, as long as they do not make any noise, they will not be easily discovered, which will also create blind spots for the security guards while they are patrolling.

[0006] Therefore, how to provide an indoor positioning system in an indoor environment, so that the drone can move between various corners while avoiding various obstacles in the environment, and how to effectively distinguish the object to be found from the surrounding furnishings in the indoor regional environment to increase the efficiency of identifying objects, people or creatures in the regional environment will be the topics, issues and focuses to be studied in this case. Summary of the invention

[0007] The purpose of the present invention is to provide an indoor positioning and object searching method for an intelligent unmanned vehicle system which can be used for indoor positioning and can avoid obstacles and move in the indoor environment.

[0008] Another object of the present invention is to provide an indoor positioning and object searching method for an intelligent unmanned vehicle system that can effectively distinguish images of a sought-after object from images of surrounding objects in a regional environment.

[0009] The present invention is an indoor positioning and object search method for an intelligent unmanned vehicle system, which is used for positioning in an indoor area and searching for an object in the indoor area. The intelligent unmanned vehicle system includes a vehicle body, an image input module, an image feature recognition module, a central processing unit, an image initial marking module, a database and a vehicle motion sensing module. The central processing unit is respectively linked to the image input module, the image feature recognition module, the image initial marking module, the database and the vehicle motion sensing module by signal, and is configured in the vehicle body with the modules. The indoor positioning and object search method includes the following steps:

[0010] a) the carrier body is controlled by the central processor to move in the indoor area, and the image initial marking module analyzes the environmental image of the indoor area where the wearing body is located captured by the image input module, identifies the original structure of the indoor area and the multiple original ornaments set in the indoor area, establishes an identification feature model corresponding to the indoor environment and marks it, and stores it in the database;

[0011] b) the vehicle motion sensing module detects the motion orientation of the vehicle body and generates reference motion orientation information, and the image input module captures the environmental image of the indoor area to generate environmental image information corresponding to the indoor area and obtain spatial relative relationship information;

[0012] c) the image feature recognition module receives the environmental image information, and performs feature recognition on the environmental image information to generate feature recognition information;

[0013] d) the image processing module establishes a region of interest corresponding to the object in the indoor area according to the environmental image information and the feature recognition information, and compares the region of interest in the environmental image information to determine whether there is an image of the object that matches an object image data stored in the database; and

[0014] e) If it is true, measure and calculate the relative distance and coordinate position between the object in the region of interest and the carrier body.

[0015] The indoor positioning and object search method for the intelligent unmanned vehicle system is to pre-position in the indoor area environment and identify various obstacles in the environment, so that the vehicle body can move between various corners and avoid various obstacles in the environment.

[0016] In the established region of interest, objects in the region of interest of the environmental image information are compared to determine whether there are images of objects, people or creatures that match those stored in the database, effectively distinguishing the images of the objects to be found from the images of surrounding furnishings, increasing the efficiency of identifying objects, people or creatures in the regional environment, and achieving the above-mentioned purpose. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is a schematic diagram of an indoor positioning and object searching method for an intelligent unmanned vehicle system according to an embodiment of the present invention;

[0018] Figure 2 yes Figure 1 System block diagram of intelligent unmanned vehicle;

[0019] Figure 3 yes Figure 1 A flowchart of an indoor positioning and object searching method; and

[0020] Figure 4 yes Figure 1 Schematic diagram of an intelligent unmanned vehicle system measuring and calculating the distance between an object and the vehicle body.

[0021] Description of reference numerals:

[0022] 1. Indoor area 10. Vehicle body

[0023] 11. Image input module 12. Image feature recognition module

[0024] 13. Image processing module 14. Ultrasonic ranging module

[0025] 15. Central Processing Unit 16. Coordinate Transformation Module

[0026] 17. Initial parameter setting module 18. Image initial marking module

[0027] 19. Immediate addressing module 20. Database

[0028] 21. Vehicle motion sensing modules 401-409 ... steps DETAILED DESCRIPTION

[0029] According to an embodiment of the present invention, a method for indoor positioning and object searching for an intelligent unmanned vehicle system is provided. Figure 1 As shown, it can be used to locate in an indoor area 1 and to search for an object in the indoor area 1, and refer to Figure 1 and Figure 2 As shown, the intelligent unmanned vehicle system includes a vehicle body 10, an image input module 11, an image feature recognition module 12, an image processing module 13, an ultrasonic ranging module 14, a central processing unit 15, a coordinate conversion module 16, an initial parameter setting module 17, an image initial marking module 18, an immediate addressing module 19, a database 20 and a vehicle motion sensing module 21. The central processing unit 15 is signal-linked with the image input module 11, the image feature recognition module 12, the image processing module 13, the ultrasonic ranging module 14, the coordinate conversion module 16, the initial parameter setting module 17, the image initial marking module 18, the immediate addressing module 19, the database 20 and the vehicle motion sensing module 21, and is configured with these modules in the vehicle body 10. In this example, the mobile vehicle body 10 takes a flying drone as an example. For those skilled in the art, it can also be easily replaced by taking a ground mobile vehicle (such as a sweeping robot) as an example.

[0030] This case is used for the object search method of the intelligent unmanned vehicle system, and is also referenced Figure 1 , Figure 2 and Figure 3As shown, at the beginning, as in step 401, each parameter of the carrier body 10 is initialized, measured and calibrated, and the coordinate origin is initialized through the initial parameter setting module 17, for example, the electrical parameters, image sensing parameters, light source parameters, ultrasonic parameters, etc. of the carrier body 21 are initialized and the distance measurement accuracy is calibrated, and the coordinate origin is initialized, or the number of rooms and the area of ​​the indoor environment 10 are further set. As in step 402, the carrier body 10 is controlled by the central processor 15 to move in the indoor area 1, and the image initial marking module 18 analyzes the environmental image of the indoor area 1 where the wearing body 10 is located captured by the image input module 11, and the image feature recognition module 12 recognizes the original structure in the indoor area 11 and the multiple structures set in the indoor area 1. An original ornament is used to establish an identification feature model corresponding to the indoor environment 1, and the model is stored in the database. The feature recognition performed by the image feature recognition module 12 in this example is based on visual image analysis to identify key features in the environment, such as points (important indoor features such as corners, fixed feature points such as windows, etc.), lines (indoor features such as straight lines on walls, straight lines on indoor beams and columns, etc.), and surfaces (main indoor walls, corner walls of rooms, etc.) to obtain a 3D model of the regional environment. The image input module 11 uses real-time image input and can automatically adjust the ambient light source. In addition, when strong outdoor light directly hits the lens, it may cause the carrier to be blinded, and use image processing and computer vision technology to avoid backlighting or other factors that may cause the carrier to be instantly lost.

[0031] As another example, in step 403, the image processing module 13 calculates the navigation position of the vehicle body 10 based on the reference motion orientation information detected by the vehicle motion sensing module 21, and provides the central processing unit 15 with a corrected navigation command. As another example, in step 404, the ultrasonic ranging module 14 measures the distance around the vehicle body 10 to generate an ultrasonic ranging information for the central processing unit 15 to control the vehicle body to measure distance and avoid obstacles. Then, as another example, in step 405, the vehicle motion sensing module 21 detects the motion position of the vehicle body 10 and generates a reference motion orientation information, and the image input module 11 captures the environmental image of the indoor area 1 to generate an environmental image information corresponding to the indoor area 1 and obtain spatial relative relationship information. As another example, in step 406, the coordinate conversion module 16 calculates the coordinate conversion calculation of the plane (2D) and three-dimensional (3D) space of the vehicle body 10 relative to the indoor environment 1 based on the environmental image information to generate a relative position coordinate information.

[0032] As another example, in step 407, the immediate addressing module 19 receives the reference motion orientation information, the ultrasonic ranging information and the relative position coordinate information, and provides for immediate addressing and calculation of the orientation, angle and relative distance of the vehicle body 10. As another example, in step 408, the image feature recognition module 12 establishes a region of interest (ROI) corresponding to the object to be searched in the indoor area 1 based on the environmental image information and the feature recognition information, and compares the region of interest of the environmental image information to determine whether there is an image of an object that matches an object image data stored in the database, wherein the object image comparison is performed by the image feature recognition module 12 based on the key features of the scanned environmental image information or the features of the object, and compares the image with the corresponding object features in the region of interest ROI with the features of the object stored in the database. If the matching object image data is compared in the region of interest of the environmental image information, then as in step 409, the relative distance and coordinate position (such as the distance between the object in the region of interest and the vehicle body 10) are measured and calculated. Figure 4 As shown), if no matching object image data is found, the vehicle body 10 is controlled by the central processor 15 to move to another indoor area 1 and returns to step 403 to continue searching for another indoor area 1.

[0033] The coordinate conversion calculation of the plane (2D) and three-dimensional (3D) space by the coordinate conversion module 16 is to first obtain the coordinates of the key points in the 2D space from the image feature recognition module 12, such as a straight line of a specific length in the room, a specific angle (such as 90 degrees), etc., and then obtain the coordinates of the 3D space marker N from the image feature recognition module to establish a 2D / 3D mapping relationship matrix for calculating the 2D to 3D coordinate conversion.

[0034] When the vehicle body 10 is positioned indoors, the 6D information of the vehicle body 10 is first obtained from the vehicle motion sensing module 21, and the real-time image is continuously input from the image input module 11. Then, the navigation position of the vehicle body 10 is measured, and the front / side distance is captured from the ultrasonic ranging module 14, and the coordinate conversion module 16 is called to calculate and correct the vehicle positioning and 6D information. Based on this, the vehicle body 10 is immediately addressed and the 6D information is output. When searching and positioning an object in an indoor environment, the real-time 3D coordinates of the vehicle body 10 are captured from the indoor positioning result of the vehicle body 10, and the real-time image is continuously input from the image input module 11. The distance of the search object relative to the vehicle is obtained from the ultrasonic ranging module 14, and the coordinate conversion module 16 is called to calculate the 3D positioning information of the object to be searched, and the immediate address of the object to be searched is also output.

[0035] The present invention discloses an indoor positioning and object search method for an intelligent unmanned vehicle system. The vehicle body uses intelligent automatic navigation (e.g., an inertial reference unit [IRU] combined with an AI navigation algorithm, etc.) and autonomous flight control in combination with AI to adjust flight parameters. A model of the indoor environment is established to locate the vehicle body. When moving in the indoor environment, the vehicle body can move between corners and directly avoid various obstacles in the environment. The vehicle body can compare the area of ​​interest corresponding to the object in the area with the area of ​​interest of the environmental image information to determine whether there is an image corresponding to the object to be found. On the other hand, the present invention can be used in stores to perform face recognition to find a person, such as a lost child, or in stores where security guards patrol after closing time to identify small animals such as cats, dogs or thieves that appear in the store. The vehicle body can not only effectively distinguish the image of the object to be found from the images of the surrounding furnishings, but also increase the efficiency of identifying objects, people or creatures in the regional environment, and achieve the above-mentioned purpose.

[0036] Although the present invention has been disclosed as above with preferred embodiments, it is not intended to limit the present invention. Anyone skilled in the art may make some changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be determined by the definition of the claims.

Claims

1. An indoor positioning and object search method for an intelligent unmanned vehicle system, which is used for positioning in an indoor area and for searching an object in the indoor area. The intelligent unmanned vehicle system includes a vehicle body, an image input module, an image feature recognition module, a central processing unit, an image initial marking module, a database, a vehicle motion sensing module and an image processing module. The central processing unit is respectively linked to the image input module, the image feature recognition module, the image initial marking module, the database and the vehicle motion sensing module by signal, and is configured in the vehicle body with these modules. The indoor positioning and object search method includes the following steps: a) the vehicle body is controlled by the central processor to move in the indoor area, and the image initial marking module analyzes the environmental image of the indoor area where the vehicle body is located captured by the image input module, identifies the original structure of the indoor area and the multiple original ornaments set in the indoor area, establishes an identification feature model corresponding to the indoor environment, and stores it in the database; b) the vehicle motion sensing module detects the motion orientation of the vehicle body and generates reference motion orientation information, and the image input module captures the environmental image of the indoor area to generate environmental image information corresponding to the indoor area and obtain spatial relative relationship information; c) the image feature recognition module receives the environmental image information, and performs feature recognition on the environmental image information to generate feature recognition information; d) the image processing module establishes a region of interest corresponding to the object in the indoor area according to the environmental image information and the feature recognition information, and compares the region of interest in the environmental image information to determine whether there is an image of the object that matches an object image data stored in the database; and e) if it is met, measuring and calculating the relative distance and spatial coordinate position between the object in the region of interest and the carrier body; in, The image processing module is signal-linked to the central processing unit and is configured in the vehicle body. In step b), the image processing module calculates the navigation orientation of the vehicle body based on the reference motion orientation information detected by the vehicle motion sensing module, and provides the central processing unit with a corrected navigation command.

2. The indoor positioning and object searching method as described in claim 1, wherein the intelligent unmanned vehicle system further comprises an ultrasonic ranging module which is signal-linked to the central processing unit and is disposed in the vehicle body, and the step b) comprises a step b1) the ultrasonic ranging module performs distance measurement around the vehicle body to generate ultrasonic ranging information for the central processing unit to control the vehicle body to perform ranging and avoid obstacles.

3. The indoor positioning and object searching method as described in claim 2, wherein the intelligent unmanned vehicle system further comprises a coordinate conversion module which is signal-linked to the central processing unit and is configured in the vehicle body, and the step b) comprises a step b2) the coordinate conversion module calculates the coordinate conversion of the plane (2D) and three-dimensional (3D) space of the vehicle body relative to the indoor environment based on the environmental image information to generate a relative position coordinate information.

4. The indoor positioning and object searching method as described in claim 3, wherein the processing module further comprises an immediate addressing module which is signal-linked to the central processing unit and is disposed in the vehicle body, and the step b) comprises a step b3) the immediate addressing module receives the reference motion orientation information, the ultrasonic ranging information and the relative position coordinate information, and immediately addresses and calculates the orientation, angle and relative distance of the vehicle body.

5. The indoor positioning and object searching method as described in claim 1, wherein the step d) includes a step d1) the image comparison of the object is performed by the image feature recognition module based on scanning the key features of the environmental image information or the features of the object, and the image in the region of interest ROI having the features corresponding to the object is compared with the features of the object stored in the database.

6. The indoor positioning and object searching method as claimed in claim 2, wherein the step e) includes a step e1) of calculating and analyzing the relative distance between the object and the vehicle body through the ultrasonic ranging module.

7. The indoor positioning and object searching method as described in claim 1, wherein the image feature recognition module is a set of dual lenses with depth recognition, and the step e) includes a step e1) the relative distance between the object and the vehicle body is calculated and analyzed based on the depth recognition of the object by the dual lenses.

8. The indoor positioning and object searching method as described in claim 1, wherein the image feature recognition module has an infrared ranging module, and the step e) includes a step e1) the relative distance between the object and the vehicle body is based on the measurement of the object by the infrared ranging module.

9. The indoor positioning and object searching method as described in claim 1, wherein the intelligent unmanned vehicle system further comprises an initial parameter setting module which is signal-linked to the central processing unit and is configured in the vehicle body, and the indoor positioning and object searching method further comprises a step a') before the step a), wherein the initial parameter setting module initializes various parameters of the vehicle body, performs measurement calibration, and initializes the coordinate origin.

Images