Image processing based navigation method, device, system and storage medium

Abstract

The application discloses a navigation method, device and system based on image processing and a storage medium, and the navigation method is applied to a navigation device, the navigation device comprises a master control device and at least one prompt device for interaction, the master control device comprises two image acquisition devices, two driving devices and at least one distance sensor, the distance sensor is arranged between the two image acquisition devices, the two driving devices are respectively used for driving the two image acquisition devices to rotate, the rotating direction is perpendicular to the collecting direction of any image acquisition device, and the swinging directions of the two image acquisition devices are always opposite, the application can acquire the image in front of a person with poor eyesight in real time through the two image acquisition devices, can judge whether a target object exists and the moving direction and speed of the target object according to the collected image, can timely send a prompt to the person with poor eyesight according to the judgment, and thus the problem that the person with poor eyesight is difficult to move and is prone to risks without help can be avoided.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of navigation technology, in particular to a navigation method, device and system based on image processing and a storage medium. BACKGROUND

[0002] When going out, the visually impaired usually use a blind stick or other tools to identify the surrounding environment, but this way cannot timely and accurately judge the external environment and obstacle information, so that the visually impaired cannot avoid the possible safety risks in an emergency, which has a great safety hazard. In the prior art, the tools that can help the visually impaired to travel are only simple blind sticks or guide dogs, but these cannot meet the daily travel safety requirements of the visually impaired. SUMMARY

[0003] In order to solve the problem of helping the visually impaired to timely judge and prompt the surrounding environment and improve the travel safety of the visually impaired, the present application provides a navigation method, device and system based on image processing and a storage medium.

[0004] According to a first aspect, the present application provides a navigation method based on image processing, which is applied to a navigation device, the navigation device comprising a master control device and at least one prompt device for interaction, the master control device being in communication connection with each prompt device, the master control device comprising two image acquisition devices, two driving devices and at least one distance sensor, the distance sensor being arranged between the two image acquisition devices, the two driving devices being respectively used to drive the two image acquisition devices to rotate, the rotation direction being perpendicular to the acquisition direction of any image acquisition device, and the swinging directions of the two image acquisition devices being always opposite; the navigation method comprising the following steps:

[0005] The distance sensor determines a first distance, the first distance being used to represent the distance between the master control device and a target object in the facing direction thereof;

[0006] When the first distance is less than a first preset distance, the first image acquisition device acquires a first image set and the second image acquisition device acquires a second image set;

[0007] According to the first image set, a first depth information set is determined, and according to the second image set, a second depth information set is determined, the first depth information set comprising a plurality of first depth information, and the second depth information set comprising a plurality of second depth information;

[0008] According to the first depth information set and the second depth information set, the moving direction and the moving speed of the target object are determined;

[0009] generating prompt information according to the moving direction and moving speed of the target object;

[0010] The prompting device sends the prompting information to the user.

[0011] According to a second aspect, the present invention provides an image processing-based navigation device, the navigation device being applied to a navigation device, the navigation device including a main control device and at least one interactive prompting device, the main control device being communicatively connected to each of the prompting devices, the main control device including two image acquisition devices, two driving devices, and at least one distance sensor, the distance sensor being disposed between the two image acquisition devices, the two driving devices being respectively configured to drive the two image acquisition devices to rotate, the rotation direction being perpendicular to the acquisition direction of either image acquisition device, and the swing directions of the two image acquisition devices being always opposite; the navigation device including:

[0012] an acquiring unit, configured to determine a first distance using the distance sensor, where the first distance is used to represent a distance between the main control device and a target object in a direction it faces;

[0013] an acquisition device control unit, configured to control the first image set acquisition device to acquire a first image set and control the second image acquisition device to acquire a second image set when the first distance is less than a first preset distance;

[0014] a depth information determining unit, configured to determine a first depth information set based on the first image set, and determine a second depth information set based on the second image set, wherein the first depth information set includes a plurality of first depth information and the second depth information set includes a plurality of second depth information;

[0015] a movement determining unit, configured to determine a movement direction and a movement speed of the target object according to the first depth information set and the second depth information set;

[0016] a prompt information generating unit, configured to generate prompt information according to the moving direction and moving speed of the target object;

[0017] The prompt unit is used to control the prompt device to send the prompt information to the user.

[0018] According to a third aspect, the present invention provides a navigation system comprising a processor, a memory, a transceiver, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, the programs comprising instructions for executing the steps in the method according to any one of claims 1 to 7.

[0019] According to a fourth aspect, the application provides a computer-readable storage medium storing a computer program for electronic data exchange, wherein the computer program causes a computer to perform the method according to any one of claims 1-7.

[0020] The application has the beneficial effect that the first image acquisition device and the second image acquisition device acquire images of the front of the visually impaired person in real time, and determine whether there is a target object and the moving direction and speed of the target object according to the acquired images, so that the visually impaired person can be prompted in time according to the determination, thereby avoiding the problem that the visually impaired person has difficulty in moving without help and is prone to risks. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description only constitute some embodiments of the application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0022] Figure 1 The gray value distribution diagram of each pixel point in the first block of an embodiment in the application;

[0023] Figure 2 The gray value distribution diagram of each pixel point in the second block of an embodiment in the application;

[0024] Figure 3 The first depth information distribution diagram of each first block in the first image set of an embodiment in the application;

[0025] Figure 4 The depth information difference value distribution diagram of each first block in the first image set of an embodiment in the application;

[0026] Figure 5 The second depth information distribution diagram of each second block in the second image set of an embodiment in the application;

[0027] Figure 6 The depth information difference value distribution diagram of each second block in the second image set of an embodiment in the application;

[0028] Figure 7 The flowchart of the image processing-based navigation method of an embodiment in the application;

[0029] Figure 8 The structural schematic diagram of the image processing-based navigation device of another embodiment in the application;

[0030] Figure 9FIG. 4 is a structural diagram of a navigation system based on image processing according to another embodiment of the present invention. DETAILED DESCRIPTION

[0031] The present invention will be further described in detail below by means of specific embodiments in conjunction with the accompanying drawings. Similar elements in different embodiments are numbered with associated similar elements. In the following embodiments, many detailed descriptions are provided to enable the present invention to be better understood. However, those skilled in the art will readily appreciate that some of the features may be omitted under different circumstances, or may be replaced by other elements, materials, or methods. In some cases, some operations related to the present invention are not shown or described in the specification. This is to avoid the core of the present invention being overwhelmed by excessive descriptions, and for those skilled in the art, it is not necessary to describe these related operations in detail. They can fully understand the related operations based on the description in the specification and the general technical knowledge in the art.

[0032] In addition, the features, operations, or characteristics described in the specification may be combined in any appropriate manner to form various embodiments. Furthermore, the steps or actions in the method description may be reordered or adjusted in a manner readily apparent to those skilled in the art. Therefore, the various sequences in the specification and drawings are provided solely for the purpose of clearly describing a particular embodiment and are not intended to be mandatory, unless otherwise specified.

[0033] The serial numbers in the specification, such as "first" and "second", are used only to distinguish the objects described and do not have any sequential or technical meaning. The terms "connection" and "coupling" used in this disclosure include both direct and indirect connections (couplings) unless otherwise specified.

[0034] Please refer to Figures 1 to 7The embodiment provides a navigation method based on image processing, which is applied to a navigation device for guiding a visually impaired person. The navigation device comprises a master device and at least one prompting device for interaction, and the master device is in communication connection with each prompting device. Specifically, the master device comprises two image acquisition devices, two driving devices and at least one distance sensor, the distance sensor is arranged between the two image acquisition devices, the two image acquisition devices are a first image acquisition device and a second image acquisition device respectively, the driving devices are a first driving device and a second driving device respectively, the first image acquisition device is connected with the first driving device, the second image acquisition device is connected with the second driving device, the first driving device is used for driving the first image acquisition device to rotate, and the second driving device is used for driving the second image acquisition device to rotate. The rotation directions of the first image acquisition device and the second image acquisition device are perpendicular to the acquisition directions of any image acquisition device, and the swinging directions of the two image acquisition devices are always opposite.

[0035] The navigation method based on image processing specifically comprises the following steps:

[0036] S10: The distance sensor collects data to determine a first distance, and the first distance is used for representing the distance between the master device and a target object in the direction faced by the master device.

[0037] The distance sensor periodically measures the distance between the distance sensor and an object along the measurement direction, and the detection of the first distance by the distance sensor can be that the master device performs scanning along the vertical direction and intermittently measures the distance, or the master device performs scanning along the horizontal direction and intermittently measures the distance.

[0038] The first distance is 10 meters, 15 meters or other distances.

[0039] S20: When the first distance is less than a first preset distance, the first image acquisition device acquires a first image set and the second image acquisition device acquires a second image set.

[0040] The first preset distance is 3 meters, 5 meters or other distances.

[0041] When the first distance is less than the first preset distance, it is indicated that there may be an obstacle in front of the visually impaired person, which affects the forward movement of the visually impaired person, and therefore, the target parameters such as the moving direction and the moving speed of the obstacle need to be judged.

[0042] In order to facilitate the determination of the target parameter of the target object, the first image acquisition device and the second image acquisition device need to be controlled to rotate simultaneously when image acquisition is performed by the first image acquisition device and the second image acquisition device. In a specific embodiment, the first image acquisition device rotates clockwise, and the second image acquisition device rotates counterclockwise. The first image acquisition device and the second image acquisition device obtain images of the target object during rotation. In a preferred embodiment, the first image acquisition device and the second image acquisition device are line array cameras.

[0043] S30: determining a first depth information set according to the first image set and determining a second depth information set according to the second image set, wherein the first depth information set includes a plurality of first depth information, and the second depth information set includes a plurality of second depth information. The first depth information is used to represent the distance between the first image acquisition device and the target object, and the second depth information is used to represent the distance between the second image acquisition device and the target object.

[0044] The process of determining the first depth information set according to the first image set and determining the second depth information set according to the second image set includes the following steps:

[0045] S31: dividing the first image set into a plurality of first blocks and dividing the second image set into a plurality of second blocks.

[0046] When the first image set is divided into a plurality of first blocks, the first depth information can include a plurality of distance information. Specifically, each first block corresponds to one distance information in the first depth information. Similarly, the second depth information also includes a plurality of distance information, and each second block corresponds to one distance information in the second depth information.

[0047] In a specific embodiment, when the first image acquisition device and the second image acquisition device are line array cameras, the image acquired by the first image acquisition device and the second image acquisition device each time is a single row of image. The first image acquisition device and the second image acquisition device continuously acquire images during rotation, and complete the acquisition of the first image set and the second image set after rotation is completed.

[0048] Therefore, in order to facilitate the division of the first image set into different first blocks, considering the size of the image acquired by the first image acquisition device each time, the width of the first block is set to one pixel, and the height is set to 10 pixels. The width and height of the first block are the same as the width and height of the second block.

[0049] In another specific embodiment, the width and height of the first block are equal. This division method can more simply determine the depth information in the first image set, thereby facilitating the judgment of the target parameter of the target object.

[0050] S32: Determine the first sharpness of each first block and the second sharpness of each second block.

[0051] In order to determine the depth information of each block, it is necessary to first determine the sharpness of each block. Specifically, when the target object is within the depth of field range of the first image acquisition device, the first sharpness of the first image set acquired by the first image acquisition device is high, and when the target object is outside the depth of field range of the first image acquisition device, the first sharpness of the first image set acquired by the first image acquisition device is low. When the first image acquisition device only acquires one image at the same rotation angle, the distance between the first image acquisition device and the target object can be determined by the high and low of the first sharpness of the first image set.

[0052] The first sharpness of each first block and the second sharpness of each second block can be determined in various ways. In a specific embodiment, the image sharpness can be determined by the Brenner gradient function. Specifically, f1(x,y) is set as the gray value of the pixel point (x,y) corresponding to the first block, and D(f1) is the image sharpness of the first block. The image sharpness of the first block can be expressed as:

[0053] D(f1) = ∑ y ∑ x |f1(x+2,y)-f1(x,y)| 2

[0054] In another specific embodiment, the image sharpness can also be determined by the gray scale variance function. Specifically, f2(x,y) is set as the gray value of the pixel point (x,y) corresponding to the second block, and D(f2) is the image sharpness of the second block. The image sharpness of the second block can be expressed as:

[0055] D(f2) = ∑ y ∑ x (|f2(x,y)-f2(x,y-1)|+|f2(x,y)-f2(x+1,y)|

[0056] S33: Calculate the first depth information of the corresponding block according to the first sharpness of each first block, and calculate the second depth information of the corresponding block according to the second sharpness of each second block, and finally obtain the first depth information set and the second depth information set.

[0057] In an embodiment, the first image set includes only one first image, and the second image set includes only one second image, and then the process includes the following steps S331-S334 when determining the first depth information of each corresponding block according to the first definition of each first block and determining the second depth information of each corresponding block according to the second definition of each second block.

[0058] S331: When the first image set includes only one first image, determining the definition difference between the image definition of each first block in the first image and the preset definition.

[0059] In an embodiment, the first block is a 5-pixel*5-pixel display area, and the image definition of the first block is calculated by the Brenner gradient function, wherein the gray value distribution of each pixel point in the first block is as follows: Figure 1 According to the Brenner gradient function, the image definition of the first block is 22802, and if the preset definition is 30000, then the definition difference corresponding to the first block is 7198.

[0060] S332: After determining the definition difference corresponding to the first block, determining the first depth information of the first block according to the definition difference. Specifically, the first depth information is calculated according to the first formula L1=D1x[1+(St1+1000) / S’], wherein L1 is the first depth information, D1 is the optimal imaging distance of the first image acquisition device, St1 is the difference between the definition of the first block and the preset definition, and S’ is the preset definition.

[0061] In an embodiment, assuming that the image definition of the first block is 22802, the preset definition S’ is 30000, and the optimal imaging distance D1 of the first image acquisition device is 2500 mm, then the first depth information L1 of the first block calculated according to the first formula is 3183 mm.

[0062] S333: When the second image set includes only one second image, determining the definition difference between the image definition of each second block in the second image and the preset definition.

[0063] In an embodiment, the second block is a 5-pixel*5-pixel display area, and the image definition of the second block is calculated by the Brenner gradient function, wherein the gray value distribution of each pixel point in the second block is as follows: Figure 2 According to the Brenner gradient function, the image definition of the second block is 7358, and if the preset definition is 30000, then the definition difference corresponding to the second block is 22642.

[0064] S334: calculating the second depth information according to a second formula L2=D2x[1+(St2+1000) / S'], wherein L2 is the second depth information, D2 is the optimal imaging distance of the second image acquisition device, St2 is the difference between the definition of the second block and the preset definition, and S' is the preset definition.

[0065] In a specific embodiment, assuming that the image definition of the second block is 7358, the preset definition S' is 30000, and the optimal imaging distance D2 of the second image acquisition device is 2500 mm, then according to the second formula, the second depth information L2 of the second block calculated is 4470 mm.

[0066] In another specific embodiment, the first image set includes a plurality of first images. Specifically, the first image acquisition device acquires a plurality of first images with different focal lengths at the same rotation angle, so when determining the first depth information of each first block in the first image set, the definition of the first block in different first images can be determined first, and the focal length corresponding to the first image with the highest definition is determined as the depth information of the first block.

[0067] For example, the first image set includes 5 first images, and the definition of the first block in the 5 first images is determined respectively. When the definition of the first block in the third first image is the largest, then the focal length corresponding to the first image acquisition device when the third first image is collected is determined as the depth information of the first block.

[0068] After determining the depth information of each first block in the first image set, the set of depth information of all the first blocks is the first depth information set.

[0069] The determination method of the second depth information set of the second image set is the same as that of the first depth information set of the first image set, which will not be described here.

[0070] S40: determining the moving direction and moving speed of the target object according to the first depth information set and the second depth information set. When the target object is too close to the visually impaired person, the moving speed and moving direction of the target object need to be further determined, so as to timely remind the visually impaired person to avoid.

[0071] The process of determining the moving direction and moving speed of the target object according to the first depth information set and the second depth information set includes the following steps S41 to S42.

[0072] S41: determining at least one first target block in the first image set according to the first depth information in the first depth information set, and determining at least one second target block in the second image set according to the second depth information in the second depth information set.

[0073] This step specifically includes:

[0074] S411: Calculating a first depth difference between a first block and an adjacent block according to first depth information;

[0075] S412: Select one or more first blocks whose first difference is greater than a preset difference as first target blocks;

[0076] S413: Calculating a second depth difference between the second block and an adjacent block according to the second depth information;

[0077] S414: Select one or more second blocks whose second difference is greater than a preset difference as second target blocks.

[0078] In a specific embodiment, the first image set includes 10*10 first blocks, and the first depth information of each first block is as follows: Figure 3 As shown in the figure, the letters A and B are used to represent the row where the first block in the first image is located, and the numbers 1, 2, 3, etc. after the letters represent the column where the first block in the first image is located. Specifically, A1, A2…A10 in the figure represent the first blocks at different positions corresponding to the first row in the first image, wherein A1 represents the first block in the first row and first column of the first image, A2 represents the first block in the first row and second column of the first image, and so on. B1, B2…B10 represent the first blocks at different positions corresponding to the second row in the first image, wherein B1 represents the first block in the second row and first column of the first image, B2 represents the first block in the second row and second column of the first image, and so on. Then, for the first image set, the difference between the average value of the depth information of each first block and the eight adjacent blocks on the surrounding side is calculated to obtain a distribution diagram of the first difference, as shown Figure 4 shown.

[0079] In another specific embodiment, the second image set includes 10*10 second blocks, and the second depth information of each second block is as follows: Figure 5 As shown, then in the second image set, the difference between the average value of the depth information of each second block and the eight adjacent blocks on the surrounding side is calculated to obtain a distribution diagram of the second difference, as shown in FIG. Figure 6 shown.

[0080] If the default difference is 2500, then select Figure 4 Among the multiple first blocks whose first depth differences are greater than 2500, some of the first blocks are selected as first target blocks. Specifically, the first target blocks may be (A9, B5), (A3, B6), (A5, B6) or other blocks.

[0081] The method for determining the second target block is the same as that for determining the first target block, and will not be described in detail here.

[0082] S42: determining the moving speed and the moving direction of the target object according to the first target block and the second target block.

[0083] After the first target block of the first image set and the second target block of the second image set are determined, the moving parameters of the target object, including the moving speed and the moving direction, are determined according to the positions of the first target block and the second target block in the corresponding image set.

[0084] In an embodiment, the process of determining the moving speed and the moving direction of the target object according to the first target block and the second target block comprises the following steps:

[0085] S421: determining the first position information of the first target block in the first image set and the second position information of the second target block in the second image set.

[0086] The first position information is used to represent the corresponding position of the first target block in the first image set, and the second position information is used to represent the corresponding position of the second target block in the second image set.

[0087] In an embodiment, the first position information is represented by block coordinates, as shown in the following table: Figure 3 As shown in the following table, the first image set and the second image set each include 10*10 blocks, wherein the first depth information distribution of the first image set is as shown in the following table: Figure 3 The second depth information distribution of the second image set is as shown in the following table: Figure 5 The first position information of the first target block is (A9, B5), and the second position information of the second target block is (A6, B5).

[0088] S422: determining the moving direction and the moving speed of the target object according to the first depth information of the first target block, the second depth information of the second target block, the first position information and the second position information.

[0089] Since the first image acquisition device and the second image acquisition device rotate in opposite directions and acquire images, when the target object moves in front of the visually impaired person, the moving direction and the moving speed can be determined through the first position information of the first target block and the second position information of the second target block.

[0090] In an optional embodiment, determining the moving direction and the moving speed of the target object according to the first depth information of the first target block, the second depth information of the second target block, the first position information and the second position information comprises the following steps:

[0091] S4221: Calculate the moving speed V1 of the target object along the first direction according to the third formula V1=[0.5*d*(L1+L2)*(x2-x1)] / t, wherein L1 represents the first depth information of the first target block, L2 represents the second depth information of the second target block, the first position information of the first target block is represented by (x1, y1), the second position information of the second target block is represented by (x2, y2), x1 and x2 represent the coordinate positions of the first target block and the second target block along the horizontal direction respectively, d is the block distance coefficient, and t is the single collection time length of the first image collection and / or the second image collection device.

[0092] S4222: Calculate the moving speed V2 of the target object along the second direction according to the fourth formula V2=[0.5*d*(L1+L2)*(y2-y1)] / t, wherein L1 represents the first depth information of the first target block, L2 represents the second depth information of the second target block, the first position information of the first target block is represented by (x1, y1), the second position information of the second target block is represented by (x2, y2), x1 and x2 represent the coordinate positions of the first target block and the second target block along the horizontal direction respectively, d is the block distance coefficient, and t is the single collection time length of the first image collection and / or the second image collection device.

[0093] In an embodiment, the first position information of the first target block is (A9, B5), the second position information of the second target block is (A6, B5), the block distance coefficient d is 0.02, the single collection time length t of the image collection is 0.5 seconds, the first depth information L1 of the first target block is 6881, and the second depth information L2 of the second target block is 5733. According to the third formula, the moving speed V1 of the target object is calculated to be 757 mm / s, and the moving direction of the target object is the moving direction of the image collection region of the first image collection device during image collection.

[0094] In another embodiment, the first position information of the first target block is (A4, B10), the second position information of the second target block is (A4, B9), the block distance coefficient d is 0.02, the single collection time length t of the image collection is 0.5 seconds, the first depth information L1 of the first target block is 6881, and the second depth information L2 of the second target block is 5733. According to the fourth formula, the moving speed of the target object is 252 mm / s, and the target object is moving towards the direction of approaching the visually impaired person.

[0095] S50: Generate the prompt information according to the moving direction and the moving speed of the target object, and send the prompt information to the prompt device.

[0096] S60: prompting the device to send the prompt information to the user.

[0097] Wherein, after determining the moving direction and moving speed of the target object, the host device determines whether the target object will cause an obstacle to the moving direction of the visually impaired person according to the moving direction and the moving speed, and when causing an obstacle, the host device sends prompt information to the visually impaired person through the prompting device of the navigation device, specifically, the prompt information can be informed to the visually impaired person through vibration reminding, voice reminding and the like, so as to facilitate the visually impaired person to avoid or move in time.

[0098] The application provides a navigation method based on image processing, which is applied to a navigation device, the navigation device comprising a host device and at least one prompting device for interaction, the host device being in communication connection with each prompting device, the host device comprising two image acquisition devices, two driving devices and at least one distance sensor, the navigation method comprising the following steps: controlling the distance sensor to determine a first distance, the first distance being used to represent the distance between the host device and a target object facing the direction of the host device; when the first distance is less than a first preset distance, controlling a first image acquisition device to acquire a first image set and a second image acquisition device to acquire a second image set; determining first depth information according to the first image set and second depth information according to the second image; determining the moving direction and moving speed of the target object according to the first depth information and the second depth information; determining prompt information according to the moving direction and moving speed of the target object; and controlling the prompting device to send the prompt information to the user. The first image acquisition device and the second image acquisition device are used to acquire images in front of the visually impaired person in real time, and whether there is a target object and the moving direction and moving speed of the target object are determined according to the acquired images, so that prompt information can be sent to the visually impaired person in time, thereby avoiding the problem that the visually impaired person has difficulty in action and is prone to risks without help.

[0099] Please refer to Figure 8 , Figure 8Figure 1 is a structural schematic diagram of a navigation device based on image processing provided by an embodiment of the present application, which is applied to a navigation device. The navigation device comprises a master device and at least one prompt device for interaction, the master device is in communication connection with each prompt device, the master device comprises two image acquisition devices, two driving devices and at least one distance sensor, the distance sensor is arranged between the two image acquisition devices, the two image acquisition devices are respectively a first image set acquisition device and a second image acquisition device, the driving devices are respectively a first driving device and a second driving device, the first image set acquisition device is connected with the first driving device, the second image acquisition device is connected with the second driving device, the first driving device is used for driving the first image set acquisition device to rotate, the second driving device is used for driving the second image acquisition device to rotate, the rotation directions of the first image set acquisition device and the second image acquisition device are perpendicular to the acquisition directions of any image acquisition device, and the swinging directions of the two image acquisition devices are always opposite.

[0100] The navigation device based on image processing of the embodiment comprises:

[0101] The acquisition unit 410 is used for determining a first distance by the distance sensor, the first distance is used for representing the distance between the master device and a target object facing the direction thereof;

[0102] The acquisition device control unit 420 is used for controlling the first image set acquisition device to acquire a first image set and controlling the second image acquisition device to acquire a second image set when the first distance is less than a first preset distance;

[0103] The depth information determination unit 430 is used for determining a first depth information set according to the first image set and determining a second depth information set according to the second image, wherein the first depth information set comprises a plurality of first depth information, and the second depth information set comprises a plurality of second depth information;

[0104] The movement determination unit 440 is used for determining the movement direction and the movement speed of the target object according to the first depth information set and the second depth information set;

[0105] The prompt information generation unit 450 is used for generating prompt information according to the movement direction and the movement speed of the target object;

[0106] The prompt unit 460 is used for controlling the prompt device to send the prompt information to the user.

[0107] In a specific embodiment of the present application, in terms of determining the first depth information set according to the first image set and determining the second depth information set according to the second image, the depth information determination unit 430 is specifically used for:

[0108] dividing the first image set into a plurality of first blocks and dividing the second image set into a plurality of second blocks;

[0109] determining a first sharpness of each first block and a second sharpness of each second block;

[0110] determining first depth information of the corresponding block according to the first sharpness of each first block and determining second depth information of the corresponding block according to the second sharpness of each second block, to finally obtain the first depth information set and the second depth information set.

[0111] Further, in a specific embodiment of the present application, in terms of calculating the first depth information of the corresponding block according to the first sharpness of each first block and calculating the second depth information of the corresponding block according to the second sharpness of each second block, the depth information determination unit 430 is specifically used for:

[0112] When the first image set only includes one first image, first determine the sharpness difference between the image sharpness of each first block in the first image and the preset sharpness, and then calculate the first depth information L1 according to the first formula L1=D1×[1+(St1+1000) / S'];

[0113] When the second image set only includes one second image, first determine the sharpness difference between the image sharpness of each second block in the second image and the preset sharpness, and then calculate the second depth information L2 according to the second formula L2=D2×[1+(St2+1000) / S'].

[0114] In another specific embodiment of the present application, in terms of calculating the first depth information of the corresponding block according to the first sharpness of each first block and calculating the second depth information of the corresponding block according to the second sharpness of each second block, the depth information determination unit 430 is specifically used for:

[0115] When the number of first images in the first image set is multiple, determine the sharpness of the corresponding first block in each first image, and determine the sharpness of the corresponding second block in each second image;

[0116] Determine the focal length corresponding to the first image with the highest sharpness as the first depth information of the block, and determine the focal length corresponding to the second image with the highest sharpness as the second depth information of the block.

[0117] In a specific embodiment of the present application, in terms of determining the moving direction and moving speed of the target object according to the first depth information and the second depth information, the determination unit 420 is specifically used for:

[0118] determine at least one first target block in the first image set according to the first depth information, and determine at least one second target block in the second image set according to the second depth information;

[0119] determine the moving direction and the moving speed of the target object according to the first target block and the second target block.

[0120] In an embodiment of the present application, in the aspect of determining at least one first target block in the first image set according to the first depth information in the first depth information set, and determining at least one second target block in the second image set according to the second depth information in the second depth information set, the determining unit 420 is specifically configured to:

[0121] determine a first depth difference between the first block and a neighboring block according to the first depth information;

[0122] when the first difference is greater than a preset difference, determine the first block as the second target block;

[0123] determine a second depth difference between the second block and a neighboring block according to the second depth information;

[0124] when the second difference is greater than the preset difference, determine the second block as the second target block.

[0125] In an embodiment of the present application, in the aspect of determining the moving direction and the moving speed of the target object according to the first target block and the second target block, the moving determining unit 440 is specifically configured to:

[0126] determine first position information of the first target block in the first image set, and determine second position information of the second target block in the second image set;

[0127] determine the moving direction and the moving speed of the target object according to the first depth information of the first target block, the second depth information of the second target block, the first position information and the second position information.

[0128] It should be noted that the depth information determining unit 430 and the prompting unit 460 can be realized by a processor, and the obtaining unit 410 can be realized by a camera. The specific function implementation method of each module in the navigation device based on image processing provided in the present embodiment can refer to the implementation method described in the above navigation method embodiments, which will not be described here.

[0129] Please refer to Figure 9 , Figure 9is a structural schematic diagram of a navigation system based on image processing provided by an embodiment of the present application, which comprises a processor, a memory, a transceiver, and one or more programs stored in the memory and configured to be executed by the processor, and the program comprises instructions for executing steps in the navigation method according to any one of the above embodiments.

[0130] The embodiment of the present application further provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program for electronic data exchange, and the computer program causes a computer to execute part or all of steps described in the above navigation method embodiment.

[0131] The embodiment of the present application further provides a computer program product, wherein the computer program product comprises a non-transitory computer readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute part or all of steps described in the above method. The computer program product can be a software installation package.

[0132] The steps of the method or algorithm described in the embodiments of the present application can be implemented by hardware, or be implemented by a combination of software and hardware, or be implemented by software executed by a processor. The software instructions can be stored in a memory, such as a Random Access Memory (RAM), a flash memory, a Read Only Memory (ROM), an Erasable Programmable ROM (EPROM), an Electrically EPROM (EEPROM), a register, a hard disk, a floppy disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, such that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can be a component of the processor. The processor and the storage medium can be located in an ASIC. The ASIC can be located in the access network device, the target network device, or the core network device. Of course, the processor and the storage medium can also be located in the access network device, the target network device, or the core network device as discrete components.

[0133] Those skilled in the art should be aware that, in the above one or more examples, the functions described in the embodiments of the present application can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the whole or part of the processes or functions according to the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable apparatus. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media. The available media can be magnetic media (such as floppy disk, hard disk, magnetic tape), optical media (such as digital video disc (DVD)), or semiconductor media (such as solid state disk (SSD)) and the like.

[0134] The above detailed description of the embodiments of the present application further illustrates the purposes, technical solutions and beneficial effects of the embodiments of the present application. It should be understood that the above detailed description is only a specific embodiment of the embodiments of the present application and is not used to limit the protection scope of the embodiments of the present application. Any modification, equivalent replacement, improvement, etc. made on the basis of the technical solutions of the embodiments of the present application should be included in the protection scope of the embodiments of the present application.

Claims

1. An image processing based navigation method, characterized by, The navigation method is applied to a navigation device, the navigation device comprising a master device and at least one prompting device for interaction, the master device being in communication connection with each of the prompting devices, the master device comprising two image acquisition devices, two driving devices and at least one distance sensor, the distance sensor being arranged between the two image acquisition devices, the two driving devices being respectively used to drive the two image acquisition devices to rotate, the rotating direction being perpendicular to the acquisition direction of any image acquisition device, and the swinging directions of the two image acquisition devices being always opposite; the navigation method comprising the following steps: The distance sensor determines a first distance, the first distance being used to represent the distance between the master device and a target object facing the direction of the master device; When the first distance is less than a first preset distance, a first image set acquisition device acquires a first image set and a second image acquisition device acquires a second image set; According to the first image set, a first depth information set is determined, and according to the second image set, a second depth information set is determined, the first depth information set comprising a plurality of first depth information, and the second depth information set comprising a plurality of second depth information; According to the first depth information set and the second depth information set, a moving direction and a moving speed of the target object are determined; According to the moving direction and the moving speed of the target object, a prompting information is generated; The prompting device sends the prompting information to a user; The process of determining the first depth information set and the second depth information set comprises the following steps: The first image set is divided into a plurality of first blocks, and the second image set is divided into a plurality of second blocks; The first clarity of each first block and the second clarity of each second block are determined; According to the first clarity of each first block, the first depth information of the corresponding block is calculated, and according to the second clarity of each second block, the second depth information of the corresponding block is calculated, to obtain the first depth information set and the second depth information set; The process of calculating the first depth information and the second depth information comprises the following steps: When the first image set only comprises one first image, the clarity difference value between the image clarity of each first block in the first image and a preset clarity is determined; According to a first formula L1=D1×[1+(St1+1000) / S’], the first depth information L1 is calculated, wherein D1 is the best imaging distance of the first image acquisition device, St1 is the difference value between the clarity of the first block and the preset clarity, and S’ is the preset clarity; When the second image set only comprises one second image, the clarity difference value between the image clarity of each second block in the second image and a preset clarity is determined; According to a second formula L2=D2×[1+(St2+1000) / S’], the second depth information L2 is calculated, wherein D2 is the best imaging distance of the second image acquisition device, and St2 is the difference value between the clarity of the second block and the preset clarity.

2. The image processing based navigation method of claim 1, wherein, The process of calculating the first depth information and the second depth information comprises the following steps: When the number of the first images in the first image set is multiple, determining the sharpness of the corresponding first block in each of the first images, and determining the sharpness of the corresponding second block in each of the second images; Determining the first depth information of the block corresponding to the first image with the highest sharpness as the first depth information of the block, and determining the second depth information of the block corresponding to the second image with the highest sharpness as the second depth information of the block.

3. The image processing based navigation method of claim 1, wherein, The process of determining the moving direction and speed of the target object includes the following steps: Determining at least one first target block in the first image set according to the first depth information, and determining at least one second target block in the second image set according to the second depth information; Determining the moving direction and speed of the target object according to the first target block and the second target block.

4. The image processing based navigation method of claim 3, wherein, The process of determining the first target block and the second target block includes the following steps: Calculating the first depth difference between the first block and the adjacent block according to the first depth information, and selecting at least one first block with the first depth difference greater than a preset difference value as the second target block; Calculating the second depth difference between the second block and the adjacent block according to the second depth information, and selecting at least one second block with the second depth difference greater than a preset difference value as the second target block.

5. The image processing based navigation method of claim 3, wherein, The process of determining the moving direction and speed of the target object includes the following steps: Determining the first position information of the first target block in the first image set, and determining the second position information of the second target block in the second image set; Determining the moving direction and speed of the target object according to the first depth information of the first target block, the second depth information of the second target block, the first position information, and the second position information.

6. An image processing based navigation device characterized by The navigation device is applied to a navigation equipment, the navigation equipment includes a master control equipment and at least one prompt device for interaction, the master control equipment is connected with each prompt device, the master control equipment includes two image acquisition devices, two driving devices and at least one distance sensor, the distance sensor is arranged between the two image acquisition devices, and the two driving devices are used to drive the two image acquisition devices to rotate, the rotating direction is perpendicular to the acquisition direction of any image acquisition device, and the swinging directions of the two image acquisition devices are always opposite. The navigation device includes: An acquisition unit is used to determine a first distance by the distance sensor, and the first distance is used to represent the distance between the master control equipment and the target object facing the direction thereof; A device control unit is used to control the first image set acquisition device to acquire the first image set and control the second image acquisition device to acquire the second image set when the first distance is less than a first preset distance; A depth information determination unit is used to determine a first depth information set according to the first image set, and determine a second depth information set according to the second image set, the first depth information set includes multiple first depth information, and the second depth information set includes multiple second depth information; The depth information determination unit is specifically used for: dividing the first image set into a plurality of first blocks and dividing the second image set into a plurality of second blocks; determining a first sharpness of each first block and a second sharpness of each second block; determining first depth information of a corresponding block according to the first sharpness of each first block and determining second depth information of a corresponding block according to the second sharpness of each second block, finally obtaining a first depth information set and a second depth information set; when the first image set only includes one first image, first determining a sharpness difference between an image sharpness of each first block in the first image and a preset sharpness, and then calculating first depth information L1 according to a first formula L1=D1×[1+(St1+1000) / S’], wherein D1 is an optimal imaging distance of a first image acquisition device, St1 is the sharpness difference between the first block and the preset sharpness, and S’ is the preset sharpness; when the second image set only includes one second image, first determining a sharpness difference between an image sharpness of each second block in the second image and a preset sharpness, and then calculating second depth information L2 according to a second formula L2=D2×[1+(St2+1000) / S’], wherein D2 is an optimal imaging distance of a second image acquisition device, St2 is the sharpness difference between the second block and the preset sharpness; a moving determination unit configured to determine a moving direction and a moving speed of the target object according to the first depth information set and the second depth information set; a prompt information generation unit configured to generate prompt information according to the moving direction and the moving speed of the target object; 7. A navigation system characterized by comprising: a prompt unit configured to control the prompt device to send the prompt information to a user.

8. A computer-readable storage medium, characterized in that, A computer program product including a processor, a memory, a transceiver, and one or more programs stored in the memory and configured to be executed by the processor, the programs including instructions for performing steps in the method of any one of claims 1-5. A computer program for electronic data exchange, wherein the computer program causes a computer to perform the method of any one of claims 1-5.

Images