Obstacle detection method, device and system

Abstract

The embodiment of the present invention relates to the field of robot positioning and navigation technology, and discloses an obstacle detection method, device and system. The system includes a first image sensor and a laser arranged at the same height and adjacent to each other, and a second image sensor arranged above or below the first image sensor. The method includes: the laser emits lasers of different brightness, and then the first image sensor and the second image sensor respectively obtain a first image and a second image containing laser stripes of different brightness. The laser stripes of different brightness are laser stripes reflected after lasers of different brightness are emitted to an obstacle. According to the positions of the laser stripes of different brightness on the first image and the second image, the distance information between the obstacle and the detection system is confirmed. The embodiment of the present invention reduces the overexposure effect of the distance on the collected laser stripes by emitting lasers of different brightness, and at the same time accurately determines the distance information of the obstacle through two image sensors, which has low cost and good effect.

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical field of robot positioning and navigation, and particularly to an obstacle detection method, device and system. Background Art

[0002] Obstacle avoidance technology is a necessity in the field of robot automatic positioning and navigation. With the increasing requirements for control accuracy and intelligence in the robot industry, obstacle avoidance sensors are becoming more and more miniaturized, and the sensing performance of sensors is getting higher and higher. Currently, for the direction of robot obstacle avoidance, common sensors include ultrasonic sensors, infrared sensors, lidar, and depth sensors, etc.; the prices of these sensors vary, so they are also applicable to different obstacle avoidance requirements. Among them, the infrared-based optical obstacle avoidance sensor has strong detection ability and high detail level, and can operate all-weather, which is the current mainstream trend of robot obstacle avoidance.

[0003] In the process of implementing the embodiments of the present invention, the inventors found that there are at least the following problems in the above related technologies: in the current infrared optical obstacle avoidance sensors, the prices of depth-based binocular, structured light, and TOF sensors are still high. The price of single-line lidar has gradually tended to the consumer level, and it is currently the solution that has started to be widely commercialized. However, single-line lidar can only detect obstacles on a single height section, and there will be a large blind area for obstacles below the lidar. For the blind area problem, some existing solutions propose to use other obstacle avoidance sensors to make up for it, including a line laser sensor combined with an image sensor, or multiple line lasers combined with an image sensor. Usually, during the movement of a mobile robot, the line lasers are emitted horizontally and form an angle with the ground, so that the line laser can hit the obstacles near the ground, and the detected point cloud information can be integrated into the map currently established by the robot for more detailed identification, so as to facilitate the robot to take corresponding obstacle avoidance measures. However, when the light source emitted by the line laser is reflected back to the vision sensor by the obstacle, the energy will change greatly with the obstacle material, angle or distance. Sometimes, the laser stripes at close range will be too bright and those at far range will be too weak, resulting in the failure of obstacle point cloud extraction. Summary of the Invention

[0004] Aiming at the above-mentioned defects of the prior art, the purpose of the embodiments of the present invention is to provide an obstacle detection method, device and system with low cost and good detection effect.

[0005] The purpose of the embodiments of the present invention is achieved by the following technical solutions:

[0006] To solve the above technical problems, in a first aspect, an obstacle detection method is provided in an embodiment of the present invention, which is applied to a detection system. The detection system includes a first image sensor and a laser that are arranged at the same height and adjacent to each other, and a second image sensor that is arranged above or below the first image sensor. The detection method includes:

[0007] The laser emits lasers with different brightnesses;

[0008] The first image sensor acquires a first image containing laser stripes with different brightnesses, where the laser stripes with different brightnesses are laser stripes that are emitted by lasers with different brightnesses to an obstacle and reflected by the obstacle;

[0009] The second image sensor acquires a second image, and the second image contains the laser stripes with different brightnesses;

[0010] According to the positions of the laser stripes with different brightnesses on the first image and the second image, the distance information between the obstacle and the detection system is confirmed.

[0011] In some embodiments, while the laser emits lasers with different brightnesses, the method further includes:

[0012] Controlling the first image sensor and the second image sensor to perform at least two exposures synchronously, where

[0013] When the second sensor is arranged below the first sensor, during each exposure process, the laser is synchronously modulated and lit, and during the process of sequentially lighting the laser, the duty cycle of the laser is controlled to gradually increase, so that the laser emits lasers with different brightnesses;

[0014] When the second sensor is arranged above the first sensor, during each exposure process, the laser is synchronously modulated and lit, and during the process of sequentially lighting the laser, the duty cycle of the laser is controlled to gradually decrease, so that the laser emits lasers with different brightnesses.

[0015] In some embodiments, the step of controlling the first image sensor and the second image sensor to perform at least two exposures synchronously further includes:

[0016] Dividing the photosensitive area in the second image sensor into a preset number of imaging areas from top to bottom, where the number of imaging areas is the number of times the laser is lit, and the preset number is greater than or equal to 2;

[0017] Controlling the preset number of imaging areas in the second image sensor to be exposed sequentially from top to bottom;

[0018] Confirm the imaging area in the first image sensor that can receive the emitted laser according to the height of the laser emitted by the laser;

[0019] Control the imaging area that can receive the emitted laser to perform synchronous exposure with a preset number of imaging areas in the second image sensor.

[0020] In some embodiments, the step of the first image sensor obtaining the first image containing laser stripes with different brightnesses further includes:

[0021] Obtain the brightness of the laser stripes in each first exposure image obtained by each exposure,

[0022] According to different laser stripe brightnesses, assign different weights to each first exposure image and perform weighting,

[0023] Synthesize the weighted first exposure images to obtain the first image with uniform brightness.

[0024] In some embodiments, the step of the second image sensor obtaining the second image containing the laser stripes with different brightnesses further includes:

[0025] Perform synthesis processing on each second exposure image obtained by each exposure of the second image sensor to obtain the second image.

[0026] In some embodiments, the step of confirming the distance information between the obstacle and the detection system according to the positions of the laser stripes with different brightnesses on the first image and the second image further includes:

[0027] In the imaging area of the first image, extract the data of the laser stripes along the horizontal direction through a sliding window, and perform a search and match in the Y-axis direction in the sliding window to extract the Y-axis coordinate information of the laser stripes from the second image;

[0028] Obtain the relative height between the first sensor and the laser;

[0029] According to the Y-axis coordinate information, the relative height, and the geometric principle of similar triangles, determine the distance information between the obstacle and the detection system.

[0030] To solve the above technical problems, in a second aspect, an obstacle detection device is provided in an embodiment of the present invention, which is applied to a detection system. The detection system includes a first image sensor and a laser that are arranged at the same height and adjacent to each other, and a second image sensor that is arranged above or below the first image sensor. The detection device includes:

[0031] A laser emission module for controlling the laser to emit lasers of different brightnesses;

[0032] A first acquisition module for controlling the first image sensor to acquire a first image containing laser stripes of different brightnesses, where the laser stripes of different brightnesses are laser stripes obtained by emitting lasers of different brightnesses onto an obstacle and reflecting the lasers by the obstacle;

[0033] A second acquisition module for controlling the second image sensor to acquire a second image, where the second image contains the laser stripes of different brightnesses;

[0034] A confirmation module for confirming the distance information between the obstacle and the detection system according to the positions of the laser stripes of different brightnesses on the first image and the second image.

[0035] In some embodiments, the detection device further includes:

[0036] An exposure control module for controlling the first image sensor and the second image sensor to perform at least two exposures synchronously, where

[0037] When the second sensor is arranged below the first sensor, during each of the exposures, the laser is synchronously modulated and lit, and during the process of successively lighting the laser, the duty cycle of the laser is controlled to gradually increase so that the laser emits lasers of different brightnesses;

[0038] When the second sensor is arranged above the first sensor, during each of the exposures, the laser is synchronously modulated and lit, and during the process of successively lighting the laser, the duty cycle of the laser is controlled to gradually decrease so that the laser emits lasers of different brightnesses.

[0039] To solve the above technical problems, in a third aspect, an embodiment of the present invention provides a detection system, including:

[0040] A laser;

[0041] A first image sensor, which is arranged at the same height as the laser and close to the laser;

[0042] A second image sensor, which is arranged directly above or directly below the first image sensor;

[0043] At least one processor, which is respectively connected to the laser, the first image sensor, and the second image sensor; and,

[0044] A memory communicatively connected to the at least one processor; where

[0045] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method described in the first aspect above.

[0046] In some embodiments, the laser is a line laser.

[0047] To solve the above technical problems, in a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium storing computer-executable instructions for causing a computer to execute the method described in the first aspect above.

[0048] To solve the above technical problems, in a fifth aspect, an embodiment of the present invention further provides a computer program product including a computer program stored on a computer-readable storage medium, the computer program including program instructions that, when executed by a computer, cause the computer to execute the method described in the first aspect above.

[0049] Compared with the prior art, the beneficial effects of the present invention are as follows: Different from the prior art, an embodiment of the present invention provides an obstacle detection method, device and system. The system includes a first image sensor and a laser that are arranged at the same height and adjacent to each other, and a second image sensor arranged above or below the first image sensor. The method includes: the laser emits lasers of different brightnesses, and then the first image sensor acquires a first image including laser stripes of different brightnesses, where the laser stripes of different brightnesses are laser stripes that are emitted by lasers of different brightnesses to an obstacle and reflected by the obstacle. At the same time, the second image sensor acquires a second image, and the second image includes the laser stripes of different brightnesses. Finally, according to the positions of the laser stripes of different brightnesses on the first image and the second image, the distance information between the obstacle and the detection system is confirmed. In the embodiment of the present invention, by emitting lasers of different brightnesses, the overexposure effect of the distance on the acquired laser stripes is reduced, and at the same time, the distance information of the obstacle is accurately determined by two image sensors, with low cost and good effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In one or more embodiments, exemplary illustrations are provided through pictures in the corresponding drawings. These exemplary illustrations do not constitute limitations on the embodiments. Elements / modules and steps with the same reference numerals in the drawings represent similar elements / modules and steps, unless otherwise stated, and the drawings in the drawings do not constitute a proportional limitation.

[0051] Figure 1 It is a schematic structural diagram of one application environment of the obstacle detection method provided by an embodiment of the present invention;

[0052] Figure 2 It is a schematic side view of one application environment of the obstacle detection method provided by the embodiment of the present invention;

[0053] Figure 3 It is a schematic flowchart of an obstacle detection method provided by Embodiment 1 of the present invention;

[0054] Figure 4(a) is a schematic diagram of a first image provided by Embodiment 1 of the present invention;

[0055] Figure 4(b) is a schematic diagram of a second image provided by Embodiment 1 of the present invention;

[0056] Figure 5 is Figure 3 A sub - flowchart schematic diagram of step 110 in the obstacle detection method shown;

[0057] Figure 6(a) is a schematic diagram of the imaging area included in the first image shown in Figure 4(a);

[0058] Figure 6(b) is a schematic diagram of the imaging area included in the second image shown in Figure 4(b);

[0059] Figure 7 It is a schematic diagram of the synthesis process of the first image shown in Figure 4(a);

[0060] Figure 8 It is a schematic diagram of the synthesis process of the second image shown in Figure 4(b);

[0061] Figure 9 It is a schematic diagram of the sliding window included in the first image and the second image shown in Figure 4(a) and Figure 4(b);

[0062] Figure 10 It is a schematic structural diagram of an obstacle detection device provided by the embodiment of the present invention;

[0063] Figure 11 It is a schematic hardware structure diagram of a detection system provided by the embodiment of the present invention. Detailed Embodiments

[0064] The present invention will be described in detail below with reference to specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but do not limit the present invention in any form. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can still be made. These all belong to the protection scope of the present invention.

[0065] In order to make the objectives, technical solutions and advantages of the present application more clear and understandable, the present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0066] It should be noted that if there is no conflict, the various features in the embodiments of the present invention can be combined with each other, and all are within the protection scope of the present application. In addition, although functional module division is carried out in the device schematic diagram and the logical sequence is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order from the module division in the device or the sequence in the flowchart. In addition, the terms "first", "second", etc. used herein do not limit the data and execution order, but only distinguish the same items or similar items with basically the same functions and roles.

[0067] It should be noted that when an element is expressed as "connected" to another element, it can be directly connected to the other element, or there can be one or more intermediate elements therebetween. The terms "vertical", "horizontal", "upper", "lower", "left", "right" and similar expressions used in this specification are only for the purpose of illustration. For the convenience of defining the connection structure, the present invention defines the positions of components with reference to the light-emitting direction of the laser.

[0068] Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as commonly understood by those skilled in the technical field to which the present invention belongs. The terms used in this specification in the description of the present invention are only for the purpose of describing specific embodiments and are not used to limit the present invention. The term "and / or" used in this specification includes any and all combinations of one or more of the related listed items.

[0069] In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0070] In order to solve the problem that in the existing robot navigation field, due to the different distances between different obstacles in space and the detection devices, systems or systems, when the laser is reflected back to the sensor, the laser stripes collected by the sensor are too bright or too dark, the embodiments of the present invention provide a detection method. This method is applied to a detection system. By emitting laser beams of different brightnesses, the detection system can also obtain images of laser stripes of different brightnesses through two image sensors and obtain the position information of obstacles in the current space through these images.

[0071] Figure 1 It is a schematic structural diagram of one application environment of the detection method provided by the embodiments of the present invention. Figure 2A schematic side view of the application environment, which includes: obstacle A, obstacle B, and the detection system provided by the present invention. The detection system includes a first image sensor 11, a second image sensor 12, and a laser 13. Among them, the second image sensor 12 is arranged directly above the first image sensor 11, and the laser 13 is attached to the right side (in the top view direction) of the first image sensor 11. Therefore, in Figure 1 In the top view, the second image sensor 12 is arranged overlapping with the first image sensor 11. In Figure 2 In the side view, the laser 13 is in front of the first image sensor 11 (close to the screen side).

[0072] As Figure 1 and Figure 2 shown, multiple parallel light beams emitted by the laser 13 reach obstacles at different orientations in space. The returned laser light that is reflected back can be captured by the first image sensor 11 and the second image sensor 12. And the returned light from obstacles at different distances hits the second image sensor 12 at different heights to form laser stripes at different heights. Therefore, the distance information of the obstacle relative to the detection system, that is, the depth information, can be accurately determined through the first image sensor 11 and the second image sensor 12.

[0073] Preferably, the laser 13 adopted in the embodiment of the present invention is a line laser. Therefore, this type of laser usually can only emit laser beams in one direction (such as the horizontal or vertical direction). Taking the example of emitting a horizontal laser beam at the same height as the obstacle in the embodiment of the present invention, therefore, in the embodiment of the present invention, only an example of how to detect obstacles at the same height is shown. In some other embodiments, the obstacles can also be obstacles at other positions and in other shapes, and the first image sensor 11, the second image sensor 12, and the laser 13 can also be arranged not only in the way shown in the embodiment of the present invention. Specifically, it can be set according to actual needs.

[0074] Preferably, the first image sensor 11 and the second image sensor 12 adopted in the embodiment of the present invention can be CCD or CMOS sensors. The laser band emitted by the laser 13 is visible light or infrared light. The first image sensor 11 is arranged horizontally aligned with the laser 13, and the first image sensor 11 and the second image sensor 12 are arranged vertically aligned. In some other embodiments, the first image sensor 11 and the second image sensor 12 can select corresponding image sensors according to actual needs, and do not need to be restricted by the limitations of the embodiment of the present invention.

[0075] The detection system provided by the embodiments of the present invention can be applied to any device or apparatus that needs to detect the obstacle or obstacles, etc. For example, it can be applied to a robot that needs to perform obstacle avoidance and / or mapping, etc. Further, through the information such as the shape and grayscale of the laser stripe collected by the first image sensor 11 and the second image sensor 12, the shape of the obstacle can also be determined. At the same time, the size of the obstacle can be determined in combination with the distance information. Specifically, the above functions can be realized by combining various existing shape recognition algorithms.

[0076] Specifically, with reference to the accompanying drawings, the embodiments of the present invention will be further described below.

[0077] Embodiment 1

[0078] The embodiments of the present invention provide a detection method, which is applied to a detection system. The detection system can be the detection system described in the above application scenario. The detection system includes a first image sensor and a laser that are arranged at the same height and close to each other, and a second image sensor that is arranged above or below the first image sensor. Please refer to Figure 3 , which shows the flow of a detection method provided by the embodiments of the present invention. The detection method includes but is not limited to the following steps 110-140, specifically as follows:

[0079] Step 110: The laser emits lasers with different brightnesses;

[0080] In the embodiments of the present invention, the first image sensor and the second image sensor are controlled to perform at least two exposures synchronously, and at the same time, the laser is controlled to be modulated and lit synchronously with the exposure process. Moreover, each time the laser is lit, the laser is set to emit lasers with different brightnesses. When the second sensor is arranged below the first sensor, during each exposure process, the laser is synchronously modulated and lit, and during the process of successively lighting the laser, the duty cycle of the laser is controlled to gradually increase so that the laser emits lasers with different brightnesses; when the second sensor is arranged above the first sensor, during each exposure process, the laser is synchronously modulated and lit, and during the process of successively lighting the laser, the duty cycle of the laser is controlled to gradually decrease so that the laser emits lasers with different brightnesses.

[0081] Specifically, during the process of lighting the laser, the duty cycle of the laser is controlled to increase successively to increase the brightness of the laser emitted by the laser. At the same time, the first image sensor and the second sensor perform synchronous exposure with the lighting process of the laser, so that in the above Figure 1 and Figure 2In the application scenario shown, the first image sensor 11 and the second image sensor 12 can respectively capture the first image P1 and the second image P2 shown in FIGS. 4(a) and 4(b). Among them, the three squares in the figure are obstacles A. In the first image P1, the near-distance laser stripe and the far-distance laser stripe are at the same height, which are uniformly referred to as the laser stripe L1 here; in the second image P2, the near-distance stripe L2 and the far-distance stripe L3 can be imaged at different heights. It should be noted that here the laser stripe is divided into a near-distance laser stripe and a far-distance laser stripe based on the assumption when the vertical direction of the images obtained from the first image sensor 11 and the second image sensor 12 is taken as the positive direction. If the order of the images obtained from the first image sensor 11 and the second image sensor 12 is vertically mirrored, or the first image sensor 11 and the second image sensor 12 are installed upside down or reversed, then the relative distance relationship of the obtained laser stripes is opposite / reversed. Similarly, the duty cycle of the laser also needs to be adjusted accordingly so that the far-distance laser stripe can obtain more brightness (the duty cycle increases), and the near-distance laser stripe obtains less brightness (the duty cycle decreases). Specifically, it can be set according to the actual situation and does not need to be restricted by the limitations of the embodiments of the present invention.

[0082] Further, please refer to Figure 5 , which shows a sub-process of step 110. The step of controlling the first image sensor and the second image sensor to synchronously perform at least two exposures further includes the following steps:

[0083] Step 111: Divide the photosensitive area in the second image sensor into a preset number of imaging areas from top to bottom;

[0084] Among them, the number of the imaging areas is the number of times the laser is lit, and the preset number is greater than or equal to 2. In the embodiments of the present invention, please also refer to the Figure 2 , since the obstacles are formed at different positions in the vertical direction in the second image sensor due to being at different distances, it is necessary to divide the photosensitive area in the second image sensor into a preset number of imaging areas from top to bottom, and the number of imaging areas should be the same as the number of times the laser is lit, that is, the same as the number of times the first image sensor and the second image sensor are synchronously exposed.

[0085] Step 112: Control the preset number of imaging areas in the second image sensor to be exposed sequentially from top to bottom;

[0086] In an embodiment of the present invention, generally, an image sensor includes a lens group and a photosensitive element (taking a CCD as an example in the present invention). The return light of a distant obstacle will form an image below the photosensitive area of the photosensitive element, and the return light of a nearby obstacle will form an image above the photosensitive area of the photosensitive element.

[0087] Further, the farther the distance is, usually the lower the brightness of the laser stripe captured by the image sensor. In an embodiment of the present invention, the inventor hopes that the laser stripe at a short distance will not be overexposed and the laser stripe at a long distance will not be too dark. Therefore, when controlling the imaging area in the second image sensor to be exposed sequentially from top to bottom, that is, when obtaining the laser stripe formed by the return light reflected at distances from near to far, the duty cycle of the laser is synchronously controlled to gradually increase, so that the brightness of the laser emitted by the laser gradually increases, so that the laser stripes formed by the return light at short and long distances can both be formed with normal brightness. It should be noted that the above is the case where the second sensor is arranged below the first sensor. When the second sensor is arranged above the first sensor, during each exposure process, the laser is synchronously modulated and lit, and during the process of sequentially lighting the laser, the duty cycle of the laser is controlled to gradually decrease, so that the laser emits lasers with different brightnesses.

[0088] Step 113: Confirm the imaging area in the first image sensor that can receive the emitted laser according to the height of the laser emitted by the laser.

[0089] In an embodiment of the present invention, since the laser used is a line laser, and for commonly used lidars on the market currently, they usually can only emit a row of laser points in one direction. Taking the horizontal scanning of the laser as an example in an embodiment of the present invention, therefore, the laser beam emitted by the laser is at the same height. Therefore, if all pixels of the first image sensor are exposed, only the photosensitive area within a certain height range can form an image, and this imaging area is obtained.

[0090] Step 114: Control the imaging area that can receive the emitted laser to be synchronously exposed with a preset number of imaging areas in the second image sensor.

[0091] After confirming the imaging area that can be imaged in the first image sensor, in order to improve the frame rate and reduce the amount of calculation, in the embodiments of the present invention, when controlling the exposure of the first image sensor, only the imaging area in the first image sensor can be controlled to perform synchronous exposure with the second image sensor. Thus, still taking the above application scenario as an example, the first image P1 including the imaging area P11 of the emitted laser as shown in FIGS. 6(a) and 6(b) and the second image P2 dividing each imaging area can be obtained, where each imaging area P21, P22... P2N can be obtained.

[0092] Step 120: The first image sensor acquires a first image including laser stripes with different brightnesses;

[0093] Among them, the laser stripes with different brightnesses are the laser stripes obtained by the laser with different brightnesses being emitted to the obstacle and reflected by the obstacle. In the embodiments of the present invention, the images obtained by each exposure of the first image sensor are subjected to synthesis processing to obtain the first image. For the step of performing synthesis processing, further, please refer to Figure 7 together, which shows the synthesis process of synthesizing the first image P1 in the present application, where P1-1, P1-2... P1-N are the images obtained by each exposure of the imaging area P11 of the first sensor. Specifically, the brightness of the laser stripes in each of the first exposure images P1-1, P1-2... P1-N obtained by each exposure is acquired, and different weights are assigned to the first exposure images P1-1, P1-2... P1-N according to different laser stripe brightnesses and weighted, and the weighted first exposure images are synthesized to obtain the first image P1 with uniform brightness.

[0094] Further, since the duty cycle modulated by the laser will be different each time the first image sensor is exposed, although the laser stripes at close range and long range in the images of each exposure are in the same row in terms of position, there will be differences in brightness changes. For example, the brightness change trends of the laser stripes reflected by obstacles at long range or weak reflection surfaces under different laser modulation duty cycles will be different. By comparing the brightness change trends of the laser stripes in multiple exposures, the distance or reflection ability information of the obstacle corresponding to the position of the stripe can be obtained. Combining this information, the image with a high duty cycle can be used to weight the position of the weak laser stripe to obtain an enhancement effect, while the image with a low duty cycle can be used to weight the position of the strong laser stripe to obtain a weakening effect, and finally the laser brightness of the synthesized first image tends to be uniform.

[0095] For example, after a laser stripe a reflected by a point on an obstacle is exposed 100 times, since the duty cycle of the laser is adjusted accordingly each time of exposure, 100 exposure images of the laser stripe a with different brightness levels can be obtained. Assuming the brightness value of the first exposure is 100 and the brightness value of the last (the 100th) exposure is 255, which is a relatively high brightness value, it can be determined that this is an overexposed situation. The obstacle may be a nearby obstacle and / or an obstacle with a strong reflective surface. Then, when assigning weights to this exposure image, more weights should be assigned to the brightness value obtained from the first exposure. As the number of exposure times accumulates, the weight assignment should become less and less, and the least weight is assigned to the brightness value obtained from the last exposure. Then, after weighting the brightness values obtained from each exposure, a weighted average laser stripe a is obtained. Taking the above brightness values as an example, the brightness value of the laser stripe a here should be a little more than 100 after brightness normalization, which is most appropriate. Correspondingly, if the obstacle is a distant obstacle and / or an obstacle with a weak reflective surface, after the laser stripe b reflected by a point on this obstacle is exposed in the same way, assuming the brightness value of the first exposure is 15 and the brightness value of the last (the 100th) exposure is 80, which is a relatively low brightness value, it can be determined that this is an underexposed situation, and at the same time, the distance of the obstacle and / or the reflection ability of its surface can also be determined. At this time, the least weight should be assigned to the brightness value obtained from the first exposure. As the number of exposure times accumulates, the weight assignment should become more and more, and the most weight is assigned to the brightness value obtained from the last exposure. Then, after weighting the brightness values obtained from each exposure, a weighted average laser stripe b is obtained. Taking the above brightness values as an example, the brightness value of the laser stripe b here should be around 75 after brightness normalization, which is most appropriate. Specifically, it can be set according to actual needs and there is no need to be restricted by the limitations of the above embodiments of the present invention. The first image synthesized in the embodiments of the present invention can greatly suppress the multi-path stray light problem caused by ground or obstacle reflection and improve the reduction accuracy of the acquired position information.

[0096] Step 130: The second image sensor acquires a second image including the laser stripes with different brightness levels;

[0097] In the embodiments of the present invention, the second exposure images obtained by each exposure of the second image sensor are synthesized to obtain the second image. Since the multiple exposure images obtained by the second image sensor do not overlap (each time an exposure image of an imaging area is obtained), the imaging of the laser stripe reflected by a nearby obstacle is close to the upper imaging area, and the corresponding laser duty cycle is low, while the imaging of the laser stripe reflected by a distant obstacle is close to the lower imaging area, and the corresponding laser duty cycle is high. By directly synthesizing the images of each exposure, a second image with similar and uniform laser brightness can be synthesized.

[0098] Please also refer toFigure 8 , which shows the synthesis process of synthesizing the second image P2 in this application. Among them, P2-1, P2-2... P2-N are the second exposure images obtained by each exposure of the second image sensor. By directly synthesizing the second exposure images obtained by exposing the lyrics, P2-1, P2-2... P2-N, the second image P2 can be obtained.

[0099] Step 140: According to the positions of the laser stripes with different brightnesses on the first image and the second image, confirm the distance information between the obstacle and the detection system.

[0100] In the embodiment of the present invention, specifically, in the imaging area of the first image, the data of the laser stripe is extracted along the horizontal direction through a sliding window, and a search and match are performed in the Y-axis direction in the sliding window to extract the Y-axis coordinate information of the laser stripe from the second image; obtain the relative height between the first sensor and the laser; according to the Y-axis coordinate information, the relative height, and the geometric principle of similar triangles, determine the distance information between the obstacle and the detection system. Please refer to Figure 9 , which shows an example of setting a sliding window in the exposure image in this application. Among them, the sliding window S1 is the sliding window in the imaging area of the first image P1, and the sliding window S2 is the imaging area searched by the sliding window S1 during the search and match in the Y direction.

[0101] Embodiment 2

[0102] The embodiment of the present invention provides an obstacle detection device applied to a detection system. The detection system can be the detection system described in the above application scenario. The detection system includes a first image sensor and a laser that are arranged at the same height and adjacent to each other, and a second image sensor that is arranged above or below the first image sensor. Among them, the adjacent arrangement means being arranged within a range of 0-50 mm. Please refer to Figure 10 , which shows an obstacle detection device provided by the embodiment of the present invention. The obstacle detection device 200 includes:

[0103] A laser emission module 210, configured to control the laser to emit lasers with different brightnesses;

[0104] A first acquisition module 220, configured to control the first image sensor to acquire a first image including laser stripes with different brightnesses, where the laser stripes with different brightnesses are laser stripes obtained by lasers with different brightnesses being emitted to an obstacle and reflected by the obstacle;

[0105] A second acquisition module 230, configured to control the second image sensor to acquire a second image, and the second image includes the laser stripes with different brightnesses;

[0106] A confirmation module 240, configured to confirm the distance information between the obstacle and the detection system according to the positions of the laser stripes with different brightnesses on the first image and the second image.

[0107] In some embodiments, please continue to refer to Figure 10 , the detection device 200 further includes:

[0108] An exposure control module 250, configured to control the first image sensor and the second image sensor to perform at least two exposures synchronously, wherein

[0109] When the second sensor is disposed below the first sensor, during each of the exposures, the laser is synchronously modulated and lit, and during the process of successively lighting the laser, the duty cycle of the laser is controlled to gradually increase, so that the laser emits lasers with different brightnesses;

[0110] When the second sensor is disposed above the first sensor, during each of the exposures, the laser is synchronously modulated and lit, and during the process of successively lighting the laser, the duty cycle of the laser is controlled to gradually decrease, so that the laser emits lasers with different brightnesses.

[0111] In some embodiments, the exposure control module 250 is further configured to divide the photosensitive area in the second image sensor into a preset number of imaging areas from top to bottom, wherein the number of the imaging areas is the number of times the laser is lit, and the preset number is greater than or equal to 2;

[0112] Control the preset number of imaging areas in the second image sensor to be exposed successively from top to bottom;

[0113] Confirm the imaging area in the first image sensor that can receive the emitted laser according to the height of the laser emitted by the laser;

[0114] Control the imaging area that can receive the emitted laser to be synchronously exposed with the preset number of imaging areas in the second image sensor.

[0115] In some embodiments, the first acquisition module 220 is further configured to acquire the brightness of the laser stripes in each of the first exposure images obtained by each exposure,

[0116] Assign different weights to the first exposure images according to different laser stripe brightnesses and perform weighting,

[0117] Synthesize the weighted first exposure images to obtain the first image with uniform brightness.

[0118] In some embodiments, the second acquisition module 230 is further configured to synthesize the second exposure images obtained by each exposure of the second image sensor to obtain the second image.

[0119] In some embodiments, the confirmation module 240 is further configured to extract the data of the laser stripe in the imaging area of the first image along the horizontal direction through a sliding window, and perform a search and match in the Y-axis direction in the sliding window to extract the Y-axis coordinate information of the laser stripe from the second image;

[0120] Obtain the relative height between the first sensor and the laser;

[0121] According to the Y-axis coordinate information, the relative height, and the geometric principle of similar triangles, determine the distance information between the obstacle and the detection system.

[0122] Embodiment III

[0123] An embodiment of the present invention further provides a detection system. Please refer to Figure 11 , which shows the hardware structure of the detection system capable of executing Figure 3 and Figure 5 the detection method described above. The detection system 10 may be Figure 1 and Figure 2 the detection system in the application scenarios shown. The detection system 10 includes: a laser 13; a first image sensor 11, which is arranged at the same height as the laser 13 and is close to the laser 13; a second image sensor 12, which is arranged above or below the first image sensor 11.

[0124] The detection system 10 further includes: at least one processor 15, which is respectively connected to the laser 13, the first image sensor 11, and the second image sensor 12; and a memory 16 communicatively connected to the at least one processor 15, Figure 9 Taking one processor 15 as an example. The memory 16 stores instructions executable by the at least one processor 15. The instructions are executed by the at least one processor 15 so that the at least one processor 15 can execute the above Figure 3 and Figure 5 detection method. The processor 15 and the memory 16 may be connected by a bus or other means, Figure 11 Taking connection by bus as an example.

[0125] The memory 16, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the detection method in the embodiments of the present application. For example,Figure 10 Each of the modules shown. The processor 15 executes various functional applications and data processing of the server by running the non-volatile software programs, instructions, and modules stored in the memory 16, that is, implementing the detection method in the above method embodiments.

[0126] The memory 16 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created according to the use of the detection device, etc. In addition, the memory 16 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some embodiments, the memory 16 may optionally include a memory remotely set relative to the processor 15, and these remote memories may be connected to the detection device through a network. Examples of the above network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0127] The one or more modules are stored in the memory 16 and, when executed by the one or more processors 15, execute the detection method in any of the above method embodiments. For example, execute the Figure 3 and Figure 5 method steps, and implement the functions of each module and each unit in Figure 10 .

[0128] The above product can execute the method provided in the embodiments of the present application, and has corresponding functional modules and beneficial effects for executing the method. For technical details not described in detail in this embodiment, reference may be made to the method provided in the embodiments of the present application.

[0129] The embodiments of the present application also provide a non-volatile computer-readable storage medium. The computer-readable storage medium stores computer-executable instructions, and these computer-executable instructions are executed by one or more processors. For example, execute the Figure 3 and Figure 5 method steps, and implement the functions of each module in Figure 10 .

[0130] The embodiments of the present application also provide a computer program product, including a computing program stored on a non-volatile computer-readable storage medium. The computer program includes program instructions. When the program instructions are executed by a computer, the computer executes the detection method in any of the above method embodiments. For example, execute the Figure 3 and Figure 5 method steps, and implement the functions of each module in Figure 10 .

[0131] In an embodiment of the present invention, an obstacle detection method, device and system are provided. The system includes a first image sensor and a laser that are arranged at the same height and adjacent to each other, and a second image sensor that is arranged above or below the first image sensor. The method includes: the laser emits lasers with different brightnesses, and then the first image sensor acquires a first image containing laser stripes with different brightnesses, where the laser stripes with different brightnesses are laser stripes that the lasers with different brightnesses are emitted to an obstacle and reflected by the obstacle. At the same time, the second image sensor acquires a second image containing the laser stripes with different brightnesses. Finally, according to the positions of the laser stripes with different brightnesses on the first image and the second image, the distance information between the obstacle and the detection system is confirmed. In the embodiment of the present invention, by emitting lasers with different brightnesses, the overexposure effect of the distance on the acquired laser stripes is reduced. At the same time, the distance information of the obstacle is accurately determined by two image sensors, with low cost and good effect.

[0132] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0133] Through the description of the above embodiments, those of ordinary skill in the art can clearly understand that each embodiment can be implemented by means of software plus a general hardware platform, and of course, it can also be implemented by hardware. Those of ordinary skill in the art can understand that all or part of the processes of implementing the above method embodiments can be completed by instructing relevant hardware through a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above method embodiments. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0134] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; under the idea of the present invention, the technical features in the above embodiments or different embodiments can also be combined, and the steps can be implemented in any order, and there are many other variations in different aspects of the present invention as described above. For the sake of brevity, they are not provided in detail; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An obstacle detection method, characterized in that: Applied to a detection system, the detection system includes a first image sensor and a laser arranged at the same height and adjacent to each other, and a second image sensor arranged above or below the first image sensor, the detection method includes: The laser emits lasers of different brightness; The first image sensor acquires a first image including laser stripes of different brightness, wherein the laser stripes of different brightness are laser stripes emitted by lasers of different brightness to an obstacle and reflected by the obstacle; the first image is obtained by weighted synthesis processing of first exposure images obtained by each exposure of the first image sensor according to different laser stripe brightnesses; The second image sensor acquires a second image including the laser stripes with different brightnesses, wherein the second image is obtained by synthesizing the second exposure images obtained by the second image sensor through exposure times; The distance information between the obstacle and the detection system is confirmed according to the positions of the laser stripes with different brightness on the first image and the second image.

2. The detection method according to claim 1, characterized in that: The method further comprises: Controlling the first image sensor and the second image sensor to synchronously perform at least two exposures, wherein: When the second image sensor is arranged below the first image sensor, during each exposure process, the laser is synchronously modulated and lit, and during the process of lighting the laser successively, the duty cycle of the laser is controlled to gradually increase so that the laser emits lasers of different brightness; When the second image sensor is arranged above the first image sensor, during each exposure process, the laser is synchronously modulated and lit, and in the process of successively lighting the laser, the duty cycle of the laser is controlled to gradually decrease so that the laser emits lasers of different brightness.

3. The detection method according to claim 2, characterized in that: The step of controlling the first image sensor and the second image sensor to synchronously perform at least two exposures further comprises: Dividing the photosensitive area in the second image sensor into a preset number of imaging areas from top to bottom, wherein the number of the imaging areas is the number of times the laser is lit, and the preset number is greater than or equal to 2; Controlling a preset number of imaging areas in the second image sensor to be exposed sequentially from top to bottom; According to the height of the laser light emitted by the laser, determining an imaging area in the first image sensor that can receive the emitted laser light; The imaging area capable of receiving the emitted laser light is controlled to be synchronously exposed with a preset number of imaging areas in the second image sensor.

4. The detection method according to claim 3, characterized in that: The step of acquiring a first image including laser stripes of different brightness by the first image sensor further comprises: Acquire the brightness of the laser stripes in each first exposure image obtained by each exposure, According to different laser stripe brightness, different weights are assigned to the first exposure images and weighted, The weighted first exposure images are synthesized to obtain the first image with uniform brightness.

5. The detection method according to claim 4, characterized in that: The step of confirming the distance information between the obstacle and the detection system according to the positions of the laser stripes with different brightness on the first image and the second image further includes: In the imaging area of ​​the first image, extract the data of the laser stripe in the horizontal direction through a sliding window, and perform search and matching in the Y-axis direction in the sliding window to extract the Y-axis coordinate information of the laser stripe from the second image; Acquire a relative height between the first image sensor and the laser; The distance information between the obstacle and the detection system is determined according to the Y-axis coordinate information and the relative height, and the geometric principle of similar triangles.

6. An obstacle detection device, characterized in that: Applied to a detection system, the detection system includes a first image sensor and a laser arranged at the same height and adjacent to each other, and a second image sensor arranged above or below the first image sensor, the detection device includes: A laser emission module, used to control the laser to emit lasers of different brightness; a first acquisition module, configured to control the first image sensor to acquire a first image including laser stripes of different brightnesses, wherein the laser stripes of different brightnesses are laser stripes emitted by lasers of different brightnesses to an obstacle and reflected by the obstacle; the first image is obtained by weighted synthesis processing of first exposure images obtained by each exposure of the first image sensor according to different laser stripe brightnesses; A second acquisition module, used for controlling the second image sensor to acquire a second image including the laser stripes with different brightnesses, wherein the second image is obtained by synthesizing the second exposure images obtained by the second image sensor through exposure times; A confirmation module is used to confirm the distance information between the obstacle and the detection system according to the positions of the laser stripes with different brightness on the first image and the second image.

7. The detection device according to claim 6, characterized in that: The detection device also includes: An exposure control module is used to control the first image sensor and the second image sensor to synchronously perform at least two exposures, wherein: When the second image sensor is arranged below the first image sensor, during each exposure process, the laser is synchronously modulated and lit, and during the process of lighting the laser successively, the duty cycle of the laser is controlled to gradually increase so that the laser emits lasers of different brightness; When the second image sensor is arranged above the first image sensor, during each exposure process, the laser is synchronously modulated and lit, and in the process of successively lighting the laser, the duty cycle of the laser is controlled to gradually decrease so that the laser emits lasers of different brightness.

8. A detection system, characterized in that: include: Lasers; A first image sensor is arranged at the same height as the laser and is arranged close to the laser; a second image sensor, disposed directly above or directly below the first image sensor; at least one processor connected to the laser, the first image sensor and the second image sensor respectively; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 5.

9. The detection system according to claim 8, characterized in that: The laser is a line laser.

Images