Ranging method based on line structured light, electronic equipment and storage medium

Abstract

The invention discloses a distance measurement method based on line structured light, electronic equipment and a storage medium. The distance measuring method comprises the following steps: acquiring position information of a ground reference light spot formed by linear structured light; acquiring an actual measurement image obtained in a distance measurement process based on the line structured light; and based on the position information of the ground reference light spot, the position relation of the line structure light in the actual measurement image and the position relation of the line structure light relative to the ground, screening out a real light spot participating in distance measurement calculation from the actual measurement image. Through the scheme disclosed by the invention, the position information between the ground reference light spot and the ground and the position information between the line structure light and the ground can be fully utilized in the distance measurement process, the required real light spot can be accurately screened out, and the multipath reduction of the line structure light is realized, so that the multipath interference pixel influence is effectively reduced.

Description

Technical Field

[0001] This disclosure generally relates to the field of ranging technology. More specifically, this disclosure relates to a ranging method based on line structured light, and an electronic device and a computer-readable storage medium for performing the aforementioned ranging method. Background Technology

[0002] Line structured light vision-based 3D measurement technology, with its advantages of non-contact operation, high flexibility, fast measurement speed, and high accuracy at close range, has broad application prospects in many ranging scenarios (such as robot obstacle avoidance). In robot obstacle avoidance, many measured objects have specular reflection characteristics, such as tiles, baseboards, and metal tables and chairs. When line structured light is projected onto the surface of an object with specular reflection characteristics, multiple light spots are imaged at the receiving camera, generating multipath interference. If the light spot detection mistakenly selects an interfering light spot (such as a "reflected light spot"), the reconstructed 3D point cloud data will be distorted, resulting in singular distances and failing to provide accurate obstacle avoidance information for the robot. Specifically, selecting a "reflected light spot" for reconstruction will create noise, affecting the robot's normal operation. If a near-field obstacle is formed, the robot may attempt to navigate aimlessly; if a distant obstacle is formed, the robot may collide.

[0003] In view of this, there is an urgent need to provide a ranging scheme based on line structured light in order to effectively reduce the influence of multipath interference pixels and accurately select the required real light spot. Summary of the Invention

[0004] In order to at least address one or more of the technical problems mentioned above, this disclosure proposes a ranging method, electronic device, and storage medium based on line structured light in several aspects.

[0005] In a first aspect, this disclosure provides a ranging method based on line structured light, comprising: acquiring position information of a ground reference light spot formed by the line structured light; acquiring an actual measurement image obtained during the ranging process based on the line structured light; and selecting real light spots for ranging calculation from the actual measurement image based on the position information of the ground reference light spot, the positional relationship of the line structured light in the actual measurement image, and the positional relationship of the line structured light relative to the ground.

[0006] In some embodiments, the line structured light is projected and received by a line structured light module. Obtaining the position information of the ground reference light spot formed by the line structured light includes: obtaining an image of the ground light spot formed by the line structured light actually projected onto the ground by the line structured light module, so as to obtain the position information of the ground reference light spot based on the ground light spot image; or obtaining an image of the ground light spot formed by the line structured light theoretically projected onto the ground by the line structured light module, so as to obtain the position information of the ground reference light spot based on the ground light spot image.

[0007] In some embodiments, the line structure module includes a projector for projecting line structured light and a camera for receiving the line structured light. Acquiring a ground spot image formed by the line structured light theoretically projected onto the ground by the line structured light module includes: determining a first plane equation of the line structured light in the camera system based on the installation information of the line structure module; determining the line structured light theoretically projected onto the ground by the projector based on the first plane equation and a second plane equation of the projector in the camera system; and mapping and distorting the line structured light theoretically projected onto the ground by the projector to obtain the ground spot image.

[0008] In some embodiments, selecting real light spots for ranging calculation from the actual measurement image includes: determining whether there is a multipath interference problem in the light spots in the actual measurement image based on the positional relationship of the line structured light in the actual measurement image; and in response to the existence of a multipath interference problem in the light spots in the actual measurement image, selecting real light spots for ranging calculation from the actual measurement image based on the positional information of the ground reference light spot and the positional relationship of the line structured light relative to the ground.

[0009] In some embodiments, determining whether a light spot in the actual measurement image has a multipath interference problem based on the positional relationship of the line structured light in the actual measurement image includes: determining the multipath problem of each column of light spots in the line structured light row by row, in response to the fact that the angle between the line length direction of the line structured light and the imaging of the line structured light in the actual measurement image in the vertical direction is smaller than the angle in the horizontal direction; or determining the multipath problem of each row of light spots in the line structured light column by column, in response to the fact that the angle between the line length of the line structured light and the imaging of the line structured light in the actual measurement image in the horizontal direction is smaller than the angle in the vertical direction.

[0010] In some embodiments, selecting real light spots for ranging calculation from the actual measurement image includes: determining the ground starting point in the actual measurement image based on the position information of the ground reference light spot, in response to the line length direction of the line structured light being approximately perpendicular to the ground; performing ground propagation from the ground starting point in the actual measurement image and determining whether the light spots in the actual measurement image can be connected according to the ground propagation direction; and using the light spots in the actual measurement image that can be connected according to the ground propagation direction as real light spots, or using the light spots in the actual measurement image that cannot be connected according to the ground propagation direction as interference light spots.

[0011] In some embodiments, determining whether light spots in the actual measurement image can be connected along the ground propagation direction includes: connecting the coordinate points of the centers of the light spots in the actual measurement image according to the installation direction of the projector of the line structured light; in response to the light spots in the actual measurement image being located in the ground region, the connectable light spots in the actual measurement image simultaneously satisfy the following conditions:

[0012] |u 2 -u 1 | <Thr1

[0013] |v 2 -v 1 | <Thr2

[0014]

[0015] Wherein, the center coordinates of the previous real light spot are (u 1 ,v 1 The center coordinates of the current spot to be connected are (u 2 ,v 2 Thr1, Thr2, and Thr3 represent the lateral coordinate difference threshold, longitudinal coordinate difference threshold, and direction and slope threshold of change of the centers of two connectable light spots, respectively; or, in response to the light spots being located in a non-ground region in the actual measurement image, the connectable light spots in the actual measurement image simultaneously satisfy the following conditions:

[0016] |u 2 -u 1 | <Thr4

[0017] |v 2 -v 1 | <Thr5

[0018]

[0019] Wherein, the center coordinates of the previous real light spot are (u 1 ,v 1The center coordinates of the current spot to be connected are (u 2 ,v 2 Thr4, Thr5, and Thr6 represent the threshold values ​​for the difference in the horizontal coordinates, the difference in the vertical coordinates, and the change direction and slope of the centers of two connectable light spots, respectively.

[0020] In some embodiments, the ranging method further includes: during the process of connecting the coordinate points of the center of the light spot in the actual measurement image, in response to a break in the connection of the light spot, searching for the light spot in the actual measurement image that is closest to the previous real light spot and continuing ground propagation.

[0021] In some embodiments, determining the ground starting point in the actual measurement image based on the location information of the ground reference spot includes: determining a target region of interest in the actual measurement image, searching for valid spots starting from the boundary of the target region of interest, and determining the first valid spot found as the ground starting point; or searching for valid spots starting from the boundary of the actual measurement image and determining the first valid spot found as the ground starting point; wherein the first valid spot includes a spot whose coordinate height is associated with the ground reference spot.

[0022] In some embodiments, selecting real light spots for ranging calculation from the actual measurement image includes: in response to the linear length direction of the line structured light being approximately parallel to the ground, selecting real light spots from the actual measurement image based on the relative positional relationship between the light spots in the actual measurement image and the ground reference light spot, as well as the brightness information of the light spots in the actual measurement image.

[0023] In some embodiments, selecting real light spots from the actual measurement image includes: setting a predetermined range of uplink and downlink margins for the pixel coordinates of the ground reference light spot to obtain a ground region band; determining the positional relationship of a plurality of light spots extracted from the actual measurement image relative to the ground region band; and combining the positional relationship of the plurality of light spots relative to the ground region band, selecting one real light spot from every two light spots in the plurality of light spots according to a predetermined selection strategy.

[0024] In some embodiments, the positional relationship of the plurality of light spots relative to the ground area includes near the ground, above the ground, and below the ground. Selecting a real light spot from every two light spots according to a predetermined filtering strategy includes: selecting the light spot near the ground as a real light spot in response to one light spot being near the ground and the other light spot being in a non-ground area; selecting the light spot above the ground as a real light spot in response to one light spot being above the ground and the other light spot being below the ground; selecting the light spot with greater brightness as a real light spot in response to both light spots being above or below the ground; and selecting the light spot with greater depth as a real light spot in response to both light spots being in the ground area.

[0025] In a second aspect, this disclosure provides an electronic device comprising: a line structured light module configured to project and receive line structured light for ranging; a processor; and a memory storing computer instructions for ranging based on line structured light, wherein when the computer instructions are executed by the processor, the electronic device causes the electronic device to perform the methods described according to the foregoing and the following embodiments.

[0026] In a third aspect, this disclosure provides a computer-readable storage medium containing program instructions for ranging based on line structured light, which, when executed by a processor, cause the methods described according to the foregoing and the following embodiments to be implemented.

[0027] Using the line-structured light-based ranging method, electronic device, and storage medium provided above, this embodiment of the invention filters out the actual light spot participating in the ranging calculation by utilizing the position information of the ground reference light spot formed by the acquired line-structured light, the position of the line-structured light in the actual measurement image, and the positional relationship of the line-structured light relative to the ground. Therefore, the scheme disclosed herein can fully utilize the positional information between the ground reference light spot and the line-structured light and the ground during the ranging process to accurately filter out the required actual light spot, achieving multipath reduction of the line-structured light and effectively reducing the impact of multipath interference pixels. Attached Figure Description

[0028] The above and other objects, features, and advantages of exemplary embodiments of this disclosure will become readily apparent upon reading the following detailed description with reference to the accompanying drawings. In the drawings, several embodiments of this disclosure are illustrated by way of example and not limitation, and like or corresponding reference numerals denote like or corresponding parts, wherein:

[0029] Figure 1 A flowchart illustrating a line structured light-based ranging method according to an embodiment of this disclosure is shown.

[0030] Figure 2A flowchart illustrating another embodiment of the line structured light-based ranging method disclosed herein is shown.

[0031] Figure 3 A flowchart illustrating a method for screening real light spots participating in ranging calculations according to an embodiment of this disclosure is shown.

[0032] Figure 4 A flowchart illustrating a method for screening real light spots participating in ranging calculations according to another embodiment of this disclosure is shown; and

[0033] Figure 5 A schematic block diagram of the structure of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0034] The technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, not all of them. Based on the embodiments in this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0035] It should be understood that the terms “comprising” and “including” used in this disclosure and claims indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0036] It should also be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this disclosure. As used in this disclosure and claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this disclosure and claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations.

[0037] As used in this specification and claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0038] Exemplary application scenarios

[0039] As mentioned in the background section, in ranging scenarios using line-structured light-based 3D visual measurement technology, many measured objects exhibit specular reflection. When line-structured light is projected onto the surface of an object with specular reflection, multiple light spots are imaged at the receiving camera, resulting in multipath interference. If the light spot detection mistakenly selects an interfering light spot, the recovered 3D point cloud data will be distorted, leading to singular distances and failing to provide accurate obstacle avoidance information for the robot. To address this, related technologies have proposed methods such as denoising by recovering point cloud data containing noise or outliers caused by multipath interference, or using multi-image fusion to avoid singular distances. However, in practical applications, these methods either leave the corresponding location with neither noise nor valid information after noise removal, or require multiple shots and fusions using a high dynamic range camera, which is time-consuming and increases the computational complexity.

[0040] To address the problems in the aforementioned scenarios, the inventors proposed a ranging scheme based on line structured light, which can fully utilize ground reference light spots and positional information between the line structured light and the ground to reduce the impact of multipath interference pixels, thereby accurately selecting the required real light spots.

[0041] The following combination Figures 1-5 The plan disclosed herein will be explained in detail.

[0042] Figure 1 A schematic flowchart of a line structured light-based ranging method 100 according to an embodiment of this disclosure is shown.

[0043] like Figure 1 As shown, in step S101, the position information of the ground reference light spot formed by the line structured light is acquired. Structured light is a system structure consisting of a projector and a camera. After the projector projects specific light information onto the surface of an object and the background, the camera collects the data. Based on the changes in the light signal caused by the object, the position and depth of the object are calculated, thereby reconstructing the entire three-dimensional space. Line structured light, as a type of structured light, can be understood as projecting a planar slit light or a line of light emitted by the projector.

[0044] In practical applications, the position information of the ground reference spot can be determined by obtaining the position information of the imaging spot of the theoretical online structured light on the ground.

[0045] In step S102, the actual measurement image obtained during the ranging process based on the aforementioned line structured light can be acquired. In actual ranging scenarios (e.g., obstacle avoidance scenarios), it is necessary to project line structured light onto the target to be measured and receive the reflected line structured light to form the actual measurement image.

[0046] In step S103, based on the aforementioned position information of the ground reference spot, the positional relationship of the line structured light in the actual measurement image, and the positional relationship of the line structured light relative to the ground, the real spot participating in the ranging calculation can be screened from the actual measurement image. Since the distribution of the real spot and the multipath interference spot in the image differs, the real spot is selected from the actual measurement image by using screening factors such as the position information of the reference spot, the positional relationship of the line structured light in the actual measurement image, and the positional relationship of the line structured light relative to the ground, combined with the distribution characteristics of the spot.

[0047] Therefore, in the process of ranging based on line structured light, the ground reference spot and the position information of the line structured light in the actual measurement image and between the line structured light and the ground can be fully utilized to accurately select the required real spot. This avoids the problem of losing effective information due to noise reduction or the time and energy consumption caused by multi-image fusion, effectively reduces the influence of multipath interference pixels, and thus reduces the generation of singular distances in the ranging process.

[0048] Figure 2 A flowchart illustrating another embodiment of the line structured light-based ranging method 100 disclosed herein is shown. It should be noted that... Figure 2 Method 200 in the middle can be understood as a... Figure 1 Further limitations or extensions of Chinese method 100. Therefore, the preceding text, combined with... Figure 1 The relevant descriptions also apply to the following text.

[0049] In this embodiment, it is necessary to first obtain the position information of the ground reference spot formed by the line structured light. Specifically, in step S201, a ground spot image can be obtained first, and the aforementioned ground reference spot can be obtained based on the ground spot image.

[0050] In practical applications, line structured light is projected and received by a line structured light module, which can be constructed using commonly available devices capable of transmitting and receiving line structured light. For example, a projector capable of emitting line structured light (such as a laser projector) and a camera capable of receiving line structured light can be used to construct the line structured light module. It should be noted that the disclosed solution does not limit the specific model of the line structured light module.

[0051] The aforementioned ground spot image can be formed by projecting line structured light onto the ground using a line structured light module. Specifically, in one embodiment, a ground spot image formed by the actual line structured light projected onto the ground by the aforementioned line structured light module can be acquired. For example, the line structured light module can be installed on a corresponding device (e.g., a robot), and the entire device can be placed flat on an open, level surface. The line structured light module can then be activated to project and receive line structured light, thereby acquiring a ground spot image formed by the line structured light projected onto the ground. The position information (e.g., coordinates) of the spot center can be extracted from this ground spot image as the position information of the ground reference spot. Alternatively, a ground calibration step can be added to the line structured light module manufacturing process. This involves acquiring a clean ground spot image according to the installation pose of the line structured light module within the device, and extracting the position information (e.g., coordinates) of the spot center from this ground spot image as the position information of the ground reference spot.

[0052] In other embodiments, an image of the ground light spot formed by the line structured light theoretically projected onto the ground by the line structured light module can be obtained. In this embodiment, the line structured light module specifically includes a projector for projecting line structured light and a camera for receiving line structured light. The image of the ground light spot theoretically formed by the line structured light can be calculated using the installation information (installation height, angle) of the line structured light module.

[0053] Specifically, without the ground calibration step described in the previous embodiments, this embodiment can determine the first plane equation (i.e., the ground equation) of the line structured light in the camera system based on the installation information of the line structured light module. For example, the ground equation in the camera coordinate system can be obtained according to the theoretical installation information (installation height, angle, etc.) of the line structured light module in the overall equipment. Then, based on the first plane equation and the second plane equation of the projector in the camera system (i.e., the laser surface equation, which is usually known), the line structured light theoretically projected onto the ground by the projector is determined. For example, the intersection line equation obtained by the intersection of the first and second plane equations can be calculated, and sampling on the intersection line can obtain the line structured light theoretically projected onto the ground. Then, the line structured light theoretically projected onto the ground by the projector is mapped and distorted to obtain a ground spot image. For example, the line structured light theoretically projected onto the ground can be reprojected back onto the image plane based on calibrated camera parameters, and distortion can be added to obtain a ground reference image. Specifically, the line structured light theoretically projected onto the ground can be mapped onto the two-dimensional image plane through the camera's pinhole model, and then mapped onto the pixel plane through the distortion model to obtain the aforementioned ground spot image. It should be noted that in actual ranging, the acquisition of the ground spot image can be completed during the initialization phase, that is, the ground spot image can be obtained in advance through the above process before the actual ranging begins. When the ground spot image is needed subsequently, it can be directly called to determine the position information of the ground reference spot. This avoids additional ranging time and ensures the efficiency of actual ranging.

[0054] After obtaining the position information of the ground reference spot, in step S202, the actual measurement image obtained during the ranging process can be generated based on the line structured light. For example, in an actual ranging scenario, line structured light can be projected onto the target based on the line structured light module, and the reflected line structured light can be received to form an actual measurement image based on the received line structured light.

[0055] In step S203, the presence of multipath interference in the light spots of the actual measurement image can be determined based on the positional relationship of the line structured light in the actual measurement image. Specifically, in some embodiments, in response to the fact that the angle between the line length direction of the line structured light and the imaging of the line structured light in the actual measurement image is smaller in the vertical direction than in the horizontal direction, the multipath problem of each column of light spots in the line structured light is determined row by row. In other embodiments, in response to the fact that the angle between the line length of the line structured light and the imaging of the line structured light in the actual measurement image is smaller in the horizontal direction than in the vertical direction, the multipath problem of each row of light spots in the line structured light is determined column by column. Therefore, by utilizing the positional relationship of the line structured light in the actual measurement image to determine whether it is necessary to determine the presence of multipath problems in the light spots in the image row by row or column by column, the accuracy of multipath problem determination can be effectively improved.

[0056] In step S204, in response to the multipath interference problem of light spots in the actual measurement image, based on the position information of the ground reference light spot and the positional relationship of the line structured light relative to the ground, the real light spots participating in the ranging calculation are selected from the actual measurement image. Specifically, based on the positional relationship between the line length direction of the line structured light and the ground, different methods (method 1 or method 2) can be used to select the real light spots participating in the ranging calculation from the actual measurement image.

[0057] Figure 3 An exemplary flowchart of a method 300 (i.e., method 1) for filtering out real light spots from actual measured images to participate in ranging calculations is shown.

[0058] like Figure 3 As shown, in step S301, in response to the linear length direction of the line structured light being nearly perpendicular to the ground, the starting point of the ground in the actual measurement image can be determined based on the position information of the ground reference light spot.

[0059] For example, a target region of interest (ROI) can be identified in the actual measured image, and valid light spots can be searched starting from the boundary of the ROI. The first valid light spot found is then designated as the ground starting point. The ROI in the actual measured image can be determined in several ways. For instance, when the consistency of the line structured light modules is good, a batch of ground spot images captured by the line structured light modules can be observed. Then, the ROI of the light spots hit by the line structured light modules on the ground can be identified, and this identified ROI is used as the ROI in the actual measured image. Alternatively, the ROI can be defined in real time when the actual measured image is obtained.

[0060] For example, valid light spots can be searched starting from the boundary of the actual measured image, and the first valid light spot found can be determined as the ground starting point. In this embodiment, after obtaining the ground light spot image through ground calibration or theoretical calculation, the position information of the ground reference light spot can be determined based on the ground light spot image. Subsequently, after obtaining the actual measured image, valid light spots can be searched directly from the boundary of the actual measured image, and the first valid light spot found can be determined as the ground starting point. The first valid light spot in this disclosed scheme includes a light spot whose coordinates are highly correlated with those of the ground reference light spot, such as the first valid light spot in the actual measured image whose coordinates are consistent with those of the ground reference light spot, or the first valid light spot in the actual measured image whose coordinates are within a predetermined range from those of the ground reference light spot.

[0061] In step S302, ground propagation can begin from the ground starting point in the actual measurement image, and it can be determined whether the light spots in the actual measurement image can be connected according to the ground propagation direction. Specifically, the coordinates of the center points of the light spots in the actual measurement image are connected according to the installation direction of the line structured light projector. In response to the fact that the light spots in the actual measurement image are located in the ground region, the connectable light spots in the actual measurement image simultaneously satisfy the following conditions:

[0062] |u 2 -u 1 | <Thr1

[0063] |v 2 -v 1 | <Thr2

[0064]

[0065] Wherein, the center coordinates of the previous real light spot are (u 1 ,v 1 The center coordinates of the current spot to be connected are (u 2 ,v 2 Thr1, Thr2, and Thr3 represent the threshold values ​​for the horizontal coordinate difference, vertical coordinate difference, and direction and slope of change of the centers of two connectable light spots, respectively. Thr1, Thr2, and Thr3 can be set according to actual application requirements.

[0066] Alternatively, in response to the fact that the light spot in the actual measurement image is located in a non-ground region, the connectable light spots in the actual measurement image simultaneously satisfy the following conditions:

[0067] |u 2 -u 1 | <Thr4

[0068] |v 2 -v 1 | <Thr5

[0069]

[0070] Wherein, the center coordinates of the previous real light spot are (u 1 ,v 1 The center coordinates of the current spot to be connected are (u 2 ,v 2 Thr4, Thr5, and Thr6 represent the threshold values ​​for the lateral coordinate difference, longitudinal coordinate difference, and direction and slope of change of the centers of two connectable light spots, respectively. Thr4, Thr5, and Thr6 can be set according to actual application requirements.

[0071] Furthermore, in some embodiments, during the process of connecting the coordinate points of the center of the light spot in the actual measurement image, in response to the break in the connection of the light spot, the light spot closest to the previous real light spot in the actual measurement image is searched to continue ground propagation, and the above method is used to continue to determine whether the light spots in the actual measurement image can be connected according to the ground propagation direction.

[0072] In step S303, the light spots that can be connected in the ground propagation direction in the actual measurement image can be taken as real light spots, or the light spots that cannot be connected in the ground propagation direction in the actual measurement image can be taken as interference light spots.

[0073] Figure 4 An exemplary flowchart of another method 400 (i.e., method 2) for filtering out real light spots from actual measured images to participate in the ranging calculation is shown.

[0074] In this embodiment, since the line length direction of the structured light is nearly parallel to the ground, the real light spot can be selected from the actual measurement image based on the relative positional relationship between the light spot in the actual measurement image and the ground reference light spot, as well as the brightness information of the light spot itself in the actual measurement image.

[0075] Specifically, such as Figure 4 As shown, in step S401, the pixel coordinates of the ground reference spot can be set with a predetermined range of up and down row number margins to obtain a ground region band. Specifically, a certain number of row number margins are set above and below the ground reference spot to obtain an upper boundary b1 and a lower boundary b2; the area between these two boundaries is called the "ground region band".

[0076] In step S402, the positional relationship of the multiple light spots extracted from the actual measurement image relative to the ground area can be determined. In this embodiment, the positional relationship of the multiple light spots relative to the ground area includes near the ground, above the ground, and below the ground. "Near the ground" means the light spot coordinates are within the ground area, "above the ground" means the light spot coordinates are above the upper boundary of the ground area, and "below the ground" means the light spot coordinates are below the lower boundary of the ground area.

[0077] In step S403, considering the positional relationships of multiple light spots relative to the ground area, a real light spot is selected from every two light spots according to a predetermined screening strategy. Specifically, for the multiple light spots, pairwise comparisons are required to select the real light spot. The process of selecting a real light spot from every two light spots according to the predetermined screening strategy is as follows:

[0078] In response to the condition that one of the two light spots is located near the ground and the other is located in a non-ground area, the light spot located near the ground is selected as the real light spot; in response to the condition that one of the two light spots is located above the ground and the other is located below the ground, the light spot located above the ground is selected as the real light spot; in response to the condition that both light spots are located above or below the ground, the light spot with the greater brightness is selected as the real light spot; and in response to the condition that both light spots are located in the ground area, the light spot with greater depth is selected as the real light spot.

[0079] In practical applications, the line length direction of the structured light may lie between the vertical and horizontal directions (e.g., an angle of approximately 45°). In this case, the line length direction of the structured light can be considered to be closer to being perpendicular to the ground or closer to being horizontal to the ground. The aforementioned... Figure 3 The method described in 300 or Figure 4 The method 400 shown is used to filter out the real light spots that participate in the distance measurement calculation from the actual measured image.

[0080] Furthermore, after filtering out the real light spots from the actual measured images, the pixel coordinates of the real light spots can be retained, and the three-dimensional point cloud information of the target to be measured (such as a real obstacle) can be calculated using the principle of triangulation.

[0081] For example, the pixel coordinates of the real light spot after distortion removal can be defined as (u, v). According to the principle of triangulation, the corresponding point cloud (x) can be obtained. c ,y c ,z c ):

[0082]

[0083] Where f is the camera focal length, c x ,c y These are the coordinates of the camera's principal point, d x ,d y , where are the physical dimensions of the pixel in the horizontal and vertical directions, respectively, and a, b, c, d are the light plane parameters of the projector in the line structure module of the camera system.

[0084] Therefore, the disclosed solution can effectively utilize ground reference spots to assist in determining whether the spots in the actual measurement image are multipath interference pixels, reducing the generation of singular distances during ranging. The disclosed solution has good universality, capable of selecting true spots in scenarios with one or multiple reflections, thereby reconstructing the correct point cloud. The transmission and reception of the line structured light are performed by the line structured light module. For devices requiring ranging based on line structured light, the line structured light module is a basic configuration; therefore, the implementation of the entire solution requires no additional hardware, thus incurring no additional costs. Furthermore, the implementation of the disclosed solution does not require multiple shots, and may even require only one shot (the location information of the ground reference spot can be obtained through ground calibration during the production stage of the line structured light module, or theoretically calculated during the initialization stage based on the installation information of the line structured light module; in practical applications, only the actual measurement image needs to be captured) to achieve the selection of true spots. In addition, the results theoretically calculated based on the installation information of the line structured light module show good consistency with the results obtained from actual image calibration, saving calibration steps in the line structured light module generation process and improving the generation efficiency of the line structured light module.

[0085] Figure 5 A schematic block diagram of an electronic device 500 according to an embodiment of the present disclosure is shown. It should be noted that the electronic device in this embodiment may include any device that requires distance measurement based on line structured light (e.g., a robotic vacuum cleaner). Specifically, as follows... Figure 5 As shown, the electronic device 500 may include a line structured light module 501, a processor 502, and a memory 503. The line structured light module 501 is configured to project and receive line structured light for ranging. For example, the line structured light module 501 may include a projector (e.g., a laser projector) for projecting the line structured light and a camera for receiving the line structured light. The memory 503 stores computer instructions for ranging based on line structured light. When the computer instructions are executed by the processor 502, the electronic device 500 performs the following: acquiring the position information of the ground reference light spot formed by the line structured light; acquiring the actual measurement image obtained during the ranging process based on the line structured light; and, based on the position information of the ground reference light spot, the positional relationship of the line structured light in the actual measurement image, and the positional relationship of the line structured light relative to the ground, selecting the actual light spot for ranging calculation from the actual measurement image.

[0086] Optionally, the line structured light is projected and received by a line structured light module. Obtaining the position information of the ground reference light spot formed by the line structured light includes: obtaining an image of the ground light spot formed by the line structured light actually projected onto the ground by the line structured light module, so as to obtain the position information of the ground reference light spot based on the ground light spot image; or obtaining an image of the ground light spot formed by the line structured light theoretically projected onto the ground by the line structured light module, so as to obtain the position information of the ground reference light spot based on the ground light spot image.

[0087] Optionally, the line structure module includes a projector for projecting line structured light and a camera for receiving line structured light. Acquiring a ground spot image formed by the line structured light theoretically projected onto the ground by the line structured light module includes: determining a first plane equation of the line structured light in the camera system based on the installation information of the line structure module; determining the line structured light theoretically projected onto the ground by the projector based on the first plane equation and a second plane equation of the projector in the camera system; and mapping and distorting the line structured light theoretically projected onto the ground by the projector to obtain the ground spot image.

[0088] Optionally, selecting the real light spot for ranging calculation from the actual measurement image includes: determining whether there is a multipath interference problem in the light spot in the actual measurement image based on the positional relationship of the line structured light in the actual measurement image; and in response to the existence of a multipath interference problem in the light spot in the actual measurement image, selecting the real light spot for ranging calculation from the actual measurement image based on the positional information of the ground reference light spot and the positional relationship of the line structured light relative to the ground.

[0089] Optionally, determining whether there is a multipath interference problem in the actual measurement image based on the positional relationship of the line structured light in the actual measurement image includes: in response to the fact that the angle between the line length direction of the line structured light and the imaging of the line structured light in the actual measurement image is smaller in the vertical direction than in the horizontal direction, determining the multipath problem of each column of light spots in the line structured light row by row; or in response to the fact that the angle between the line length of the line structured light and the imaging of the line structured light in the actual measurement image is smaller in the horizontal direction than in the vertical direction, determining the multipath problem of each row of light spots in the line structured light column by column.

[0090] Optionally, selecting the real light spots for ranging calculation from the actual measurement image includes: in response to the line length direction of the line structured light being approximately perpendicular to the ground, determining the ground starting point in the actual measurement image based on the position information of the ground reference light spot; starting ground propagation from the ground starting point in the actual measurement image, and determining whether the light spots in the actual measurement image can be connected according to the ground propagation direction; and taking the light spots in the actual measurement image that can be connected according to the ground propagation direction as real light spots, or taking the light spots in the actual measurement image that cannot be connected according to the ground propagation direction as interference light spots.

[0091] Optionally, determining whether the light spots in the actual measurement image can be connected according to the ground propagation direction includes: connecting the coordinate points of the centers of the light spots in the actual measurement image according to the installation direction of the projector of the line structured light; in response to the light spots in the actual measurement image being located in the ground area, the connectable light spots in the actual measurement image simultaneously satisfy the following conditions:

[0092] |u 2 -u 1 | <Thr1

[0093] |v 2 -v 1 | <Thr2

[0094]

[0095] Wherein, the center coordinates of the previous real light spot are (u 1 ,v 1 The center coordinates of the current spot to be connected are (u 2 ,v 2 Thr1, Thr2, and Thr3 represent the threshold values ​​for the lateral coordinate difference, longitudinal coordinate difference, and direction and slope of change of the centers of two connectable light spots, respectively; or

[0096] In response to the fact that the light spot in the actual measurement image is located in a non-ground region, the connectable light spots in the actual measurement image simultaneously satisfy the following conditions:

[0097] |u 2 -u 1 | <Thr4

[0098] |v 2 -v 1 | <Thr5

[0099]

[0100] Wherein, the center coordinates of the previous real light spot are (u 1 ,v1 The center coordinates of the current spot to be connected are (u 2 ,v 2 Thr4, Thr5, and Thr6 represent the threshold values ​​for the difference in the horizontal coordinates, the difference in the vertical coordinates, and the change direction and slope of the centers of two connectable light spots, respectively.

[0101] Optionally, the ranging method further includes: during the process of connecting the coordinate points of the center of the light spot in the actual measurement image, in response to the break in the connection of the light spot, searching for the light spot in the actual measurement image that is closest to the previous real light spot and continuing to propagate on the ground.

[0102] Optionally, determining the ground starting point in the actual measurement image based on the position information of the ground reference spot includes: determining a target region of interest in the actual measurement image, searching for valid spots starting from the boundary of the target region of interest, and determining the first valid spot found as the ground starting point; or searching for valid spots starting from the boundary of the actual measurement image and determining the first valid spot found as the ground starting point; wherein the first valid spot includes a spot whose coordinate height is associated with the ground reference spot.

[0103] Optionally, selecting the real light spot for distance calculation from the actual measurement image includes: in response to the linear length direction of the line structured light being nearly parallel to the ground, selecting the real light spot from the actual measurement image based on the relative positional relationship between the light spot in the actual measurement image and the ground reference light spot, as well as the brightness information of the light spot itself in the actual measurement image.

[0104] Optionally, selecting real light spots from the actual measured image includes: setting a predetermined range of uplink and downlink margins for the pixel coordinates of the ground reference light spot to obtain a ground region band; determining the positional relationship of multiple light spots extracted from the actual measured image relative to the ground region band; and combining the positional relationship of the multiple light spots relative to the ground region band, selecting one real light spot from every two light spots according to a predetermined selection strategy.

[0105] Optionally, the positional relationship of the plurality of light spots relative to the ground area includes near the ground, above the ground, and below the ground. Selecting a true light spot from every two light spots according to a predetermined filtering strategy includes: selecting the light spot near the ground as the true light spot in response to one light spot being near the ground and the other light spot being in a non-ground area; selecting the light spot above the ground as the true light spot in response to one light spot being above the ground and the other light spot being below the ground; selecting the light spot with greater brightness as the true light spot in response to both light spots being above or below the ground; and selecting the light spot with greater depth as the true light spot in response to both light spots being in the ground area.

[0106] Through the above implementation methods, the electronic device can make full use of the ground reference spot and the position information of the line structured light in the actual measurement image and between the line structured light and the ground to accurately select the required real spot during the distance measurement process based on line structured light. This avoids the problem of losing effective information due to noise reduction or the time and energy consumption caused by multi-image fusion, effectively reduces the influence of multipath interference pixels, and thus reduces the generation of singular distances during the distance measurement process.

[0107] It should be noted that the specific details of the operating method and steps of this electronic device are combined with the foregoing. Figures 1-4 The specific implementation methods described are the same or similar, so they will not be elaborated here.

[0108] Furthermore, this disclosure also provides a computer-readable storage medium storing program instructions configured to execute at runtime. Figure 1 and Figure 2 The distance measurement method based on line structured light shown and Figure 3 and Figure 4 The method shown is for selecting the actual light spot to participate in the distance measurement calculation.

[0109] Specifically, in this embodiment, the storage medium may include, but is not limited to, USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks, and other media capable of storing computer programs.

[0110] While numerous embodiments of this disclosure have been shown and described herein, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Many modifications, alterations, and alternatives will occur to those skilled in the art without departing from the spirit and intent of this disclosure. It should be understood that various alternatives to the embodiments of this disclosure described herein may be employed in the practice of this disclosure. The appended claims are intended to define the scope of this disclosure and therefore cover equivalents or alternatives within the scope of these claims.

Claims

1. A ranging method based on line structured light, characterized in that, include: Obtain the position information of the ground reference spot formed by the line structured light; Acquire the actual measurement image obtained during the ranging process based on the line structured light; as well as Based on the position information of the ground reference spot, the positional relationship of the line structured light in the actual measurement image, and the positional relationship of the line structured light relative to the ground, the real spot for participating in the distance measurement calculation is selected from the actual measurement image.

2. The ranging method according to claim 1, characterized in that, in, The line structured light is projected and received by a line structured light module, and the position information of the ground reference light spot formed by the line structured light is obtained, including: Obtain a ground spot image formed by the line structured light actually projected onto the ground by the line structured light module, and obtain the position information of the ground reference spot based on the ground spot image; or The ground light spot image formed by the line structured light theoretically projected onto the ground by the line structured light module is obtained, so as to obtain the position information of the ground reference light spot based on the ground light spot image.

3. The ranging method according to claim 2, characterized in that, The line structure module includes a projector for projecting line structured light and a camera for receiving the line structured light. Acquiring an image of the ground light spot formed by the line structured light theoretically projected onto the ground by the line structured light module includes: Based on the installation information of the line structure module, determine the first plane equation of the line structured light in the camera system; Based on the first plane equation and the second plane equation of the projector in the camera system, the line structured light theoretically projected onto the ground by the projector is determined; and The line structured light theoretically projected onto the ground by the projector is mapped and distorted to obtain the ground light spot image.

4. The ranging method according to claim 1, characterized in that, The actual light spots selected from the actual measured images for ranging calculation include: Based on the positional relationship of the line structured light in the actual measured image, determine whether there is multipath interference in the light spot in the actual measured image; and In response to the multipath interference problem of light spots in the actual measurement image, based on the position information of the ground reference light spot and the positional relationship of the line structured light relative to the ground, the real light spots participating in the distance measurement calculation are selected from the actual measurement image.

5. The ranging method according to claim 4, characterized in that, Determining whether the light spot in the actual measurement image has multipath interference based on the positional relationship of the line structured light in the actual measurement image includes: In response to the fact that the angle between the line length direction of the line structured light and the imaging of the line structured light in the actual measured image is smaller in the vertical direction than in the horizontal direction, the multipath problem of each column of light spots in the line structured light is determined row by row; or In response to the fact that the angle between the line length of the line structured light and the imaging of the line structured light in the actual measured image is smaller in the horizontal direction than in the vertical direction, the multipath problem of each row of light spots in the line structured light is determined column by column.

6. The ranging method according to claim 4, characterized in that, The actual light spots selected from the actual measured images for ranging calculation include: In response to the fact that the line length direction of the structured light is approximately perpendicular to the ground, the starting point of the ground in the actual measurement image is determined based on the position information of the ground reference light spot; Ground propagation begins from the starting point on the ground in the actual measured image, and it is determined whether the light spots in the actual measured image can be connected according to the ground propagation direction; and The light spots that can be connected in the ground propagation direction in the actual measured image are taken as real light spots, or the light spots that cannot be connected in the ground propagation direction in the actual measured image are taken as interference light spots.

7. The ranging method according to claim 6, characterized in that, Determining whether the light spots in the actual measured image can be connected according to the ground propagation direction includes: Connect the coordinates of the center of the light spot in the actual measured image according to the installation direction of the projector of the line structured light; In response to the fact that the light spot in the actual measurement image is located in the ground area, the connectable light spots in the actual measurement image simultaneously satisfy the following conditions: |in 2 -in 1 | <Thr1 |v 2 -v 1 |<Thr2 Wherein, the center coordinates of the previous real light spot are (u 1 ,v 1 The center coordinates of the current spot to be connected are (u 2 ,v 2 Thr1, Thr2, and Thr3 represent the threshold values ​​for the lateral coordinate difference, longitudinal coordinate difference, and direction and slope of change of the centers of two connectable light spots, respectively; or In response to the fact that the light spot in the actual measurement image is located in a non-ground region, the connectable light spots in the actual measurement image simultaneously satisfy the following conditions: |in 2 -in 1 | <Thr4 |v 2 -v 1 | <Thr5 Wherein, the center coordinates of the previous real light spot are (u 1 ,v 1 The center coordinates of the current spot to be connected are (u 2 ,v 2 Thr4, Thr5, and Thr6 represent the threshold values ​​for the difference in the horizontal coordinates, the difference in the vertical coordinates, and the change direction and slope of the centers of two connectable light spots, respectively.

8. The ranging method according to claim 7, characterized in that, The ranging method further includes: During the process of connecting the coordinates of the center of the light spot in the actual measurement image, in response to the break in the connection of the light spot, the light spot that is closest to the previous real light spot is found in the actual measurement image to continue ground propagation.

9. The ranging method according to claim 6, characterized in that, Determining the ground starting point in the actual measurement image based on the position information of the ground reference spot includes: Determine the target region of interest in the actual measured image, and start searching for effective light spots from the boundary of the target region of interest, and determine the first effective light spot found as the ground starting point; or The search for effective light spots begins at the boundary of the actual measured image, and the first effective light spot found is determined as the ground starting point. The first effective light spot includes a light spot whose coordinate height is correlated with that of the ground reference light spot.

10. The ranging method according to claim 4, characterized in that, The actual light spots selected from the actual measured images for ranging calculation include: In response to the fact that the line length direction of the structured light is nearly parallel to the ground, the real light spot is selected from the actual measurement image based on the relative positional relationship between the light spot in the actual measurement image and the ground reference light spot, as well as the brightness information of the light spot in the actual measurement image.

11. The ranging method according to claim 10, characterized in that, The process of filtering out real light spots from the actual measured images includes: The pixel coordinates of the ground reference spot are set with a predetermined range of uplink and downlink margins to obtain the ground area band; Determine the positional relationship of multiple light spots extracted from the actual measured image relative to the ground region; and Based on the positional relationship of the multiple light spots relative to the ground area, and according to a predetermined screening strategy, a real light spot is selected from every two light spots among the multiple light spots.

12. The ranging method according to claim 11, characterized in that, The positional relationship of the plurality of light spots relative to the ground area includes near the ground, above the ground, and below the ground. Selecting a true light spot from every two light spots according to a predetermined filtering strategy includes: In response to the condition that one of the two light spots is located near the ground and the other light spot is located in a non-ground area, the light spot located near the ground is selected as the real light spot; In response to the condition that one of the two light spots is above the ground and the other light spot is below the ground, the light spot above the ground is selected as the real light spot; In response to the condition that every two light spots are simultaneously above or below the ground, the brighter light spot is selected as the real light spot; and In response to the fact that every two light spots are located in the ground area at the same time, the light spot with the greater depth is selected as the real light spot.

13. An electronic device, characterized in that, Also includes: A line structured light module configured to project and receive line structured light for ranging; processor; as well as A memory storing computer instructions for ranging based on line structured light, which, when executed by the processor, cause the electronic device to perform the method according to any one of claims 1 to 12.

14. A computer-readable storage medium, characterized in that, It includes program instructions for ranging based on line structured light, which, when executed by a processor, cause the method according to any one of claims 1 to 12 to be implemented.

Images