Method for measuring three-dimensional information of object, controller, and system

By projecting a surface array of light onto the area of ​​the object being measured and recording the set of pixel coordinates, combined with a list of light stripe mappings, the problem of difficult light stripe matching in line structured light measurement is solved, and efficient and accurate three-dimensional information measurement is achieved.

WO2024235028A1PCT designated stage expired Publication Date: 2024-11-21HUAWEI TECH CO LTD
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
PCT/CN2024/091135
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-17
Filing Date
2024-05-06
Publication Date
2024-11-21

AI Technical Summary

Technical Problem

When using line structured light for 3D information measurement, the matching of light stripes is difficult to judge accurately due to factors such as differences in surface depth, low reflectivity, and specular reflection of the object being measured, leading to incorrect 3D measurement results.

Method used

By projecting a coverage array of light onto multiple areas of the object under test, recording the set of pixel coordinates and the region mapping relationship, capturing the reflected light of the line light using a camera, and determining the projection parameters of the line light based on the pixel coordinates and mapping list of the light stripes, the projection parameters of the line light are ensured to match the correct projection parameters of the light stripes.

Benefits of technology

It improves the efficiency and accuracy of 3D measurement, and can correctly identify the projection parameters of light stripes in complex scenes, ensuring the accuracy of 3D information calculation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2024091135_21112024_PF_FP_ABST
    Figure CN2024091135_21112024_PF_FP_ABST
Patent Text Reader

Abstract

A method for measuring three-dimensional information of an object, a controller, and a system, applicable to the technical field of optical measurement. The measurement method comprises: projecting area array light to different regions among a plurality of regions of a measured object by means of a first light projector among at least one light projector (301); by means of a camera (220), capturing light obtained by reflecting the area array light by each region to obtain a pixel coordinate set, and recording the mapping relationship between the pixel coordinate set and the region to obtain a mapping list (302); respectively projecting line light to at least two regions among the plurality of regions by means of a second light projector among the at least one light projector (303); by means of the camera (220), capturing light obtained by reflecting the line light by each region to obtain a light stripe (304); and, on the basis of pixel coordinates of the light stripe and the mapping relationship, obtaining a projection parameter of the line light corresponding to the light stripe, and calculating three-dimensional information of the measured object (305). The measurement method allows for matching of correct projection parameters to light stripes corresponding to a plurality of pieces of line light projected simultaneously.
Need to check novelty before this filing date? Find Prior Art

Description

Method, controller and system for measuring three-dimensional information of objects

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on May 17, 2023, with application number 202310563191.1 and application name “Method, controller and system for measuring three-dimensional information of objects”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of optical measurement technology, and in particular to a method, controller, and system for measuring three-dimensional information of an object. Background Art

[0003] Measuring three-dimensional information about objects is a crucial component of technologies such as 3D reconstruction and flatness measurement. Structured light, an active, non-contact, high-precision 3D vision measurement technology, is widely used in industrial and consumer applications.

[0004] Based on the pattern projected during structured light measurement, structured light can be divided into point structured light, line structured light, and area array structured light. Compared with point structured light and area array structured light, line structured light has both high detection accuracy and high detection efficiency. Therefore, line structured light is more widely used.

[0005] To improve the measurement efficiency of line structured light, multiple lines of light are projected simultaneously onto the object being measured. However, due to large depth variations, low reflectivity, and mirror reflections on the surface of the object being measured, the light streaks captured by the camera, generated by the reflections from the object's surface, may be missing or intersecting. This makes it difficult to determine which line of light the streaks correspond to, making it difficult to match the light streaks with the correct projection parameters, leading to erroneous 3D measurement results.

[0006] Summary of the Invention

[0007] The embodiments of the present application provide a method, controller, and system for measuring three-dimensional information of an object, which can match the light stripes of multiple line lights projected simultaneously to the correct projection parameters.

[0008] In a first aspect, a method for measuring three-dimensional information of an object is provided, which is applied to a controller in a measurement system, and the measurement system also includes at least one light projector and a camera; the method includes: projecting a plane array light to different areas of a plurality of areas of the object to be measured by a first light projector among the at least one light projector; wherein the plane array light projected to each area can cover the area; capturing the reflected light of each area to the plane array light by a camera to obtain a set of pixel coordinates, and recording the mapping relationship between the set of pixel coordinates and the area to obtain a mapping list; projecting a line light to at least two areas of the plurality of areas by a second light projector among the at least one light projector; wherein the projection position of the line light on the object to be measured belongs to a subset of the projection area of ​​the plane array light on the object to be measured; capturing the reflected light of the area to the line light by the camera to obtain light stripes; obtaining the projection parameters of the line light corresponding to the light stripes based on the pixel coordinates of the light stripes and the above-mentioned mapping relationship, and calculating the three-dimensional information of the object to be measured.

[0009] Specifically, projecting the line light to the at least two areas respectively may be projecting a line light to each of the at least two areas.

[0010] A region is a block-shaped area with an area. Specifically, if the surface of the object being measured is relatively flat, the region may be relatively flat and can be called a planar region. If the surface of the object being measured is uneven, the region may be an uneven area.

[0011] The area array light covering an area refers to the light that can illuminate or reach all or most areas in the area. In other words, the area array light covering an area refers to the light that can illuminate or reach all or most points in the area.

[0012] In this method, a planar array of light covering an area is projected onto the area. This allows a camera to capture the light reflected from all or most points in the area, obtaining pixels corresponding to all or most points in the area and recording the coordinates of these pixels. This allows the locations of the pixels corresponding to the light reflected from the area to be recorded, thereby marking the area. After multiple areas have been marked, a line light can be simultaneously projected onto at least two of the areas to simultaneously scan the at least two areas. The camera can capture the light reflected from the object under test from the line light, obtaining light streaks. The projected position of the line light on the object under test belongs to a subset of the projected area of ​​the planar array light on the object under test; that is, the projected position of the line light is contained within the projected area of ​​the planar array light. Accordingly, the coordinates of the pixels reflected from the line light by the object under test are also contained within the coordinates of the pixels reflected from the corresponding planar array light by the object under test. A mapping list records the mapping relationship between pixel sets and areas. Based on the pixel coordinates of the light streak and the mapping list, the projection parameters of the line light corresponding to the light streak can be obtained, thereby matching the light streak with the correct projection parameters.

[0013] In short, this method can scan at least two areas simultaneously, has high measurement efficiency, and can match the correct projection parameters for the light stripes corresponding to each line of light, ensuring the accuracy of the three-dimensional measurement results.

[0014] In one possible implementation, obtaining projection parameters of the line light corresponding to the light streak based on the pixel coordinates of the light streak and a mapping relationship includes: identifying a target pixel coordinate set containing the pixel coordinates of the light streak; obtaining a target area mapped by the target pixel coordinate set based on a mapping list; and confirming that the projection parameters of the line light projected onto the target area are the projection parameters of the line light corresponding to the light streak.

[0015] As described above, the pixel coordinates of the light reflected from the object under test by the line light are included in the pixel coordinates of the light reflected from the corresponding area light array. Therefore, the pixel coordinates of the light streak are included in the set of pixels obtained by reflecting the area light array from a certain area. In this way, the pixel coordinate set containing the pixel coordinates of the light streak can be identified. The area to which this pixel coordinate set is mapped is the area projected by the line light corresponding to the light streak. A mapping list records the mapping relationship between areas and pixel coordinate sets. After obtaining the pixel coordinate set containing the pixel coordinates of the light streak, the mapping list can be used to query the area to which this pixel coordinate set is mapped. This area is the area projected by the line light corresponding to the light streak, thereby obtaining the projection parameters of the line light corresponding to the light streak.

[0016] In a possible implementation, at least one region is spaced between two adjacent regions of the at least two regions, and the at least one region belongs to a region other than the at least two regions among the multiple regions.

[0017] In this implementation, there is at least one other area between the two areas closest to each other (i.e., two adjacent areas) in at least two areas where line light is projected simultaneously, thereby enlarging the projection positions of the two simultaneously projected line lights, as well as the distance between the projection position of the line light and the adjacent area of ​​the area where the line light is projected. Accordingly, the distance between the light stripes obtained by the camera capturing the reflected light of the line light at the projection position is also enlarged, as is the distance between the pixel coordinates in the light stripe and the pixel coordinates corresponding to the adjacent area of ​​the area where the line light corresponding to the light stripe is projected. Therefore, even in the event of blurred light stripes due to problems such as camera defocusing, the area to which the light stripe belongs can be correctly distinguished, thereby matching the correct projection parameters for the stripe.

[0018] In a possible implementation, the first light projector and the second light projector are the same projector.

[0019] In this implementation, the same light projector can be used to project both the area array and line light, simplifying the measurement system's hardware configuration, reducing measurement costs, and reducing the system's size. In particular, the use of the same light projector naturally ensures consistency in projection direction for both the area array and line light, eliminating the need for calibration.

[0020] In a possible implementation, the first light projector and the second light projector are different projectors located on the same side of the object to be measured.

[0021] In this implementation, the light projector that projects the area array light and the light projector that projects the line light can be different, so that a suitable light projector can be selected according to the different characteristics of the area array light and the line light to improve the projection effect of the area array light and the line light.

[0022] In a possible implementation, projecting the area array light to different areas among the multiple areas of the object to be measured includes: projecting the area array light to each of the multiple areas in sequence.

[0023] In this implementation, each area can be marked using a time-division method. That is, the area array light is projected onto one area at a time, while different areas are projected at different times. This allows the camera to capture the reflection of the area array light from one area without interference from reflections from other areas. This allows the camera to accurately capture the reflection of the area array light from that area and accurately record the pixel coordinates. Furthermore, this method is simple and easy to implement.

[0024] In a possible implementation, projecting the area array light to different areas among the multiple areas of the object to be measured includes: simultaneously projecting the area array light to different areas among the multiple areas, wherein the area array lights projected to different areas are different.

[0025] In this implementation, different areas can be marked by simultaneously projecting multiple different area array lights, and marking of multiple areas can be achieved in parallel, thereby improving the efficiency of area marking and further improving the efficiency of three-dimensional measurement.

[0026] In one possible implementation, projecting a plane array light onto different areas of a plurality of areas of the object to be measured includes: projecting a first plane array light covering the plurality of areas, the first plane array light including a plurality of coding patterns, each coding pattern corresponding to a characteristic value; projecting a second plane array light covering the plurality of areas, the second plane array light including a plurality of coding patterns; wherein the arrangement order of the plurality of coding patterns in the first plane array light and the second plane array light is different; capturing the reflected light of each area to the plane array light to obtain a set of pixel coordinates, and recording the mapping relationship between the set of pixel coordinates and the area, and obtaining a mapping list includes: capturing the reflected light of the area to the coding pattern in the first plane array light to obtain a first eigenvalue of the area; capturing the reflected light of the area to the coding pattern in the second plane array light to obtain a second eigenvalue of the area; and obtaining a set of pixel coordinates corresponding to the area based on the first eigenvalue and the second eigenvalue of the area.

[0027] That is to say, in this implementation, the first light projector can project multiple area arrays of light to multiple planar areas in sequence, wherein each area array of light can cover multiple areas, the projection time of different area arrays of light is different, each area array of light includes multiple coding patterns, and the arrangement order of the multiple coding patterns in different area arrays of light is different, and each coding pattern corresponds to a characteristic value; the camera captures the reflected light of the coding pattern in each of the multiple area arrays of light in the area to obtain multiple characteristic values ​​of the area; then, based on the multiple characteristic values, the pixel coordinate set corresponding to the area can be obtained.

[0028] In this implementation, a single area array light beam projected across multiple regions includes multiple coded patterns. Each coded pattern can mark a characteristic value for the pixel coordinates corresponding to a region. This means that a single projection can add a mark to the pixel coordinates corresponding to each of the multiple regions. Multiple projections of the area array light beam, with the coded patterns arranged in different order in each projection, allow for multiple marks to be added to the pixel coordinates corresponding to each region. These marks can also vary from region to region. Subsequently, based on the multiple marks added to the pixel coordinates, the pixel coordinate sets can be partitioned to obtain the pixel coordinate sets corresponding to the regions, achieving region labeling.

[0029] In this implementation, the pixel coordinates corresponding to each area are marked with multiple characteristic values, and the areas are marked using multiple characteristic values, thereby eliminating the need to set a different coding pattern for each area, thereby achieving the goal of marking more areas with fewer coding patterns.

[0030] In one possible implementation, projecting a plane array light onto different areas of a plurality of areas of the object to be measured includes: projecting a first plane array light covering the plurality of areas, the first plane array light including a plurality of coding patterns, each coding pattern corresponding to a characteristic value; projecting a second plane array light covering the plurality of areas, the second plane array light including a plurality of coding patterns; wherein the plurality of coding patterns included in the first plane array light and the plurality of coding patterns included in the second plane array light are not completely the same; capturing the reflected light of each area to the plane array light to obtain a set of pixel coordinates, and recording a mapping relationship between the set of pixel coordinates and the area to obtain a mapping list including: capturing the reflected light of the area to the coding pattern in the first plane array light to obtain a first eigenvalue of the area; capturing the reflected light of the area to the coding pattern in the second plane array light to obtain a second eigenvalue of the area; and obtaining a set of pixel coordinates corresponding to the area based on the first eigenvalue and the second eigenvalue of the area.

[0031] In this implementation, a single area array light beam projected across multiple regions includes multiple coded patterns. Each coded pattern can mark a characteristic value for the pixel coordinates corresponding to a region. This means that a single projection can add a mark to the pixel coordinates corresponding to each of the multiple regions. Multiple projections of the area array light beam, with different coded patterns in the projections, allow for multiple marks to be added to the pixel coordinates corresponding to each region, and the marks added to the pixel coordinates corresponding to different regions are not identical. The pixel coordinate sets can then be partitioned based on the multiple marks added to the pixel coordinates to obtain the pixel coordinate sets corresponding to the regions, thus achieving region labeling.

[0032] In this implementation, the pixel coordinates corresponding to each area are marked with multiple characteristic values, and the areas are marked using multiple characteristic values, thereby eliminating the need to set a different coding pattern for each area, thereby achieving the goal of marking more areas with fewer coding patterns.

[0033] According to a second aspect, a device for measuring three-dimensional information of an object is provided, which is a controller configured in a measuring system, and the measuring system further includes at least one light projector and a camera; the device includes: a first projection unit, which is used to project a plane array light to different areas of a plurality of areas of the measured object through a first light projector among the at least one light projector; wherein the plane array light projected to each area can cover the area; a recording unit, which is used to capture the reflected light of each area to the plane array light through a camera, obtain a set of pixel coordinates, and record the mapping relationship between the set of pixel coordinates and the area to obtain a mapping list; a second projection unit, which is used to project a line light to at least two areas of the plurality of areas through a second light projector among the at least one light projector; wherein the projection position of the line light on the measured object belongs to a subset of the projection area of ​​the plane array light on the measured object; a capturing unit, which is used to capture the reflected light of the area to the line light through a camera to obtain light stripes; an obtaining unit, which is used to obtain the projection parameters of the line light corresponding to the light stripes based on the pixel coordinates and the mapping relationship of the light stripes, and calculate the three-dimensional information of the measured object.

[0034] In a third aspect, a controller is provided, comprising a processor and a memory; wherein the memory is used to store computer instructions; and the processor is used to execute the computer instructions, so that the controller implements the method of any one of claims 1 to 8.

[0035] In a fourth aspect, a measurement system is provided, comprising the controller provided in the third aspect, at least one light projector and a camera.

[0036] In a fifth aspect, a computer storage medium is provided, which includes computer instructions. When the computer instructions are executed on a computing device, the computing device executes the method provided in the first aspect.

[0037] In a sixth aspect, a computer program product is provided. The computer program product includes program code. When the program code is executed by a processor in a computing device, the computing device implements the method provided in the first aspect.

[0038] The beneficial effects of the second to sixth aspects can be referred to the above introduction to the beneficial effects of the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] FIG1 is a schematic diagram of a system structure of a line structured light;

[0040] FIG2A is a schematic diagram of a measurement system provided in an embodiment of the present application;

[0041] FIG2B is a schematic diagram of a working mode of an event camera provided in an embodiment of the present application;

[0042] FIG3 is a flow chart of a method for measuring three-dimensional information of an object provided in an embodiment of the present application;

[0043] FIG4A is a schematic diagram of a planar array light projection provided by an embodiment of the present application;

[0044] FIG4B is a schematic diagram of a planar array light projection provided by an embodiment of the present application;

[0045] FIG4C is a schematic diagram of a planar array light projection provided in an embodiment of the present application;

[0046] FIG4D is a schematic diagram of a planar array light projection provided by an embodiment of the present application;

[0047] FIG5A is a schematic diagram of a planar array light provided in an embodiment of the present application;

[0048] FIG5B is a schematic diagram of a pixel coordinate set provided in an embodiment of the present application;

[0049] FIG6A is a schematic diagram of a planar array light provided in an embodiment of the present application;

[0050] FIG6B is a schematic diagram of a pixel coordinate set encoding provided by an embodiment of the present application;

[0051] FIG7A is a schematic diagram of a line light projection provided by an embodiment of the present application;

[0052] FIG7B is a schematic diagram of reflected light from an object under test according to an embodiment of the present application;

[0053] FIG8A is a schematic diagram of a line light projection provided by an embodiment of the present application;

[0054] FIG8B is a schematic diagram of a line light projection provided by an embodiment of the present application;

[0055] FIG9 is a flow chart of a line light projection method provided in an embodiment of the present application;

[0056] FIG10 is a flow chart of a method for region marking and line light projection provided by an embodiment of the present application;

[0057] FIG11A is a schematic diagram of a planar array light projection provided by an embodiment of the present application;

[0058] FIG11B is a schematic diagram of a planar array light projection provided by an embodiment of the present application;

[0059] FIG11C is a schematic diagram of a line light projection provided by an embodiment of the present application;

[0060] FIG12A is a schematic diagram of a planar array light projection provided by an embodiment of the present application;

[0061] FIG12B is a schematic diagram of a planar array light projection provided by an embodiment of the present application;

[0062] FIG12C is a schematic diagram of a line light projection provided by an embodiment of the present application;

[0063] FIG13A is a schematic diagram of a light stripe provided in an embodiment of the present application;

[0064] FIG13B is a schematic diagram of a light stripe provided in an embodiment of the present application;

[0065] FIG14 is a schematic structural diagram of a device for measuring three-dimensional information of an object provided in an embodiment of the present application;

[0066] FIG15 is a schematic diagram of the structure of a controller provided in an embodiment of the present application;

[0067] Figure 16 is a schematic diagram of the structure of a measurement system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0068] The technical solution of the present application is further described in detail below through the accompanying drawings and embodiments. In this specification, "multiple" refers to two or more.

[0069] Generally speaking, structured light is a light system consisting of a light projector, a camera, and a controller. Among them, the light projector can also be called a projector, which is used to project light with a specific pattern onto the object to be measured. The object to be measured reflects the light projected by the light projector, and the reflected light is captured by the camera to obtain a reflection pattern. The controller can use the projection parameters of the light projected by the light projector (such as the light plane equation), the coordinates of each pixel in the reflection pattern captured by the camera (that is, the position of the pixel in the pattern), and the internal and external parameters of the camera to calculate information such as the depth of the surface of the object to be measured and obtain three-dimensional information of the object.

[0070] FIG1 shows a system structure of line structured light, which includes a light projector 110 , a camera 120 , and a controller 130 .

[0071] The light projector 110 may project one or more line lights toward the object 140. Line lights refer to light that is linear along a projection direction, such as the light 111 and the light 112 shown in FIG1 .

[0072] The object 140 under test can reflect the light projected onto it, and the camera 120 can capture the light reflected from the object 130 to obtain a reflection pattern. For line structured light, the light projected onto the object 140 is a line of light, and the camera 120 captures the reflected light of the line of light, resulting in a striped reflection pattern. This striped reflection pattern can be referred to as a light streak.

[0073] The controller 130 uses the coordinates of each pixel in the light stripe, the projection parameters of the line light projected by the light projector 110 (e.g., the light plane equation), and the intrinsic and extrinsic parameters of the camera to calculate three-dimensional information, such as depth, at the projection position of the line light on the object 140. The projection position of the line light on the object 140 refers to the position on the object 140 where the line light irradiates or reaches, that is, the position on the object 140 where the line light is received or reflected.

[0074] In order to improve measurement efficiency, it is necessary to simultaneously project multiple line lights onto the object under test 140, such as line lights 111 and 112 shown in FIG1 . Since the surface of the object under test 140 may be uneven, the projection positions of the line lights on the object under test 140 may also be uneven, that is, the projection positions may have positions of different sizes. As shown in FIG1 , under the modulation of the different depths of the projection positions, the imaging of the reflected light of the line lights at the projection positions (that is, the light stripes produced by the camera capturing the reflected light) is deformed, and may be broken or missing. For example, as shown in FIG1 , the imaging of the reflected light of line light 111 by the object under test 140 is light stripe 121, and the imaging of the reflected light of line light 112 by the object under test 140 is light stripe 122. Since the projection positions of line lights 111 and line lights 112 on the object under test 140 have different depths, the light stripes 121 and light stripes 122 are broken.

[0075] In complex scenarios involving the depth of the object 140 and the presence of obstructions, specular reflections, and other factors, the multiple light streaks simultaneously projected onto the object 140 may intersect. These broken or missing light streaks, as well as the intersecting light streaks, can prevent the controller 130 from accurately identifying the correspondence between the light streaks and the light projected by the light projector 110. Consequently, the controller 130 cannot accurately determine the projection parameters corresponding to the light streaks, and thus cannot accurately calculate the 3D information at the projection location of the light streaks. The greater the number of projected light streaks and the denser their arrangement, the more difficult it is to accurately calculate the 3D information.

[0076] In response to the above situation, an embodiment of the present application provides a method for measuring three-dimensional information of an object. In this method, a light projector is used to project a plane array light that can cover an area in multiple areas of the object to be measured, so that each position point in the area can receive the light projected by the light projector and reflect the received light. The camera can capture the light reflected by each position point in the area to obtain an image, and record the coordinates of the pixels in the image, thereby recording the coordinates of the pixels of the light reflected by each position point in the area. The coordinates of the pixels of the light reflected by all position points in an area (that is, the entire area) constitute a pixel coordinate set. The mapping relationship between the area and the pixel coordinate set can be recorded to obtain a mapping list. After obtaining the mapping list, line light can be projected to the multiple areas at the same time by one or more light projectors, wherein a line light is projected to each area. The camera captures the reflected light of the area to the projected line light to obtain light stripes. Then, determine to which pixel coordinate set the pixel coordinates of the light streak belong; then, based on the mapping list and the pixel coordinate set to which the pixel coordinates of the light streak belong, obtain the area corresponding to the light streak, so as to obtain the projection parameters of the line light projected by the light projector to the area, that is, obtain the projection parameters corresponding to the light streak, and then use the light streak, the projection parameters corresponding to the light streak, and other information to calculate the three-dimensional information of the projection position corresponding to the light streak.

[0077] In the embodiments of this application, a region refers to a block-shaped area with an area, and the projection of the region in the projection direction of the light projector is two-dimensional. Specifically, if the surface of the object being measured is relatively flat, the region may be relatively flat and can be called a planar region. If the surface of the object being measured is uneven, the region may be an uneven area.

[0078] Through this method, multiple line lights can be projected onto the object being measured simultaneously, thereby improving measurement efficiency. Moreover, through the relationship between the coordinates of the light stripes corresponding to the line lights projected onto the area and the coordinates of the pixels of the light reflected by the entire area, the area corresponding to the light stripes can be accurately identified, and then the projection parameters corresponding to the light stripes can be accurately identified, so that the three-dimensional information of the projection position corresponding to the light stripes can be accurately calculated.

[0079] Next, the method for measuring three-dimensional information of an object provided in an embodiment of the present application is specifically introduced.

[0080] FIG2A shows a measurement system provided by an embodiment of the present application, which includes at least one light projector (eg, light projector 211 and / or light projector 212 ), a camera 220 , and a controller 230 .

[0081] A light projector is a device, apparatus, or component that can project light toward another object, such as the object to be measured 230. In some embodiments, the light projector can be a laser projector that can project laser light outward. In one embodiment, the same light projector (e.g., light projector 211 or light projector 212) can project both line light and area light. In some embodiments, the measurement system includes two or more light projectors, wherein some light projectors (e.g., light projector 211) can project area light, and some light projectors (e.g., light projector 212) can project line light.

[0082] The area array light is light that can cover a two-dimensional plane. That is, when the area array light is projected onto a block-shaped area with an area, it can cover the area.

[0083] The camera 220 is a device, apparatus or component that can convert an optical signal into a digital signal. The camera 200 has a photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) phototransistor. The photosensitive element can convert the optical signal into an electrical signal, which can then be processed into a digital signal. Based on the digital signal, pixels and related information about the pixels, such as the position of the pixels, can be obtained. The position of the pixels can be represented by coordinates in a coordinate system.

[0084] In some embodiments, the camera 220 may be an event-based camera. An event camera is a visual sensor inspired by biological vision. The working mode of the event camera is shown in formula (1) shown in Figure 2B, which senses changes in light intensity and outputs an event signal reflecting changes in pixel brightness. The event signal generally includes event polarity, pixel position, event time, etc. Among them, the pixel position can be represented by pixel coordinates. The event time refers to the time when the pixel brightness changes, which can be represented by a timestamp. The event polarity includes "-1" and "1". Among them, "-1" represents one of the brightness increase and brightness decrease of the pixel, and "1" represents the other of the brightness increase and brightness decrease of the pixel.

[0085] The event camera only reports event signals and outputs an asynchronous event stream at the microsecond level. Therefore, the event camera has the characteristics of low latency and can improve measurement efficiency.

[0086] The controller 230 can be any device, apparatus, or platform with data processing capabilities. The controller 230 can obtain projection parameters of the light projector projecting light onto the object under test, obtain the coordinates of pixels captured by the camera, and the camera's intrinsic and extrinsic parameters from the camera, and calculate three-dimensional information of the object under test based on the projection parameters, pixel coordinates, and the camera's intrinsic and extrinsic parameters.

[0087] The controller 230 can also control the light projector to project light toward the object under test, for example, by controlling the start and end times of the light projector's light projection toward the object under test, controlling the light projector to project a planar array of light or a line of light toward the object under test according to preset projection parameters, and controlling the characteristics of the planar array of light projected toward the object under test. In one example, the components of the planar array of light refer to its color. In one example, the characteristics of the planar array of light refer to its brightness. In one example, the planar array of light is composed of multiple light patterns, and the characteristics of the planar array of light refer to the light patterns that comprise the planar array of light.

[0088] The above examples introduce a measurement system provided in an embodiment of the present application. The measurement system can be used to implement the object three-dimensional information measurement method provided in an embodiment of the present application, wherein the method can be specifically executed by the controller 230 in the measurement system.

[0089] Referring to FIG3 , the method can be divided into two parts: region marking and 3D information measurement.

[0090] First, the region labeling scheme is introduced. As shown in FIG3 , the region labeling scheme includes steps 301 and 302 , which are described in detail as follows.

[0091] In step 301, a first light projector among at least one light projector projects an area array light onto different areas of a plurality of areas of an object under test, wherein the area array light projected onto each area covers that area. When the at least one light projector comprises multiple light projectors, the first light projector may be one or more of the multiple light projectors. For example, when the at least one light projector comprises light projector 211 and light projector 212, the first light projector may be light projector 211 and / or light projector 212.

[0092] The area covered by the area array light refers to all areas or most areas within the area illuminated by the area array light.

[0093] There are multiple implementations of step 301 . The following describes possible implementations of step 301 in different embodiments.

[0094] In embodiment 1, a planar array light covering each of multiple areas on the object to be measured can be projected sequentially to the area. That is, at the same time, a planar array light covering the area is projected to one area, and the time for projecting the planar array light to different areas is different.

[0095] Taking the measurement of object 240 as an example, as shown in FIG4A , light projector 211 can be set as the first light projector to project a planar array light covering area 241 onto area 241 on object 240. Then, as shown in FIG4B , light projector 211 projects a planar array light covering area 242 onto area 242 on object 240. Then, as shown in FIG4C , light projector 211 projects a planar array light covering area 243 onto area 243 on object 240. Then, as shown in FIG4D , light projector 211 projects a planar array light covering area 244 onto area 244 on object 240.

[0096] In one example of this embodiment, referring to FIG5A , the first light projector can continuously project multiple binary patterns onto the object under test, for example, binary pattern 501, binary pattern 502, binary pattern 503, binary pattern 504, binary pattern 505, and binary pattern 506. In each binary pattern, one portion is a bright area with a grayscale value of 255, and the other portion is a dark area with a grayscale value of 0. The bright area is a planar array light that can cover an area on the object under test. As shown in FIG5A , the positions of the bright areas are different in different binary patterns, so that the bright area in one binary pattern covers one area on the object under test, and the bright area in another binary pattern covers another area on the object under test. In this way, by continuously projecting multiple binary patterns, multiple areas on the object under test can be covered with the bright areas, that is, covered with the planar array light.

[0097] In the second embodiment, different area array lights can be projected simultaneously onto different areas on the object being measured. In other words, multiple area array lights can be projected simultaneously, with different area array lights projected onto and covering different areas. Exemplarily, different area array lights refer to area array lights of different colors, i.e., area array lights of different colors can be referred to as different area array lights. Exemplarily, different area array lights refer to area array lights of different brightness, i.e., area array lights of different brightness can be referred to as different area array lights, i.e., area array lights of different brightness can be referred to as different area array lights. Exemplarily, the area array light is composed of multiple patterns, and different area array lights refer to area array lights of different patterns, i.e., area array lights composed of different patterns can be referred to as different area array lights.

[0098] In the third embodiment, a plurality of planar array lights can be projected onto the object to be measured in sequence, wherein one of the plurality of planar array lights is projected onto the object to be measured each time, each planar array light covers multiple areas of the object to be measured, and each planar array light is composed of multiple coding patterns. The coding pattern is also called a sub-planar array light, which is also a light that can cover a block area with an area. A plurality of coding patterns constitute the planar array light projected onto the object to be measured, wherein the number of coding patterns constituting the planar array light is equal to the number of areas in the plurality of areas, one coding pattern covers one of the plurality of areas, and one coding pattern corresponds to one characteristic value. The area covered by the coding pattern can be marked with the characteristic value corresponding to the coding pattern, and a plurality of planar array lights are projected onto the object to be measured in sequence, so that an area is covered by a plurality of coding patterns in sequence, so as to mark a plurality of characteristic values ​​for the area. This process can be called coding. The multiple characteristic values ​​of the area can be used to mark the area.

[0099] The multiple coding patterns comprising the area array light include at least two types of coding patterns, and the area array light contains one or more coding patterns of each type. Different types of coding patterns correspond to different characteristic values. Thus, through multiple projections, different areas can be marked with different characteristic values.

[0100] In an example of the third embodiment, the area array light composed of multiple coding patterns can be generated based on Gray code (binary gray code). The area array light generated based on Gray code is composed of two types of coding patterns, the characteristic value corresponding to one type of coding pattern is "1", and the characteristic value corresponding to the other type of coding pattern is "0". Figure 6A shows a schematic diagram of an area array light generated based on Gray code. As shown in Figure 6A, area array light 601, area array light 602, and area array light 603 are all composed of 8 coding patterns. Among the 8 coding patterns, the 4 coding patterns with low brightness are coding patterns of type A1, and the 4 coding patterns with high brightness are coding patterns of type A2. Among them, the characteristic value corresponding to the coding pattern of type A1 is one of "1" and "0", and the characteristic value corresponding to the coding pattern of type A2 is the other of "1" and "0".

[0101] In embodiment three, a planar array light includes multiple coding patterns, each coding pattern can mark a characteristic value for an area, that is, one projection can add a characteristic value for the pixel coordinates corresponding to each area in multiple areas. After multiple projections, multiple characteristic values ​​can be added to the pixel coordinates corresponding to each area. Multiple characteristic values ​​are used to mark the area, so there is no need to set a different coding pattern for each area, and more areas can be marked with fewer coding patterns.

[0102] 3 , in step 302 , the camera 220 captures the reflected light of each area from the array light to obtain a pixel coordinate set, and the mapping relationship between the pixel coordinate set and the area is recorded to obtain a mapping list.

[0103] When the area array light covering the area reaches the area, the area may reflect the area array light. Camera 220 can capture the area array light reflected by the area, obtain an image, and record the coordinates of the pixels in the image to obtain a set of pixel coordinates. In this way, camera 220 can obtain the coordinates of the pixels in the image obtained by the light reflected from the area.

[0104] Exemplarily, the mapping list may also be referred to as a mapping table and may be implemented as a table.

[0105] As described above, there are different implementations of step 301. Depending on the different implementations of step 301, the implementations of step 302 may also be different, as detailed below.

[0106] Regarding the implementation of step 301 described in the first embodiment above, in step 302, pixel coordinate sets obtained from reflected light captured at different times are mapped to different regions. Taking the scenario shown in Figures 4A and 4B as an example, when light projector 211 projects area light onto region 241, camera 220 captures the area light reflected from region 241 and obtains a set of pixel coordinates. Camera 220 can map the most recently obtained set of pixel coordinates to region 241 and record this mapping in a mapping list. When light projector 211 projects area light onto region 242, camera 220 captures the area light reflected from region 242 and obtains a set of pixel coordinates. Camera 220 can map the most recently obtained set of pixel coordinates to region 242 and record this mapping in a mapping list. And so on, camera 220 can obtain a set of pixel coordinates corresponding to each region and, further, record the mapping between each region and its corresponding set of pixel coordinates in a mapping list.

[0107] In an illustrative example, for the situation shown in FIG5A , when light projector 211 projects binary pattern 501, camera 220 can capture pixel coordinate set 501′ shown in FIG5B . The pixel coordinate set consisting of the coordinates of pixel coordinate set 501′ is mapped to an area, and the mapping relationship between the pixel coordinate set and the area can be recorded. When light projector 211 projects binary pattern 502, camera 220 can capture pixel coordinate set 502′ shown in FIG5B . The pixel coordinate set consisting of the coordinates of pixel coordinate set 502′ is mapped to an area, and the mapping relationship between the pixel coordinate set and the area can be recorded. When light projector 211 projects binary pattern 503, camera 220 can capture pixel coordinate set 503′ shown in FIG5B . The pixel coordinate set consisting of the coordinates of pixel coordinate set 503′ is mapped to an area, and the mapping relationship between the pixel coordinate set and the area can be recorded. When light projector 211 projects binary pattern 504, camera 220 can capture pixel coordinate set 504' shown in FIG5B . The pixel coordinate set formed by the coordinates of pixel coordinate set 504' is mapped to a region, and the mapping relationship between the pixel coordinate set and the region can be recorded. When light projector 211 projects binary pattern 505, camera 220 can capture pixel coordinate set 505' shown in FIG5B . The pixel coordinate set formed by the coordinates of pixel coordinate set 505' is mapped to a region, and the mapping relationship between the pixel coordinate set and the region can be recorded. When light projector 211 projects binary pattern 506, camera 220 can capture pixel coordinate set 506' shown in FIG5B , and the pixel coordinate set formed by the coordinates of pixel coordinate set 506' is mapped to a region, and the corresponding relationship between the pixel coordinate set and the region can be recorded. In this way, the pixel coordinate set corresponding to each region and the mapping relationship between each region and its mapped pixel coordinate set can be obtained and recorded in the mapping list.

[0108] Regarding the implementation of step 301 described in the second embodiment above, in step 302, the camera may group the coordinates of pixels in the image obtained by the same reflected light reflected from the object under test into one set of pixel coordinates, and group the coordinates of pixels in the images obtained by different reflected light into different sets of pixel coordinates. It will be understood that the reflected light reflected by the same area array light from the object under test remains the same, while the reflected light reflected by different area array light from the object under test is different. In step 301, different area array lights are projected onto different areas, so the reflected light reflected by different areas is different. In step 302, the camera groups the coordinates of pixels in the image obtained by the same reflected light reflected from the object under test into one set of pixel coordinates, and group the coordinates of pixels in the images obtained by different reflected light into different sets of pixel coordinates, thereby obtaining sets of pixel coordinates corresponding to different areas. The mapping relationship between pixel coordinate sets and their corresponding areas can be recorded to obtain a mapping list.

[0109] Regarding the implementation of step 301 described in the third embodiment above, as described above, each of the multiple coded patterns comprising the area array light can cover an area. When a coded pattern is projected or illuminated onto an area, the area reflects the coded pattern. The camera 220 captures the coded pattern within the area, obtaining multiple pixels. The coordinates of these multiple pixels are recorded, and each of these multiple pixel coordinates is labeled with a feature value corresponding to the coded pattern. In this way, a feature value can be added to the pixel coordinates of the reflected light from each of the multiple areas. Projecting a single area array light onto multiple areas can add a feature value to the pixel coordinates. Projecting multiple area array lights sequentially onto multiple areas can add multiple feature values ​​to the pixel coordinates. Then, based on the multiple feature values ​​of the pixel coordinates, a set of pixel coordinates for the corresponding area is obtained. Specifically, multiple pixel coordinates with identical feature values ​​are grouped into one pixel coordinate set, while multiple pixel coordinates with different feature values ​​are grouped into different pixel coordinate sets. In this way, multiple pixel coordinate sets can be obtained, each corresponding to a region. The mapping relationship between pixel coordinate sets and their corresponding regions can be recorded to obtain a mapping list. The tags added to the pixel coordinates may also be referred to as encoding the pixel coordinates, and the multiple tags added form a code for representing the pixel coordinates.

[0110] The pixel coordinates of the reflected light from an area are the coordinates of the pixels in the image captured by the camera after the light reflected from the area is captured. A coordinate set corresponds to an area, which can also be called a mapping of a coordinate set to an area.

[0111] In one example, the area array light shown in Figure 6A can be projected onto multiple areas to obtain the encoding results shown in Figure 6B. Specifically, the characteristic value corresponding to the encoding pattern of type A1 can be set to "0", and the characteristic value corresponding to the encoding pattern of type A2 can be set to "1". When area array light 601, area array light 602, and area array light 603 are sequentially projected onto the object being measured, the pixel coordinates of the reflected light in the area projected by the type A1 coding pattern are marked with a characteristic value of "0", and the pixel coordinates of the reflected light in the area projected by the type A2 coding pattern are marked with a characteristic value of "1". In this way, among the coordinates of all pixels in the image obtained by the camera, some of the coordinates are marked with multiple characteristic values ​​of "000", another portion of the coordinates are marked with multiple characteristic values ​​of "001", another portion of the coordinates are marked with multiple characteristic values ​​of "011", another portion of the coordinates are marked with multiple characteristic values ​​of "010", another portion of the coordinates are marked with multiple characteristic values ​​of "110", another portion of the coordinates are marked with multiple characteristic values ​​of "111", another portion of the coordinates are marked with multiple characteristic values ​​of "101", and another portion of the coordinates are marked with multiple characteristic values ​​of "100". Pixel coordinates with identical characteristic values ​​can be grouped into one pixel coordinate set, and pixel coordinates with different characteristic values ​​can be grouped into different pixel coordinate sets, resulting in eight pixel coordinate sets. The eight pixel coordinate sets correspond to the eight regions one-to-one, and the mapping relationship between the pixel coordinate sets and the corresponding regions is recorded to obtain a mapping list.

[0112] After steps 301 and 302 , a mapping list can be obtained to complete the marking of multiple areas on the object to be measured, or in other words, the area to be measured in the object to be measured is divided into multiple areas, or the field of view of the light projector is divided into multiple areas.

[0113] Next, we will introduce the measurement of three-dimensional information.

[0114] Continuing to refer to FIG. 3 , the three-dimensional information measurement includes steps 303 , 304 , and 305 , as detailed below.

[0115] First, in step 303, line light is projected onto at least two areas of the multiple areas through a second light projector among the at least one light projector; wherein the projection position of the line light on the object to be measured belongs to a subset of the projection area of ​​the array light on the object to be measured.

[0116] In some embodiments, the second light projector and the first light projector can be the same projector. In some embodiments, the second light projector and the first light projector are different projectors, and the second light projector and the first light projector are located on the same side of the object being measured. For example, as shown in Figures 4A and 7A, the first light projector can be light projector 211, and the second light projector can be light projector 212.

[0117] The projection position of the line light on the object to be measured refers to the position on the object to be measured where the line light reaches, that is, the position on the object to be measured where the line light is received, or the position on the object to be measured where the line light is illuminated or projected. The projection area of ​​the array light on the object to be measured refers to the area when the array light reaches the object to be measured, that is, the position on the object to be measured where the array light is received, or the position on the object to be measured where the array light is illuminated or projected. The projection position of the line light projected onto the area by the second light projector is located within the projection area on the object to be measured of the array light projected onto the area by the first light projector, that is, the projection position of the line light projected onto the area on the object to be measured is a subset of the projection area of ​​the array light on the object to be measured. Thus, the line light can be projected into the area marked by the array light.

[0118] In some embodiments, the at least two areas are the multiple planar areas, that is, the second light projector can project line light to all areas in the multiple areas at the same time, wherein one line light is projected to one area.

[0119] In an illustrative example of this embodiment, as shown in FIG7A , it is assumed that the area to be measured on the object 240 is divided into area 241, area 242, area 243, and area 244 through steps 301 and 302. The light projector 212, which serves as the second light projector, simultaneously projects line light onto areas 241, 242, 243, and 244, wherein one line light is projected onto each area.

[0120] In some embodiments, the at least two areas are partial areas of the plurality of areas. Specifically, the plurality of areas can be divided into at least two groups, each group including at least two areas, and at least one other area is separated between two adjacent areas in the at least two areas. Then, a line light is projected onto all areas in a group of areas each time, wherein one line light is projected onto one area. Since there is at least one other area between the two areas closest to each other in the same group (i.e., the two adjacent areas), the projection positions of the two simultaneously projected line lights and the distance between the projection position of the line light and the adjacent areas of the area projected by the line light are enlarged. Accordingly, the distance between the light stripes obtained by the camera 220 capturing the reflected light of the line light at the projection position is also enlarged, and the distance between the pixel coordinates in the light stripes and the pixel coordinates corresponding to the adjacent areas of the area projected by the line light corresponding to the light stripes is also enlarged. Therefore, even if the light stripes are blurred due to problems such as camera defocus, the area to which the light stripes belong can be correctly distinguished, and the correct projection parameters can be matched to the light stripes.

[0121] In an illustrative example of this embodiment, referring to FIG8A , steps 301 and 302 can be used to divide the area to be measured on object 240 into the following order: area 241, area 242, area 243, and area 244. Areas 241 and 243 can be divided into group B1, while areas 242 and 244 can be divided into group B2. In step 303, a line light can be projected onto the area to be measured on object 240 using the projection method shown in FIG9 . The details are as follows.

[0122] At step 901, light projector 212 projects line light onto each area within group B1. The projection results are shown in Figure 8A. Within the area, the projection position of the line light projected onto that area can be continuously moved, for example, by adjusting the projection direction of the line light or moving the object under test 240. Each time the projection position is moved, step 902 is executed to determine whether the projection of group B1 is complete. Group B1 projection is complete when the projection position of the line light projected onto an area in group B1 reaches the edge of the projection area of ​​the area array light projected onto that area. Otherwise, the projection position of the line light continues to be moved.

[0123] When the projection of group B1 is complete, light projector 212 projects line light onto each area within group B2 in step 903. The projection results are shown in Figure 8B. Within an area, the projection position of the line light projected onto that area can be continuously moved. Each time the projection position is moved, step 904 is executed to determine whether the projection of group B2 is complete. Group B2 projection is complete when the projection position of the line light projected onto an area in group B1 reaches the edge of the projection area of ​​the area array light projected onto that area. Otherwise, the projection position of the line light continues to be moved.

[0124] When the projection of group B2 is completed, the line light projection can be ended.

[0125] The previous example introduces a method of marking all areas first and then casting line lights. Next, we will introduce a method of completing area marking in batches and distributing the cast line lights.

[0126] Still taking the object 240 to be measured, and assuming that the area to be measured of the object 240 is composed of area 241, area 242, area 243, and area 244 arranged in sequence, area 241 and area 243 can be divided into group B1, and area 242 and area 243 can be divided into group B2.

[0127] Referring to Figure 10 , step 1001 can be performed first to mark the regions within group B1. Specifically, as shown in Figures 11A and 11B , light projector 211 projects an array of light onto each region within group B1 (i.e., region 241 and region 243). Camera 220 captures the reflected light from the regions within group B1, obtaining a set of pixel coordinates. The mapping relationship between the pixel coordinates and the regions is recorded, completing the region marking of the regions within group B1. For details, refer to the above description of steps 301 and 302 and will not be repeated here.

[0128] Next, step 1002 is executed to project line light onto the areas within group B1 to measure three-dimensional information. Specifically, as shown in FIG11C , light projector 212 projects line light onto each area within group B1. The details of this process can be found in the description of step 901 above and will not be repeated here.

[0129] Within the area, the projection position of the line light projected onto the area can be continuously moved. Each time the projection position is moved, step 1003 can be executed to determine whether group B1 has been projected. For details, please refer to the above description of step 902 and will not be repeated here.

[0130] When the projection of group B1 is complete, step 1004 can be executed to mark the areas within group B2. Specifically, as shown in Figures 12A and 12B, light projector 211 projects an array of light onto each area within group B2 (i.e., area 242 and area 244). Camera 220 captures the reflected light from the area within group B2, obtaining a set of pixel coordinates. The mapping relationship between the pixel coordinates and the area is recorded, completing the area marking of the areas within group B2. For details, please refer to the above description of steps 301 and 302, and will not be repeated here.

[0131] Next, step 1005 is executed to project line light onto the areas within group B2 to measure three-dimensional information. Specifically, as shown in FIG12C , light projector 212 projects line light onto each area within group B2. The details of step 903 can be found above and will not be repeated here.

[0132] Within the area, the projection position of the line light projected onto the area can be continuously moved. Each time the projection position is moved, step 1006 can be executed to determine whether group B2 has been projected. For details, please refer to the above description of step 904 and will not be repeated here.

[0133] When the projection of group B2 is completed, the line light projection can be ended.

[0134] Returning to FIG. 3 , in step 304 , the controller 230 may capture the reflected light of the area relative to the line light through the camera 220 to obtain light stripes.

[0135] When the second light projector projects a line light onto a region, the region reflects the line light toward camera 220, which captures the light reflected from the region, producing a light streak composed of multiple pixels. Taking the scenario shown in Figure 7A as an example, when the line light is projected onto the region according to Figure 7A, the reflection of the line light from the region is shown in Figure 7B. Specifically, region 241 reflects the line light projected onto region 241 toward camera 220, which captures the light reflected from region 241, producing a light streak composed of multiple pixels. Region 242 reflects the line light projected onto region 242 toward camera 220, which captures the light reflected from region 242, producing a light streak composed of multiple pixels. And so on. In this way, light streaks can be obtained for the light reflected from each region.

[0136] In step 305 , based on the pixel coordinates of the light streak and the mapping list obtained in step 302 , the projection parameters of the line light corresponding to the light streak are obtained, and the three-dimensional information of the object to be measured is calculated.

[0137] As described above, the projection position of the line light on the area is a subset of the projection area of ​​the area array light on the area. Therefore, the pixel coordinates of the reflected light of the area to the line light are a subset of the set of pixel coordinates of the reflected light of the area to the area array light, that is, the pixel coordinates of the reflected light of the area to the line light are included in the set of pixel coordinates of the reflected light of the area to the area array light.

[0138] The pixel coordinates of the reflected light of the area to the line light refer to the coordinates of the pixels in the image obtained after the camera captures the reflected light of the area to the line light, and the pixel coordinates of the reflected light of the area to the array light refer to the coordinates of the pixels in the image obtained after the camera captures the reflected light of the area to the array light.

[0139] As described above, the mapping list records the mapping relationship between the region and the pixel coordinate set of the reflected light of the array light.

[0140] In this way, the area corresponding to the light streak can be obtained based on the pixel coordinates of the light streak and the mapping list obtained in step 302. Specifically, a pixel coordinate set containing the pixel coordinates of the light streak can be first identified to obtain a pixel coordinate set containing the pixel coordinates of the light streak. For ease of description, the pixel coordinate set containing the pixel coordinates of the light streak can be referred to as a target pixel coordinate set. Then, the mapping list is searched for the area mapped to the target pixel coordinate set to obtain a target area. The obtained target area is the area corresponding to the light streak.

[0141] In one example, using the scenarios shown in Figures 5A and 5B as an example, when line light is projected onto regions 501, 502, 503, 504, 505, and 506, multiple light streaks are generated, as shown in Figure 13A. By identifying the pixel coordinate set to which the coordinates of the pixels of each light streak belong, the pixel coordinate set to which the light streak belongs is determined. Then, based on the pixel coordinate set to which each light streak belongs and the mapping list described above, the region corresponding to each light streak is determined. Specifically, the mapping list is searched for the region corresponding to the pixel coordinate set to which the light streak belongs, and the resulting region is used as the region corresponding to the light streak. Among them, the light stripes corresponding to area 501 are shown as 501" in Figure 13B, the light stripes corresponding to area 502 are shown as 502" in Figure 13B, the light stripes corresponding to area 503 are shown as 503" in Figure 13B, the light stripes corresponding to area 504 are shown as 504" in Figure 13B, the light stripes corresponding to area 505 are shown as 505" in Figure 13B, and the light stripes corresponding to area 506 are shown as 506" in Figure 13B. In other words, the pixels of the light stripes corresponding to area 501 constitute 501" in Figure 13B, the pixels of the light stripes corresponding to area 502 constitute 502" in Figure 13B, the pixels of the light stripes corresponding to area 503 constitute 503" in Figure 13B, the pixels of the light stripes corresponding to area 504 constitute 504" in Figure 13B, the pixels of the light stripes corresponding to area 505 constitute 505" in Figure 13B, and the pixels of the light stripes corresponding to area 506 constitute 506" in Figure 13B. In Figures 13A and 13B , the grayscale value of the pixels representing the light streak is set to 255, while the grayscale values ​​of the other pixels are set to 0. In other words, in Figures 13A and 13B , the pixels with a grayscale value of 255 are the pixels representing the light streak. The "other pixels" here refer to pixels in the image obtained by camera 220 capturing light other than the reflected light from the line light.

[0142] In some embodiments, camera 220 may be an event camera. As described above, an event camera used as camera 220 can output information such as pixel coordinates by outputting event signals. An event signal can be output for each pixel. Whenever camera 220 outputs an event signal, controller 230 can identify the region corresponding to the pixel coordinates in the event signal. In this way, the region corresponding to each pixel in the light streak can be identified, thereby obtaining the region corresponding to the light streak.

[0143] Based on the area corresponding to the light streak, projection parameters for the line light corresponding to the light streak are obtained. The line light corresponding to the light streak is the line light projected onto and reflected from the area corresponding to the light streak. As described above, controller 230 controls the second light projector to project line light onto each area. Therefore, controller 230 maintains a corresponding relationship between the projection parameters for the line light projected onto each area and each area. After obtaining the area corresponding to the light streak, controller 230 can obtain the projection parameters for the line light projected onto that area based on this corresponding relationship and the area corresponding to the light streak, and use the obtained projection parameters as the projection parameters for the line light corresponding to the light streak.

[0144] In some embodiments, the projection parameter is specifically a light plane equation.

[0145] After obtaining the projection parameters of the line light corresponding to the light streaks, the controller 230 can calculate the three-dimensional information of the projection position of the line light corresponding to the light streaks, such as depth, point cloud, etc., based on the pixel coordinates of the light streaks, the projection parameters of the line light corresponding to the light streaks, the internal parameters of the camera 220, and the external parameters of the camera 220.

[0146] The method for measuring three-dimensional information of an object provided in embodiments of the present application can project a planar array of light onto an area, covering the area. This allows a camera to capture the light reflected from all or most points in the area, obtain pixels corresponding to all or most points in the area, and record the coordinates of these pixels. This allows the positions of the pixels corresponding to the light reflected from the area to be recorded, thereby marking the area. After multiple areas have been marked, a line light can be simultaneously projected onto at least two of the areas to simultaneously scan the at least two areas. The camera can capture the light reflected from the measured object by the line light, generating light streaks. The projected position of the line light on the measured object belongs to a subset of the projected area of ​​the planar array light on the object; that is, the projected position of the line light is contained within the projected area of ​​the planar array light. Accordingly, the coordinates of the pixels reflected from the measured object by the line light are also contained within the coordinates of the pixels reflected from the corresponding planar array light. A mapping list records the mapping relationship between pixel sets and areas. Based on the pixel coordinates of the light streak and the mapping list, the projection parameters of the line light corresponding to the light streak can be obtained, thereby matching the light streak with the correct projection parameters.

[0147] Referring to FIG14 , an embodiment of the present application provides a device 1400 for measuring three-dimensional information of an object. Device 1400 can be configured as a controller in a measurement system, which also includes at least one light projector and a camera. As shown in FIG14 , device 1400 includes:

[0148] A first projection unit 1410 is configured to project a planar array light onto different areas of the plurality of areas of the object to be measured through a first light projector among the at least one light projector; wherein the planar array light projected onto each area can cover the area;

[0149] The recording unit 1420 is configured to capture the reflected light of each area to the area array light through the camera to obtain a pixel coordinate set, and record a mapping relationship between the pixel coordinate set and the area to obtain a mapping list;

[0150] A second projection unit 1530 is configured to project a line light onto at least two of the multiple areas through a second light projector in the at least one light projector; wherein a projection position of the line light onto the object to be measured belongs to a subset of a projection area of ​​the area array light onto the object to be measured;

[0151] A capturing unit 1440 is configured to capture the reflected light of the area to the line light through the camera to obtain light streaks;

[0152] The obtaining unit 1450 is configured to obtain projection parameters of the line light corresponding to the light stripes based on the pixel coordinates of the light stripes and the mapping relationship, and calculate the three-dimensional information of the object under test.

[0153] The functions of the various functional units in the device 1400 may be implemented with reference to the above description of the method embodiment shown in FIG3 , and will not be described in detail here.

[0154] The present embodiment provides a controller 1500. Referring to FIG. 15 , the controller 1500 includes a processor 1510 and a memory 1520. The memory 1520 is configured to store computer instructions, and the processor 1510 is configured to execute the computer instructions, thereby enabling the controller 1500 to implement the method shown in FIG. 3 or to perform the operations performed by the controller 230 described above.

[0155] The processor 1510 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0156] The memory 1520 may include a volatile memory, such as a random access memory (RAM). The memory 1520 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0157] The present application also provides a measurement system 1600. As shown in FIG16 , the measurement system 1600 includes a controller 1600, at least one projector 1600, and a camera 1620. The at least one projector 1600 can be implemented as described above with reference to projectors 211 and 212, and the camera 1620 can be implemented as described above with reference to camera 220, and will not be further described here.

[0158] The present application also provides a computer program product comprising instructions. The computer program product may be software or a program product comprising instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on a computing device, the computing device executes the method shown in FIG3 .

[0159] The present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center that contains one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to execute the method shown in FIG3.

[0160] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the protection scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for measuring three-dimensional information of an object, characterized in that: A controller applied to a measurement system, wherein the measurement system further comprises at least one light projector and a camera; the method comprises: Projecting a planar array of light to different regions of the plurality of regions of the object to be measured by a first light projector among the at least one light projector; wherein the planar array of light projected to each of the regions can cover the region; The camera captures the reflected light of each area to the area array light to obtain a pixel coordinate set, and records the mapping relationship between the pixel coordinate set and the area to obtain a mapping list; Projecting line light to at least two of the multiple areas respectively through a second light projector in the at least one light projector; wherein the projection position of the line light on the object to be measured belongs to a subset of the projection area of ​​the area array light on the object to be measured; Capturing the reflected light of the area to the line light by the camera to obtain light stripes; Based on the pixel coordinates of the light stripes and the mapping relationship, the projection parameters of the line light corresponding to the light stripes are obtained, and the three-dimensional information of the measured object is calculated.

2. The method according to claim 1, characterized in that Based on the pixel coordinates of the light stripes and the mapping relationship, obtaining the projection parameters of the line light corresponding to the light stripes includes: identifying a target pixel coordinate set including pixel coordinates of the light streak; Based on the mapping list, obtaining a target area mapped by the target pixel coordinate set; Confirm that the projection parameters of the line light projected onto the target area are the projection parameters of the line light corresponding to the light stripes.

3. The method according to claim 1 or 2, characterized in that: At least one region is spaced between two adjacent regions of the at least two regions, and the at least one region belongs to a region other than the at least two regions among the plurality of regions.

4. The method according to any one of claims 1 to 3, characterized in that The first light projector and the second light projector are the same projector.

5. The method according to any one of claims 1 to 3, characterized in that: The first light projector and the second light projector are different projectors located on the same side of the object to be measured.

6. The method according to any one of claims 1 to 5, characterized in that The projecting of the planar array light to different regions among the multiple regions of the object to be measured includes: projecting the planar array light to each of the multiple regions in sequence.

7. The method according to any one of claims 1 to 5, characterized in that The projecting of the planar array light to different regions among the multiple regions of the object to be measured comprises: At the same time, the area array light is projected to different areas among the multiple areas, wherein the area array light projected to different areas is different.

8. The method according to any one of claims 1 to 5, characterized in that The projecting of the planar array light to different regions among the multiple regions of the object to be measured comprises: Projecting a first array light covering the multiple areas, the first array light comprising a plurality of coding patterns, each of the coding patterns corresponding to a characteristic value; Projecting a second area array light covering the multiple areas, the second area array light comprising the multiple coding patterns; wherein the arrangement order of the multiple coding patterns in the first area array light and the second area array light is different; The capturing of the reflected light of each area to the area array light to obtain a pixel coordinate set, and recording the mapping relationship between the pixel coordinate set and the area to obtain a mapping list includes: Capturing the reflected light of the region to the coding pattern in the first area array light to obtain a first characteristic value of the region; capturing the reflected light of the region to the coding pattern in the second area array light, and obtaining a second characteristic value of the region; Based on the first eigenvalue and the second eigenvalue of the region, a set of pixel coordinates corresponding to the region is obtained.

9. A device for measuring three-dimensional information of an object, characterized in that: A controller configured in a measurement system, wherein the measurement system further comprises at least one light projector and a camera; the device comprises: A first projection unit, configured to project a planar array light to different regions of the plurality of regions of the object to be measured through a first light projector in the at least one light projector; wherein the planar array light projected to each of the regions can cover the region; A recording unit, used for capturing the reflected light of each area to the area array light through the camera to obtain a pixel coordinate set, and recording a mapping relationship between the pixel coordinate set and the area to obtain a mapping list; The second projection unit is used to project line light to at least two of the multiple areas through a second light projector in the at least one light projector; wherein the projection position of the line light on the object to be measured belongs to the projection position of the area array light on the object to be measured. A subset of the shadow region; A capturing unit, used for capturing the reflected light of the area to the line light through the camera to obtain light fringes; The obtaining unit is used to obtain the projection parameters of the line light corresponding to the light stripes based on the pixel coordinates of the light stripes and the mapping relationship, and calculate the three-dimensional information of the measured object.

10. A controller comprising a processor and a memory; wherein: The memory is used to store computer instructions; the processor is used to execute the computer instructions, so that the controller implements the method according to any one of claims 1 to 8.

11. A measurement system comprising the controller of claim 10, at least one light projector and a camera.

12. A computer storage medium, characterized in that: The computer storage medium includes computer instructions, which, when executed on a computing device, cause the computing device to perform the method of any one of claims 1-8.

13. A computer program product, characterized in that The computer program product includes program codes, and when the program codes are executed by a processor in a computing device, the computing device implements the method according to any one of claims 1 to 8.

Citation Information

Patent Citations

  • Uncalibrated linear structured light vision sensor coordinate measurement method and system

    CN111649669A

  • Mobile projection type three-dimensional measurement method and device

    CN115876083A

  • Binocular multi-line three-dimensional reconstruction method and system based on regional coding

    CN115880432A

  • Multiframe correspondence estimation

    US20040151365A1

  • Measurement apparatus and method thereof

    US20150362312A1