Optical system, depth calculation method, and electronic device

Abstract

The invention provides an optical system, a depth calculation method and an electronic device. The optical system includes: a projector configured to project light to a target object; a first camera configured to collect an image of the target object to obtain a first target image associated with first depth information; the projector is arranged between the first camera and the second camera, the projector is arranged between the first camera and the second camera, the projector is arranged between the first camera and the second camera, and the projector is arranged between the first camera and the second camera. And the distance between the projector and the first camera is greater than the distance between the projector and the second camera. The optical system can realize the function of simultaneously identifying far and near distances, and can further reduce the computing power consumption of the chip.

Description

Technical Field

[0001] The present disclosure belongs to the field of optics and relates to optical systems, and in particular to optical systems, depth calculation methods and electronic devices. Background Art

[0002] The active vision mode based on structured light coding can obtain the depth information of the image more accurately. Compared with binocular stereo cameras, this mode has the advantages of more stable and reliable depth map information, no influence from ambient light, simple stereo matching process, and small algorithm calculation. For example, Microsoft's somatosensory interaction device Kinect adopts the active vision mode of infrared structured light, that is, infrared laser projects a fixed pattern of image onto the surface of the object, and diffuse reflection of the surface of the object forms scattered spots, which are collected by the camera to obtain the speckle image, and then the depth map information of the object is calculated by the image depth perception device. Summary of the invention

[0003] The purpose of the present disclosure is to provide an optical system, a depth calculation method and an electronic device, which are used to solve the problem that the current 3D camera group cannot realize short-distance depth calculation while performing long-distance depth calculation.

[0004] In a first aspect, an embodiment of the present disclosure provides an optical system. The optical system includes: a projector configured to project light onto a target object; a first camera configured to capture an image of the target object to obtain a first target image associated with first depth information; and a second camera configured to capture an image of the target object to obtain a second target image associated with second depth information, wherein the projector is disposed between the first camera and the second camera, and the distance between the projector and the first camera is greater than the distance between the projector and the second camera.

[0005] In one embodiment of the present disclosure, the first camera is configured to perform a speckle search within a first depth distance range to acquire the first target image, and the second camera is configured to perform a speckle search within a second depth distance range to acquire the second target image, and the maximum distance value within the second depth distance range is smaller than the maximum distance value within the first depth distance range.

[0006] In one embodiment of the present disclosure, the first camera is a first infrared camera, the second camera is a second infrared camera, the sensor of the first infrared camera is the same as the sensor of the second infrared camera, and the lens of the first infrared camera is the same as the lens of the second infrared camera.

[0007] In one embodiment of the present disclosure, the projector includes a vertical cavity surface emitting laser or an edge emitting laser, the emission spatial distribution of the projector is a pseudo-randomly distributed infrared speckle, and the wavelength range of the light projected by the projector is 800nm ​​to 1550nm.

[0008] In one embodiment of the present disclosure, the number of scattered spots emitted by the projector is 1,000 to 50,000.

[0009] In one embodiment of the present disclosure, the distance between the projector and the first camera is between 30 mm and 100 mm, and the distance between the projector and the second camera is between 5 mm and 20 mm.

[0010] In one embodiment of the present disclosure, the optical system also includes: a chip configured to acquire the first depth information based on the first target image and a first reference image corresponding to the first target image, and to acquire the second depth information based on the second target image and a second reference image corresponding to the second target image.

[0011] In an embodiment of the present disclosure, the chip is further configured to acquire third depth information based on the first target image, the second target image, and texture information associated with the target object.

[0012] In an embodiment of the present disclosure, the chip is further configured to perform depth fusion on the second depth information and the third depth information or to perform depth fusion on the first depth information and the second depth information.

[0013] In an embodiment of the present disclosure, the chip is further configured to perform depth fusion on the first depth information and the third depth information.

[0014] In one embodiment of the present disclosure, the optical system further includes a color camera.

[0015] In a second aspect, an embodiment of the present disclosure provides an electronic device, wherein the electronic device comprises the optical system according to any one of the first aspects.

[0016] In one embodiment of the present disclosure, the electronic device includes a sweeper or a robot.

[0017] In a third aspect, an embodiment of the present disclosure provides a depth calculation method. The depth calculation method includes: acquiring a first target image, a first reference image corresponding to the first target image, a second target image, and a second reference image corresponding to the second target image, wherein the first target image is generated by capturing a target image projected by a projector by a first camera, and the second target image is generated by capturing a target image projected by the projector by a second camera, wherein the projector is disposed between the first camera and the second camera, and the distance between the projector and the first camera is greater than the distance between the projector and the second camera; performing matching processing on the first target image and the first reference image to obtain a first matching processing result, and obtaining first depth information based on the first matching processing result and the distance between the projector and the first camera; performing matching processing on the second target image and the second reference image to obtain a second matching processing result, and obtaining second depth information based on the first matching processing result and the distance between the projector and the second camera.

[0018] In one embodiment of the present disclosure, binocular matching processing is performed on the first target image and the second target image to obtain a binocular matching processing result, and third depth information is obtained based on the binocular matching processing result.

[0019] In one embodiment of the present disclosure, the depth calculation method also includes: performing depth fusion processing on the first depth information and the second depth information to obtain first optimized depth information or performing depth fusion processing on the second depth information and the third depth information to obtain second optimized depth information.

[0020] In an embodiment of the present disclosure, the depth calculation method further includes: performing depth fusion processing on the first depth information and the third depth information to obtain third optimized depth information.

[0021] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed, the depth calculation method of any one of the third aspects is implemented.

[0022] In a fifth aspect, an embodiment of the present disclosure provides an electronic device, comprising: a memory configured to store a processor executable program; and a processor configured to call the program to execute any one of the depth calculation methods of the third aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 Shown is a schematic diagram of the structure of the optical system described in an embodiment of the present disclosure.

[0024] Figure 2Shown is a flowchart of the depth calculation algorithm described in an embodiment of the present disclosure.

[0025] Figure 3 Shown is a schematic diagram of calculating the first depth information according to an embodiment of the present disclosure.

[0026] Figure 4 Shown is a schematic diagram of calculating the third depth information according to an embodiment of the present disclosure.

[0027] Figure 5 Shown is a schematic diagram of the structure of the electronic device described in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0028] The following is an explanation of the embodiments of the present disclosure by specific examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the contents disclosed in this specification. The present disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0029] It should be noted that the illustrations provided in the following embodiments are only used to schematically illustrate the basic concept of the present disclosure. Therefore, the drawings only show components related to the present disclosure rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0030] The technical solutions in the embodiments of the present disclosure are described in detail below in conjunction with the drawings in the embodiments of the present disclosure.

[0031] The current 3D camera groups, including single infrared camera structured light solutions and active projection binocular structured light solutions, are limited by chip computing power and cannot achieve close-range depth search while performing long-range depth search, which leads to obstacles in actual product applications.

[0032] By setting a far camera and a near camera on both sides of the projector in the optical system, the function of simultaneous identification of far and near distances can be realized according to the first camera and the second camera, and by setting the second camera to realize close-range depth search, the computing power consumption of the chip can be further reduced.

[0033] Figure 1 Schematic diagram showing the structure of an optical system according to an embodiment of the present disclosure. Figure 1As shown, the optical system 10 includes a projector 110, a first camera 120 and a second camera 130. The projector 110 is configured to project light to a target object. The first camera 120 is configured to capture an image of the target object to obtain a first target image associated with first depth information. The second camera 130 is configured to capture an image of the target object to obtain a second target image associated with second depth information. The projector 110 is disposed between the first camera 120 and the second camera 130, and the distance between the projector 110 and the first camera 120 is greater than the distance between the projector 110 and the second camera 130. Figure 1 In the figure, B1 represents the distance between the projector 110 and the first camera 120, and B2 represents the distance between the projector 110 and the second camera 130, wherein B1>B2.

[0034] In some implementations, the first camera 120 , the second camera 130 , and the projector 110 are located on the same baseline.

[0035] In some implementations, the image of the target object may refer to an image formed by interference imaging of the light beam projected by the projector 110 and diffuse reflection of the surface of the target object. For example, when the projector 110 is a speckle projector, the image of the target object may be a speckle image composed of scattered spots formed by interference imaging of the light beam projected by the speckle projector and diffuse reflection of the surface of the target object. The target object is the object for which depth information needs to be measured.

[0036] In some implementations, the first camera 120 is configured to perform a speckle search within a first depth distance range to acquire the first target image, and the second camera 130 is configured to perform a speckle search within a second depth distance range to acquire the second target image, and the maximum distance value within the second depth distance range is smaller than the maximum distance value within the first depth distance range.

[0037] In some implementations, the first depth distance range may be 0.4m-10m, and the second depth distance range may be 0.05m-0.5m. The first depth distance may refer to the depth distance of the center point of the projected image block in the first target image, the second depth distance may refer to the depth distance of the center point of the projected image block in the second target image, and the depth distance may refer to the depth distance between the corresponding position of the center point of the projected image block on the target object and the projector. The projected image block may refer to an image block used for matching calculation in the first target image or the second target image during the image matching process. The speckle search process may refer to the process of the camera collecting the speckle image composed of the above speckles.

[0038] In some implementations, the first depth information may include the depth distance of the center points of several projected image blocks in the first target image, and the first target image associated with the first depth information may refer to the image to which the projected image block associated with the first depth information belongs. The second depth information may include the depth distance of the center points of several projected image blocks in the second target image, and the second target image associated with the second depth information may refer to the image to which the projected image block associated with the second depth information belongs. For example, the image to which the projected image block associated with the second depth information belongs is the second target image.

[0039] Preferably, the first camera 120 and the second camera 130 are provided with filters, and the filters are used to enable the first camera 120 and the second camera 130 to only receive the target object image projected by the projector within a certain wavelength range.

[0040] In some implementations, the first camera 120 is a first infrared camera, the second camera 130 is a second infrared camera, the sensor of the first infrared camera is the same as the sensor of the second infrared camera, and the lens of the first infrared camera is the same as the lens of the second infrared camera.

[0041] In some implementations, since the first camera 120 and the second camera 130 are located at different positions, the first target image and the second target image acquired by the first camera 120 and the second camera 130 respectively capturing images of the target object are different.

[0042] In some implementations, the projector includes a vertical cavity surface emitting laser or an edge emitting laser, the emission spatial distribution of the projector is a pseudo-randomly distributed infrared speckle, and the wavelength range of the light projected by the projector is 800nm ​​to 1550nm.

[0043] In some implementations, the number of scattered spots emitted by the projector is between 1,000 and 50,000.

[0044] In some implementations, the distance between the projector and the first camera is between 30 mm and 100 mm, and the distance between the projector and the second camera is between 5 mm and 20 mm.

[0045] In some implementations, the optical system 10 may further include a chip 140. The chip 140 is configured to acquire the first depth information based on the first target image and a first reference image corresponding to the first target image, and acquire the second depth information based on the second target image and a second reference image corresponding to the second target image.

[0046] In some implementations, the first reference image corresponding to the first target image may refer to a reference image pre-collected and fixed by the first camera 120. The reference image may be a reference plane that is perpendicular to the optical center axis of the projector 110 and at a certain distance from the projector 110 and that is projected by the projector 110 to form a fixed pattern. The reference plane may be composed of a projection cloth or a flat plate. The static image collected by the first camera 120 and stored in the memory after pre-processing is the first reference image. Similarly, the second reference image corresponding to the second target image may refer to a reference image pre-collected and fixed by the second camera 130. The generation principle of the second reference image is similar to that of the first reference image, which will not be described in detail here.

[0047] In some implementations, the chip 140 is further configured to obtain third depth information based on the first target image, the second target image, and the texture information associated with the target object. The third depth information may refer to the depth distance between the center points of several projected image blocks at the same position in the first target image and the second target image. Among them, the projected image block at the same position in the second target image as that in the first target image may refer to the projected image block in the second target image that has the highest similarity with the projected image block in the first target image, and the same position may be determined based on the texture information associated with the target object, and the texture information associated with the target object may refer to the texture information of the target object on the first target image and the second target image. The range of the third depth information may be 0.5m-20m, and by obtaining the third depth information, the optical system 10 may be applied to outdoor scenes.

[0048] In some implementations, the chip 140 is further configured to perform deep fusion on the second depth information and the third depth information or to perform deep fusion on the first depth information and the second depth information. Performing deep fusion on the second depth information and the third depth information may refer to integrating and merging the second depth information and the third depth information to obtain more comprehensive depth information. Performing deep fusion on the first depth information and the second depth information may refer to integrating and merging the first depth information and the second depth information to obtain more comprehensive depth information.

[0049] In some implementations, the chip 140 is further configured to perform depth fusion on the first depth information and the third depth information. Performing depth fusion on the first depth information and the third depth information may refer to filtering out depth information with lower confidence from the first depth information and the third depth information to obtain more accurate depth information.

[0050] In some implementations, the chip 140 is further configured to generate a first depth map based on the first depth information, the first depth map may refer to a two-dimensional image storing depth values ​​of all pixels in the first target image, the range of the first depth map is the same as the first depth distance range, and may be 0.4m-10m. The chip 140 is further configured to generate a second depth map based on the second depth information, the second depth map may refer to a two-dimensional image storing depth values ​​of all pixels in the second target image, the range of the second depth map is the same as the second depth distance range, and may be 0.05m-0.5m.

[0051] In some implementations, the optical system 10 further includes a color camera 150. Figure 1 The chip 140 is connected to the first camera 120, the second camera 130 and the color camera 150, which means that the chip 140 is electrically connected to the first camera 120, the chip 140 is electrically connected to the second camera 130, and the chip 140 is electrically connected to the color camera 150.

[0052] In another aspect of the present disclosure, an electronic device is provided, which includes the optical system 10 as described above. In some implementations, the electronic device includes a sweeper or a robot. In some implementations, the sweeper or the robot includes the optical system 10.

[0053] Figure 2 is a flow chart showing a depth calculation algorithm according to an embodiment of the present disclosure. Figure 2The depth calculation method can be applied to a chip, and the depth calculation method includes the following steps S11 to S13.

[0054] In S11, a first target image, a first reference image corresponding to the first target image, a second target image and a second reference image corresponding to the second target image are obtained, wherein the first target image is generated by capturing an image of a target object projected by a projector with a first camera, and the second target image is generated by capturing an image of a target object projected by the projector with a second camera, wherein the projector is disposed between the first camera and the second camera, and a distance between the projector and the first camera is greater than a distance between the projector and the second camera.

[0055] In S12, matching processing is performed on the first target image and the first reference image to obtain a first matching processing result, and first depth information is obtained based on the first matching processing result and the distance between the projector and the first camera.

[0056] In S13, matching processing is performed on the second target image and the second reference image to obtain a second matching processing result, and second depth information is obtained based on the first matching processing result and the distance between the projector and the second camera.

[0057] It should be understood that, in some implementations, the depth calculation method may also include various operations or actions performed by the chip 140 as described above.

[0058] Figure 3 is a schematic diagram showing how to calculate the first depth information according to an embodiment of the present disclosure. In some implementations, see Figure 3 , the first matching processing result may refer to the translation amount of the image block used for matching calculation in the first target image, that is, Δy. The implementation method of acquiring the first depth information based on the first matching processing result and the distance between the projector and the first camera may include: acquiring the first depth information based on a depth distance calculation formula, the first matching processing result, and the distance between the projector and the first camera. The depth distance calculation formula is a relationship between the first depth information and the first matching processing result, the distance between the projector and the first camera, and the focal length of the first camera, which will not be described in detail in this embodiment.

[0059] In some implementations, the second matching process result may refer to the translation amount of the image block used for matching calculation in the second target image. The implementation method of acquiring the second depth information based on the second matching process result and the distance between the projector and the second camera may include: acquiring the second depth information based on a depth distance calculation formula, the second matching process result, and the distance between the projector and the second camera. The depth distance calculation formula is a relationship between the second depth information and the second matching process result, the distance between the projector and the second camera, and the focal length of the second camera, which is not described in detail in this embodiment.

[0060] In some implementations, the first depth information may include the depth distance of the center points of several projected image blocks in the first target image, and the second depth information may include the depth distance of the center points of several projected image blocks in the second target image. The depth distance may refer to the depth distance between the corresponding position of the center point of the projected image block on the target object and the projector. The projected image block may refer to an image block used for matching calculation in the first target image or the second target image during the matching process. For example, the first depth information may include the depth distances of the center points of the projected image blocks at three locations, d 1,1 ,d 1,2 and d 1,3 The second depth information may include the depth distances of the center point of the projected image block at the same position as the three positions, which are d 2,1 ,d 2,2 and d 2,3 .

[0061] In some implementations, the depth calculation method further includes: performing binocular matching processing on the first target image and the second target image to obtain a binocular matching processing result, and obtaining third depth information based on the binocular matching processing result.

[0062] In some implementations, the third depth information may refer to the depth distance of the center points of a plurality of projected image blocks at the same position in the first target image and the second target image. It should be noted that although the first depth information, the second depth information, and the third depth information all represent the depth distance of the center point of the projected image block, the first depth information, the second depth information, and the third depth information may be different due to different data processing processes.

[0063] Figure 4 is a schematic diagram showing how to calculate the third depth information according to an embodiment of the present disclosure. Figure 4, the binocular matching processing result may refer to the difference between the coordinate value of the center point of the projected image block at the same position in the first target image and the coordinate value in the second target image, that is, Figure 4 In x1-x2, x1 represents the coordinate value of the center point of the projected image block at the same position in the first target image, and x2 represents the coordinate value of the center point of the projected image block at the same position in the second target image.

[0064] In some implementations, the method for acquiring the third depth information based on the binocular matching processing result includes: acquiring the third depth information based on the binocular matching processing result and required parameters of a binocular depth distance calculation formula.

[0065] The binocular depth distance calculation formula can be expressed as:

[0066]

[0067] Wherein, B is the distance between the optical center of the first camera and the optical center of the second camera, the focal length of the first camera is equal to the focal length of the second camera, and f is the focal length of the first camera. The required parameters include the distance between the optical center of the first camera and the optical center of the second camera and the focal length of the first camera.

[0068] In some implementations, the first depth information and the second depth information are depth-fused to obtain first optimized depth information, or the second depth information and the third depth information are depth-fused to obtain second optimized depth information.

[0069] In some implementations, the first optimized depth information may refer to the depth information after the first depth information is optimized. The implementation method of performing depth fusion processing on the first depth information and the second depth information to obtain the first optimized depth information includes: determining whether the area associated with each depth distance in the first depth information in the first target image is a shadow area, and if so, replacing the depth distance in the first depth information with the corresponding depth distance in the second depth information. Whether the area associated with the depth distance in the first depth information in the first target image is a shadow area may refer to whether the center point of the projected image block corresponding to the depth distance is in a shadow area. The shadow area may refer to an unclear area. The corresponding depth distance in the second depth information may refer to the depth distance of the center point of the projected image block at the same position as the depth distance in the first depth information. In addition, the process of performing depth fusion processing on the first depth information and the second depth information also includes the process of integrating and merging the first depth information and the second depth information, which will not be repeated here. By performing depth fusion processing on the first depth information and the second depth information, the search range of the depth distance can be improved. For example, the range of the first depth information is 0.4m-10m, the range of the second depth information is 0.1m-0.5m, and the first optimized depth information is 0.1m-10m.

[0070] In some implementations, the second optimized depth information may refer to the depth information after the second depth information is optimized, and the second depth information and the third depth information are subjected to depth fusion processing to obtain the second optimized depth information. The implementation method includes: determining whether the area associated with each depth distance in the second depth information in the second target image is a shadow area, and if so, replacing the depth distance in the second depth information with the corresponding depth distance in the third depth information. Whether the area associated with the depth distance in the second depth information in the second target image is a shadow area may refer to whether the center point of the projected image block corresponding to the depth distance is in a shadow area. The corresponding depth distance in the third depth information may refer to the depth distance of the center point of the projected image block at the same position as the depth distance in the second depth information. In addition, the process of performing depth fusion processing on the second depth information and the third depth information also includes the process of integrating and merging the first depth information and the second depth information, which will not be repeated here. By performing depth fusion processing on the second depth information and the third depth information, the search range of the depth distance can be improved. For example, the range of the second depth information is 0.1m-0.5m, the range of the third depth information is 0.5m-20m, and the second optimized depth information is 0.1m-20m.

[0071] In some implementations, the depth calculation method further includes: performing a depth fusion process on the first depth information and the third depth information to obtain third optimized depth information. The third optimized depth information may be the depth information with a higher confidence level between the first depth information and the third depth information. By performing a depth fusion process on the first depth information and the third depth information to obtain the third optimized depth information, the depth effect can be further improved and the depth holes can be reduced. Determining the depth information with a higher confidence level between the first depth information and the third depth information can be achieved by a depth restoration algorithm, which will not be described in detail in this embodiment.

[0072] The protection scope of the depth calculation algorithm described in the embodiment of the present disclosure is not limited to the execution order of the steps listed in this embodiment. All solutions implemented by adding, reducing or replacing steps in the prior art based on the principles of the present disclosure are included in the protection scope of the present disclosure.

[0073] In the several embodiments provided in the present disclosure, it should be understood that the disclosed device or method can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules / units is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules or units can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules or units, which can be electrical, mechanical or other forms.

[0074] The modules / units described as separate components may or may not be physically separated, and the components displayed as modules / units may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules / units may be selected according to actual needs to achieve the purpose of the embodiments of the present disclosure. For example, the functional modules / units in the various embodiments of the present disclosure may be integrated into one processing module, or each module / unit may exist physically separately, or two or more modules / units may be integrated into one module / unit.

[0075] Those of ordinary skill in the art should further appreciate that the units and steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this disclosure.

[0076] Figure 5 Schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure. Figure 5 As shown, the electronic device 500 includes a memory 510 and a processor 520. The memory 510 is configured to store a processor executable program. The processor 520 is configured to call the program to execute the above reference Figure 2 Describes the depth calculation method.

[0077] The present disclosure embodiment also provides a computer-readable storage medium. A person of ordinary skill in the art can understand that all or part of the steps in the method for implementing the above embodiment can be completed by instructing the processor through a program, and the program can be stored in a computer-readable storage medium, and the storage medium is a non-transitory medium, such as a random access memory, a read-only memory, a flash memory, a hard disk, a solid-state hard disk, a magnetic tape, a floppy disk, an optical disc, and any combination thereof. The above storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a digital video disc (DVD)), or a semiconductor medium (e.g., a solid-state disk (SSD)), etc.

[0078] The embodiments of the present disclosure may also provide a computer program product, which includes one or more computer instructions. When the computer instructions are loaded and executed on a computing device, the process or function described in the embodiments of the present disclosure is generated in whole or in part. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website site, computer or data center to another website site, computer or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means.

[0079] When the computer program product is executed by a computer, the computer executes the method described in the above method embodiment. The computer program product may be a software installation package, and when the above method is required, the computer program product may be downloaded and executed on a computer.

[0080] The descriptions of the processes or structures corresponding to the above-mentioned figures have different emphases. For parts that are not described in detail in a certain process or structure, please refer to the relevant descriptions of other processes or structures.

[0081] The above embodiments are merely illustrative of the principles and effects of the present disclosure, and are not intended to limit the present disclosure. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present disclosure. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed in the present disclosure shall still be covered by the claims of the present disclosure.

Claims

1. An optical system, characterized in that: include: a projector configured to project light onto a target object; A first camera is configured to capture an image of the target object to obtain a first target image associated with the first depth information; as well as A second camera is configured to capture an image of the target object to obtain a second target image associated with the second depth information, The projector is disposed between the first camera and the second camera, and the distance between the projector and the first camera is greater than the distance between the projector and the second camera.

2. The optical system according to claim 1, characterized in that The first camera is configured to perform a speckle search within a first depth distance range to acquire the first target image, and the second camera is configured to perform a speckle search within a second depth distance range to acquire the second target image, wherein a maximum distance value within the second depth distance range is smaller than a maximum distance value within the first depth distance range.

3. The optical system according to claim 1, characterized in that The first camera is a first infrared camera, the second camera is a second infrared camera, the sensor of the first infrared camera is the same as the sensor of the second infrared camera, and the lens of the first infrared camera is the same as the lens of the second infrared camera.

4. The optical system according to claim 1, characterized in that The projector comprises a vertical cavity surface emitting laser or an edge emitting laser, the emission spatial distribution of the projector is a pseudo-randomly distributed infrared speckle, and the wavelength range of the light projected by the projector is 800nm ​​to 1550nm.

5. The optical system according to claim 4, characterized in that The number of scattered spots emitted by the projector is 1000 to 50000 points.

6. The optical system according to claim 1, characterized in that The distance between the projector and the first camera is between 30 mm and 100 mm, and the distance between the projector and the second camera is between 5 mm and 20 mm.

7. The optical system according to claim 1, characterized in that Also includes: The chip is configured to obtain the first depth information based on the first target image and a first reference image corresponding to the first target image, and to obtain the second depth information based on the second target image and a second reference image corresponding to the second target image.

8. The optical system according to claim 7, characterized in that The chip is further configured to acquire third depth information based on the first target image, the second target image, and texture information associated with the target object.

9. The optical system according to claim 8, characterized in that The chip is further configured to perform depth fusion on the second depth information and the third depth information or to perform depth fusion on the first depth information and the second depth information.

10. The optical system according to claim 8, characterized in that The chip is further configured to perform depth fusion on the first depth information and the third depth information.

11. The optical system according to claim 1, characterized in that Also includes a color camera.

12. An electronic device, characterized in that: An optical system comprising any one of claims 1 to 11.

13. The electronic device according to claim 12, characterized in that: The electronic device includes a sweeper or a robot.

14. A depth calculation method, characterized in that: include: Acquire a first target image, a first reference image corresponding to the first target image, a second target image, and a second reference image corresponding to the second target image, wherein the first target image is generated by capturing a target image projected by a projector through a first camera, and the second target image is generated by capturing the target image projected by the projector through a second camera, wherein the projector is disposed between the first camera and the second camera, and a distance between the projector and the first camera is greater than a distance between the projector and the second camera; Performing matching processing on the first target image and the first reference image to obtain a first matching processing result, and obtaining first depth information based on the first matching processing result and the distance between the projector and the first camera; as well as Matching processing is performed on the second target image and the second reference image to obtain a second matching processing result, and second depth information is obtained based on the first matching processing result and the distance between the projector and the second camera.

15. The depth calculation method according to claim 14, characterized in that: Also includes: A binocular matching process is performed on the first target image and the second target image to obtain a binocular matching process result, and third depth information is obtained based on the binocular matching process result.

16. The depth calculation method according to claim 15, characterized in that: Also includes: The first depth information and the second depth information are subjected to depth fusion processing to obtain first optimized depth information, or the second depth information and the third depth information are subjected to depth fusion processing to obtain second optimized depth information.

17. The depth calculation method according to claim 15, characterized in that: Also includes: A depth fusion process is performed on the first depth information and the third depth information to obtain third optimized depth information.

18. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed, the depth calculation method according to any one of claims 14 to 17 is implemented.

19. An electronic device, characterized in that: include: a memory configured to store a processor executable program; as well as A processor is configured to call the program to execute the depth calculation method according to any one of claims 14 to 17.

Images