Amphibious binocular depth camera and electronic equipment

Abstract

The invention discloses an amphibious binocular depth camera and electronic equipment. The structured light projector is used for projecting structured light rays; the first receiver is used for receiving a reflection signal of the structured light rays and generating a first image; the second receiver is used for receiving a reflection signal of the structured light rays and generating a second image; the processor is used for receiving an overwater or underwater signal and correspondingly reading an overwater calibration file or an underwater calibration file; performing image mapping on the first image and the second image according to the overwater calibration file or the underwater calibration file to generate a left image and a right image meeting binocular epipolar constraint; generating a disparity map according to the left image and the right image; and generating a depth map according to the internal reference and the external reference of the overwater calibration file or the underwater calibration file. According to the invention, the imaging problem of a traditional camera during switching between land and water environments can be effectively solved.

Description

Technical Field

[0001] The present invention relates to the technical field of depth cameras, and in particular to an amphibious binocular depth camera and electronic equipment. Background Art

[0002] In numerous fields, such as ocean exploration, underwater archaeology, water rescue, and some industrial scenarios requiring amphibious operations, there is an urgent need for imaging equipment that can accurately capture environmental information. Traditional camera equipment, whether purely surface or underwater, has significant limitations. When used underwater, conventional surface cameras experience significant image distortion and reduced resolution due to the significant differences between the optical properties of water, such as refraction and scattering, and those of air. This makes it impossible to accurately capture critical information such as the location, shape, and size of underwater objects. Underwater cameras, when used afloat, struggle to adapt to the lighting conditions and imaging requirements of air, similarly failing to provide clear, usable images.

[0003] As amphibious operations continue to demand ever-increasing precision and efficiency, the development of a camera system capable of stable and accurate operation both above and below water has become a necessity. Existing camera technology cannot meet the imaging requirements in these complex environments, so a novel amphibious binocular depth camera solution is urgently needed.

[0004] The disclosure of the above background technology content is only used to assist in understanding the inventive concept and technical solution of the present invention. It does not necessarily belong to the prior art of this patent application. In the absence of clear evidence that the above content has been disclosed on the filing date of this patent application, the above background technology should not be used to evaluate the novelty and creativity of this application. Summary of the Invention

[0005] To this end, the present invention ensures that high-quality depth images can be generated in different environments through unique structural design and image processing methods, and can effectively solve the imaging problems of traditional cameras when switching between water and land environments.

[0006] In a first aspect, the present invention provides an amphibious binocular depth camera, characterized in that it includes:

[0007] Waterproof housing;

[0008] A structured light projector, used for projecting structured light;

[0009] A first receiver, configured to receive a reflection signal of the structured light and generate a first image;

[0010] a second receiver, configured to receive a reflection signal of the structured light and generate a second image;

[0011] A processor is configured to receive signals above or below water and read an above-water calibration file or an underwater calibration file accordingly; perform image mapping on the first image and the second image according to the above-water calibration file or the underwater calibration file to generate a left image and a right image that satisfy a binocular epipolar constraint; generate a disparity map according to the left image and the right image; and generate a depth map according to intrinsic and extrinsic parameters of the above-water calibration file or the underwater calibration file.

[0012] Optionally, the amphibious binocular depth camera is characterized in that the mapping matrix and internal parameters of the above-water calibration file are different from those of the underwater calibration file.

[0013] Optionally, the amphibious binocular depth camera is characterized in that, compared with the above-water calibration file, the mapping matrix of the underwater calibration file also includes an underwater distortion compensation matrix.

[0014] Optionally, the amphibious binocular depth camera is characterized in that, compared with the above-water calibration file, the internal parameters of the underwater calibration file also include a refraction correction coefficient.

[0015] Optionally, the amphibious binocular depth camera is characterized in that the structured light is a combination of one or more of a Gray code pattern, a stripe pattern or a speckle pattern.

[0016] Optionally, the amphibious binocular depth camera is characterized in that it further includes a sensor for detecting whether the camera is located above water or underwater, and the sensor is electrically connected to the processor to transmit the detection signal to the processor.

[0017] Optionally, the amphibious binocular depth camera is characterized in that the sensor is one of a water pressure sensor, an optical sensor or a capacitive sensor.

[0018] Optionally, the amphibious binocular depth camera is characterized in that the waterproof housing includes: an optical window assembly, including a parallel waterproof glass layer and a refractive compensation prism layer, and the prism layer has an inclination angle that matches the refractive index of the water body.

[0019] Optionally, the amphibious binocular depth camera is characterized in that the structured light projector includes switchable: infrared structured light emission module and blue-green laser emission module; wherein,

[0020] The infrared structured light emission module is used for active light projection in an aquatic environment;

[0021] The blue-green laser emission module is used for light penetration in underwater environments;

[0022] The emission angle of the blue-green laser emission module is adjustable to compensate for the light spot diffusion caused by underwater light scattering.

[0023] In a second aspect, the present invention provides an electronic device, characterized in that it includes any of the aforementioned amphibious binocular depth cameras.

[0024] Compared with the prior art, the present invention has the following beneficial effects:

[0025] The present invention has a waterproof housing that can easily cope with two completely different environments, above and below water. Whether on the surface or underwater, it can ensure that the internal precision components are not damaged and can work normally, breaking through the limitation of traditional cameras' single environmental adaptability.

[0026] The structured light projector of the present invention works in conjunction with a first receiver and a second receiver to accurately receive the reflected signal of the structured light and generate an image. By leveraging the high brightness of structured light and the stability of a binocular camera, the projector can more clearly capture object details in complex aquatic and terrestrial environments, accurately acquiring key information such as the object's position, shape, and size. This robustness solves the problems of blurred imaging and inaccurate information acquisition associated with traditional cameras in both aquatic and terrestrial environments.

[0027] The processor in this invention intelligently reads the corresponding calibration file based on the above-water or underwater signal. Based on this, it maps the generated image to produce left and right images that meet the binocular epipolar constraints, and further generates a disparity map and depth map. This intelligent, targeted image processing process significantly improves image quality and the accuracy of depth information, ensuring the generation of high-quality depth images in a variety of environments, an advantage not possessed by traditional cameras.

[0028] The hardware of the present invention is compatible with both surface and underwater modes, and the algorithms of the two processing modes are also fully compatible. The only difference is the mapping matrix and internal parameters used for the polar constraints. The processes are exactly the same, which greatly improves the integration, modularity and simplicity of the algorithm. It can be used in soc (system on chip) with DPU (data processor) function and dedicated depth chip, and has very good adaptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the embodiments or the description of the prior art. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without inventive work. Other features, purposes and advantages of the present invention will become more apparent by reading the detailed description of the non-limiting embodiments with reference to the following drawings:

[0030] Figure 1 Schematic diagram of the structure of an amphibious binocular depth camera according to an embodiment of the present invention;

[0031] Figure 2 Schematic diagram of the structure of an optical window assembly according to an embodiment of the present invention;

[0032] Figure 3 Schematic diagram of the structure of another amphibious binocular depth camera in an embodiment of the present invention.

[0033] 1- Waterproof housing;

[0034] 2-Structured light projector;

[0035] 3- First receiver;

[0036] 4- Second receiver;

[0037] 5-Processor;

[0038] 6- Optical window assembly;

[0039] 7- Waterproof glass layer;

[0040] 8-refractive compensation prism layer;

[0041] 9-Infrared structured light emission module;

[0042] 10-blue-green laser emission module; DETAILED DESCRIPTION

[0043] The present invention will be described in detail below with reference to specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those skilled in the art, several variations and improvements can be made without departing from the scope of the present invention. These all fall within the scope of protection of the present invention.

[0044] The terms "first," "second," "third," "fourth," and the like (if any) in the description and claims of the present invention and in the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of the invention described herein, for example, can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having," and any variations thereof, are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or apparatus.

[0045] An amphibious binocular depth camera provided in an embodiment of the present invention is intended to solve the problems existing in the prior art.

[0046] The following describes in detail the technical solutions of the present invention and how the technical solutions of this application solve the above-mentioned technical problems using specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments. The following embodiments of the present invention are described in conjunction with the accompanying drawings.

[0047] Since water and air have different effects on the light path in above-water and underwater scenes, depth cameras in the prior art cannot simultaneously address the depth acquisition issues above and below water.

[0048] The present invention uses structured light as an active light source and a binocular composed of a first receiver and a second receiver to obtain depth data, and automatically performs corresponding processing modes according to underwater or underwater signals to obtain accurate above-water or underwater depth maps.

[0049] Figure 1 FIG. 1 is a schematic diagram of the structure of an amphibious binocular depth camera according to an embodiment of the present invention. Figure 1 As shown, an amphibious binocular depth camera in an embodiment of the present invention includes:

[0050] Waterproof housing 1.

[0051] Specifically, the waterproof housing, as the camera's external protective structure, has the core function of ensuring the camera's normal operation both above and below water, preventing water from entering the camera and damaging other electronic components. It provides a safe and stable physical environment for the entire camera system, capable of withstanding a certain level of water pressure and ensuring the camera's sealing and reliability at various underwater depths.

[0052] Waterproof enclosures are typically made of high-strength, water-resistant, and corrosion-resistant materials, such as specialized engineering plastics or metal alloys. Sealing devices, such as sealants and rings, may be used at the joints to ensure good waterproofing. The enclosure design also considers factors such as heat dissipation, mounting, and ease of use. This may include providing cooling holes (with waterproofing measures), increasing the heat dissipation area, and installing interfaces.

[0053] The structured light projector 2 is used to project structured light.

[0054] Specifically, a structured light projector is used to project structured light onto a target scene. This structured light has a specific coded pattern, such as stripes or dots. By projecting this specially structured light, it reflects when it hits an object's surface. The reflected light at different locations carries three-dimensional information about the object's surface. The structured light projector is a key component in acquiring three-dimensional information about the target object, providing the fundamental information source for subsequent image acquisition and depth calculation.

[0055] Structured light is generally based on light source technologies such as lasers or LEDs. The light emitted by the light source is modulated by optical elements (such as lenses and gratings) to form the desired structured light pattern and project it. For example, a laser-based structured light projector can produce high-brightness, highly directional structured light, which can be effectively projected onto the target object at a long distance and in complex environments. Compared to traditional solutions that use high-power lasers underwater, structured light projectors have a shorter measurement distance and can provide more information.

[0056] The first receiver 3 is configured to receive a reflection signal of the structured light and generate a first image.

[0057] Specifically, the first receiver is responsible for receiving the reflected signal of the structured light projected by the structured light projector and reflected by the target object. Based on the received reflected signal, it converts the optical signal into an electrical signal and further processes it to generate a first image. This first image contains partial optical information about the target object's surface and is one of the important data for subsequent depth calculation.

[0058] The first receiver typically uses an image sensor (such as a CCD or CMOS sensor) as its core component. The pixels on the image sensor can sense information such as light intensity and color. When reflected light strikes a pixel, it generates a corresponding electrical signal. After amplification, filtering, and analog-to-digital conversion by the signal processing circuit, the electrical signal is converted into digital image data, thereby generating the first image.

[0059] The second receiver 4 is configured to receive a reflection signal of the structured light and generate a second image.

[0060] Specifically, similar to the first receiver, the second receiver is used to receive the reflected signal of the structured light and generate a second image. This second image, combined with the first image, provides the binocular vision system with image information of the target object from different perspectives. By analyzing and processing these two images, the depth information of the target object can be calculated, enabling binocular vision's 3D reconstruction capabilities.

[0061] The second receiver operates on a similar principle to the first, also using an image sensor to receive reflected light and generate an image. However, to achieve the epipolar constraint of binocular vision, the two receivers must be mounted at specific locations and angles to ensure proper parallax information is captured.

[0062] Processor 5 is used to receive signals above or below the water surface and read the above-water calibration file or the underwater calibration file accordingly; perform image mapping on the first image and the second image according to the above-water calibration file or the underwater calibration file to generate a left image and a right image that satisfy the binocular epipolar constraint; generate a disparity map according to the left image and the right image; and generate a depth map according to the intrinsic and extrinsic parameters of the above-water calibration file or the underwater calibration file.

[0063] Specifically, the processor receives signals from the camera while operating above or below water, determines the camera's current environment (above or below water) based on these signals, and reads a pre-stored above-water or underwater calibration file accordingly. Because water's refraction and scattering properties of light differ from those of air, the camera's imaging characteristics vary in different environments, requiring different calibration files to correct and process the image. In this embodiment, the above-water and underwater processing differ only in the calibration files; otherwise, they remain identical, ensuring excellent compatibility between the above-water and underwater processing, with both software algorithms and hardware fully compatible.

[0064] Based on the read surface or underwater calibration file, an image mapping operation is performed on the first image generated by the first receiver and the second image generated by the second receiver. This image mapping operation matches corresponding points in the two images to satisfy the binocular epipolar constraints, thereby obtaining the required left and right images. Then, based on the positional differences between corresponding points in the left and right images, a disparity map is calculated. This disparity map reflects the positional differences of the target object under different viewing angles.

[0065] Using the intrinsic parameters (such as the camera's focal length and principal point coordinates) and extrinsic parameters (such as the camera's rotation and translation parameters) contained in the above-water or underwater calibration files, combined with the disparity map, a depth map is calculated using a specific algorithm. The depth map represents the distance from each point on the target object's surface to the camera and is critical data for applications such as 3D reconstruction and object detection.

[0066] The processor, typically a high-performance microprocessor or digital signal processor (DSP), runs specific software algorithms to implement the aforementioned functions. These algorithms include image preprocessing algorithms (such as filtering and denoising), image matching algorithms (such as feature point-based matching and region-based matching), disparity calculation algorithms (such as block matching and dynamic programming), and depth calculation algorithms. By executing these algorithms, the processor processes and analyzes the image data captured by the camera, ultimately obtaining the required depth information.

[0067] In some embodiments, the mapping matrices and internal parameters in the above-water and underwater calibration files differ. This difference in mapping matrices and internal parameters is due to the different effects of water and air on light propagation. This, combined with this technical solution, can significantly improve processing efficiency.

[0068] The mapping matrix primarily describes the transformation relationship between image coordinates and world coordinates. In binocular vision systems, it is also used for matching and correcting left and right images. Water and air have different optical properties. Light refracts when traveling through water, changing its path and thus affecting the position of objects in the image.

[0069] Water conditions: In aquatic environments, light primarily propagates in air, where the refractive index is relatively stable and close to 1. Light propagation is direct, and the transformation relationship described by the mapping matrix is relatively simple. It primarily considers factors such as the camera's installation position and posture, and lens distortion.

[0070] Underwater: When the camera is underwater, light enters the camera lens through the water. Because water has a higher refractive index than air, the light is refracted, resulting in a different image position than above water. Therefore, the underwater mapping matrix needs to account for the refraction effect of light at the water-air interface and perform additional corrections on the image coordinates to accurately reflect the true position of objects.

[0071] Due to the different mapping matrices, when performing image mapping, the processor needs to select the corresponding mapping matrix according to the current environment (above water or underwater), match and correct the first image and the second image to meet the binocular epipolar constraint, thereby obtaining accurate left and right images, providing a basis for subsequent disparity calculation and depth map generation.

[0072] Camera intrinsic parameters describe the camera's internal optical and geometric properties, primarily including focal length, principal point coordinates, and distortion coefficients. These parameters determine how the camera projects objects in the three-dimensional world onto the two-dimensional image plane. The optical properties of water affect the camera's imaging process, resulting in differences in camera intrinsic parameters above and below water.

[0073] Focal length change: Above water, the camera's focal length is designed and calibrated based on the air medium. Underwater, due to light refraction, the effective focal length of the camera lens changes. Water has a greater refractive index than air, causing light to bend more significantly upon entering the lens, resulting in a shorter effective focal length.

[0074] Principal point coordinate offset: The principal point is the center of the image plane, theoretically the intersection of the camera's optical axis and the image plane. Underwater, due to light refraction, the direction of the optical axis changes, causing the principal point's position on the image plane to shift.

[0075] Distortion coefficient changes: Camera lenses typically exhibit certain distortions, such as radial and tangential distortion. Underwater, the lens distortion changes due to the altered light propagation path, requiring recalibration of the distortion coefficients.

[0076] Differences in intrinsic parameters directly affect depth map calculation. After generating a disparity map based on the left and right images, the camera's intrinsic and extrinsic parameters are used to convert the disparity information into depth information. Using incorrect intrinsic parameters can lead to inaccurate depth values, affecting the accuracy and reliability of 3D reconstruction. Therefore, when calculating the depth map, the processor needs to select the intrinsic parameters from the corresponding above-water or underwater calibration file based on the current environment to ensure the accuracy of the depth calculation.

[0077] In some embodiments, compared with the above-water calibration file, the mapping matrix of the underwater calibration file further includes an underwater distortion compensation matrix.

[0078] Underwater, light propagates under vastly different physical conditions than above water (in air). The refractive index of water is approximately 1.33, significantly higher than that of air (approximately 1.0003). When light passes from water into the air medium of a camera lens, it undergoes significant refraction. This refraction not only alters the light's path but also creates unique distortions in the image of objects.

[0079] On the one hand, because water's uniformity is more complex than air (it may experience density variations due to factors such as temperature and salinity), light may refract irregularly when traveling through water, further exacerbating image distortion. On the other hand, water's scattering of light can also affect image quality, causing blurring and distortion of image edges and details—all of these distortions are unique to underwater imaging.

[0080] The underwater distortion compensation matrix, as part of the underwater calibration file mapping matrix, is specifically used to correct for these special distortions caused by the underwater environment. Its main function is to reposition and transform the pixels in the image to offset the distortion caused by factors such as refraction and scattering.

[0081] Specifically, the underwater images captured by the camera are analyzed and processed to determine the type and degree of distortion, and then a corresponding underwater distortion compensation matrix is constructed. During image mapping, this matrix is combined with other transformation matrices (such as those describing the camera's pose and position) to transform the original image so that the transformed image more accurately reflects the object's true shape and position.

[0082] For example, for image edge stretching or compression distortion caused by refraction, the underwater distortion compensation matrix can restore the position of edge pixels to the correct position; for image blur and deformation caused by scattering, the matrix can enhance the clarity and accuracy of the image by adjusting the grayscale value and spatial position of the pixels.

[0083] In aquatic environments, although camera lenses may exhibit some distortion (such as radial and tangential distortion), this distortion is relatively simple and regular due to the uniformity and stability of the air medium, and can usually be compensated using conventional calibration methods and distortion correction matrices. However, the complexity of the underwater environment makes conventional calibration and correction methods alone incapable of meeting the requirements for accurate imaging. Therefore, a specialized underwater distortion compensation matrix must be introduced to address the additional distortion caused by the unique properties of water.

[0084] Therefore, compared with the above-water calibration file, the mapping matrix of the underwater calibration file contains the underwater distortion compensation matrix. This is to adapt to the complex underwater optical environment, improve the imaging quality and measurement accuracy of the camera underwater, and ensure that image-based depth calculation and three-dimensional reconstruction functions can be accurately implemented.

[0085] In some embodiments, the internal parameters of the underwater calibration file also include a refraction correction coefficient, compared to the above-water calibration file. As part of the internal parameters of the underwater calibration file, the refraction correction coefficient is primarily used to compensate for the effects of light refraction on the camera's internal parameters. It adjusts the camera's imaging parameters by comprehensively considering factors such as the light refraction angle and propagation path.

[0086] Specifically, the refraction correction factor can be used to modify the camera's focal length. Because light refraction underwater changes the camera's effective focal length, the refraction correction factor can be used to calculate the actual effective focal length based on the water's refractive index and the camera's optical structure, allowing for more accurate projection of 3D objects onto the 2D image plane.

[0087] The refraction correction factor also plays a role in the offset of the principal point coordinates. It adjusts the position of the principal point on the image plane based on the change in the optical axis direction caused by light refraction, so that the center of the image accurately corresponds to the camera's optical axis, improving the geometric accuracy of the image.

[0088] In addition, when dealing with special distortion caused by refraction, the refraction correction coefficient combined with other distortion correction parameters can more comprehensively correct the distortion of the image, making the shape and position of objects in the image closer to the actual situation.

[0089] The internal parameters of the above-water calibration file are primarily based on the inherent optical properties of the camera in air, without considering the additional effects of light refraction. The internal parameters of the underwater calibration file, on the other hand, include a refraction correction factor, fully accounting for the effect of light refraction on camera imaging in underwater environments. This allows the camera to more accurately capture image information of objects underwater.

[0090] This difference reflects the adaptability of the underwater calibration file to the special underwater optical environment. By introducing the refraction correction coefficient, the camera's underwater measurement accuracy and reliability can be improved, providing a more accurate data basis for applications such as depth calculation and three-dimensional reconstruction based on underwater images.

[0091] In some embodiments, the structured light is a combination of one or more of a Gray code pattern, a stripe pattern, or a speckle pattern.

[0092] Gray code is a special binary encoding method in which adjacent codes differ by only one binary digit. In structured light projection, different Gray code patterns are sequentially projected onto the target object. The camera receives the reflected light and records each image. By analyzing the encoding status of pixels in different images based on the Gray code's encoding rules, the unique spatial position of each pixel can be determined, enabling precise surface measurement.

[0093] Gray code patterns offer high encoding accuracy and robustness. Because adjacent codes differ by only one bit, even in the presence of noise or partial image loss during image acquisition and processing, accurate encoding information can be recovered through a sound algorithm, yielding reliable 3D information about the object. This makes it suitable for applications requiring high precision, such as high-precision inspection of industrial parts and 3D modeling of tiny objects. However, the projection and processing of Gray code patterns is relatively complex, requiring the projection of multiple different patterns, resulting in lengthy measurement times and making it unsuitable for rapid measurement of dynamic objects.

[0094] A fringe pattern is typically a series of parallel or regularly arranged light and dark stripes. When projecting a fringe pattern onto an object's surface, variations in the surface's height cause the stripes to deform. By analyzing this deformation (such as offset and curvature), the depth information of each point on the surface can be calculated. Common analysis methods, such as phase measurement, extract and analyze the phase information of the fringe to obtain the object's three-dimensional profile.

[0095] The projection and processing of fringe patterns are relatively simple, resulting in fast measurement speeds. These applications are suitable for scenarios requiring high real-time performance, such as robot vision guidance and rapid 3D reconstruction of objects. Furthermore, fringe patterns can be tailored to varying measurement ranges and accuracy requirements by adjusting parameters such as the spacing and number of fringe patterns. However, complex textures or color variations on the surface of an object can disrupt the fringe pattern, making fringe identification and analysis difficult and thus affecting measurement accuracy.

[0096] A speckle pattern is composed of randomly distributed light spots. When projected onto an object's surface, the shape and distribution of the speckles change due to the surface's height fluctuations. By comparing speckle images from different viewing angles and using algorithms to match the speckle feature points, three-dimensional information about the object's surface can be calculated.

[0097] Speckle patterns are characterized by good randomness and uniqueness, and are highly adaptable to surface conditions, enabling accurate measurement even of surfaces with complex textures or color variations. Furthermore, speckle pattern projection does not require complex optical systems, resulting in low cost and suitability for cost-sensitive applications. However, speckle pattern matching algorithms are relatively complex and computationally intensive, placing high demands on processor performance.

[0098] Combining one or more of the following patterns—a Gray code pattern, a fringe pattern, or a speckle pattern—can leverage their respective strengths and offset the shortcomings of a single pattern. For example, in applications requiring both precision and speed, a Gray code pattern can be projected first for high-precision initial measurement, determining the object's general outline and position. A fringe pattern can then be projected for rapid detail addition and optimization, improving measurement efficiency. Alternatively, combining speckle patterns with other patterns can improve measurement reliability and adaptability when processing objects with complex surfaces.

[0099] In some embodiments, a sensor is further included for detecting whether the environment in which the camera is located is above water or underwater. The sensor is electrically connected to the processor and transmits the detection signal to the processor. During the operation of the amphibious binocular depth camera, it is crucial to accurately judge whether the environment in which the camera is located is above water or underwater. Because the optical properties above water and underwater are very different, as mentioned above, the refraction, scattering and other effects of water will affect the propagation of light, thereby changing the imaging characteristics of the camera. Therefore, it is necessary to call the corresponding above-water or underwater calibration file according to different environments to ensure that the camera can accurately perform image mapping, parallax calculation and depth map generation. The main function of the sensor is to detect the environment in which the camera is located in real time, and transmit the detection results to the processor in the form of an electrical signal, providing a basis for the processor to select an appropriate calibration file.

[0100] The sensor is a water pressure sensor, an optical sensor or a capacitive sensor.

[0101] Capacitive sensors use the difference in dielectric constant between water and air to detect the environment. Water has a much greater dielectric constant than air, and when the sensor comes into contact with water, its capacitance changes significantly. By measuring this change in capacitance, it's possible to determine whether the camera is underwater.

[0102] Advantages: simple structure, high sensitivity, fast response speed, and the ability to detect environmental changes quickly and accurately.

[0103] Disadvantages: It is susceptible to external electromagnetic interference and certain shielding measures need to be taken.

[0104] Pressure sensors operate based on the pressure difference between water and air. Underwater pressure is generated by gravity, while above water pressure is primarily atmospheric pressure. The pressure sensor measures the ambient pressure and compares it to a preset threshold to determine whether the camera is above or below water.

[0105] Advantages: high measurement accuracy, strong reliability, suitable for underwater environment detection at different depths.

[0106] Disadvantages: Requires precise calibration to ensure accurate pressure measurement at varying altitudes and water temperatures.

[0107] Optical sensors assess the environment by detecting differences in the propagation characteristics of light through water and air. For example, water absorbs and scatters light more strongly than air. Optical sensors can measure changes in light intensity, color, and other parameters to infer the camera's surroundings.

[0108] Advantages: Non-contact detection does not interfere with the environment and is suitable for some application scenarios with high hygiene requirements.

[0109] Disadvantages: It is easily affected by factors such as light intensity and light source color, and requires complex signal processing and calibration.

[0110] The sensor is electrically connected to the processor. When the sensor detects a change in the camera's environment, it transmits a detection signal (such as a voltage signal or current signal) to the processor. After receiving the signal, the processor analyzes and processes it to determine whether the camera is currently in an above-water or underwater environment.

[0111] If it is determined to be an aquatic environment, the processor will read the pre-stored aquatic calibration file and process the images generated by the first receiver and the second receiver according to the mapping matrix, internal parameters and other parameters in the file, including image mapping, disparity calculation and depth map generation.

[0112] If it is determined to be an underwater environment, the processor will read the underwater calibration file and use the special parameters contained therein, such as the underwater distortion compensation matrix and refraction correction coefficient, to process the image more accurately to adapt to the complex underwater optical environment and ensure the accuracy of the depth map.

[0113] Through the collaborative work of sensors and processors, the amphibious binocular depth camera can automatically adapt to different working environments, improving the camera's versatility and practicality, and providing strong support for three-dimensional measurement and imaging applications in above-water and underwater environments.

[0114] In some embodiments, Figure 2 As shown, the waterproof housing includes: an optical window assembly 6, which includes a waterproof glass layer 7 and a refractive compensation prism layer 8 arranged in parallel, and the prism layer has an inclination angle that matches the refractive index of the water body.

[0115] The waterproof glass layer is the outermost protective structure of the optical window assembly. Its primary function is to prevent water from entering the camera interior, protecting the optical and electronic components within the camera from water erosion. At the same time, the waterproof glass layer must have excellent optical properties to ensure smooth transmission of structured light and reflected light, minimize light reflection and absorption, and ensure image acquisition quality.

[0116] It is usually made of high-strength, high-transparency glass material with good wear resistance and impact resistance, and can withstand the pressure and possible collisions in underwater environments. In addition, the surface of the waterproof glass layer may be specially treated, such as coating, to reduce light reflection and increase transmittance.

[0117] In underwater environments, when light enters the camera through water, it refracts due to the different refractive indices of water and air. This changes the light's path, affecting the camera's image quality and depth measurement accuracy. The refraction-compensating prism layer primarily compensates for this refraction effect. By adjusting the refraction of light, the prism layer allows it to enter the camera along a path closer to that of air, thus reducing the interference of refraction on imaging.

[0118] The design of the prism layer is based on the principles of optical refraction. When light passes from one medium to another, it refracts according to the law of refraction. By properly designing the shape and angle of the prism, light can be refracted multiple times within the prism, ultimately adjusting the light's propagation path.

[0119] It is crucial that the prismatic layer's tilt angle matches the water's refractive index. Different water bodies (e.g., freshwater, seawater, etc.) have different refractive indices, which are also affected by factors such as water temperature and salinity. By matching the prismatic layer's tilt angle to the water's refractive index, we ensure that light entering the prismatic layer is refracted along a predetermined path, maximally compensating for the refractive effect at the water-air interface.

[0120] For example, if the inclination angle of the prism layer is designed reasonably, when light enters the prism layer from the water at a certain angle, after being refracted by the prism layer, its propagation direction can be closer to the propagation direction in the air, thereby reducing the distortion effect of light refraction on the image and improving the camera's underwater imaging quality and depth measurement accuracy.

[0121] In actual design, the optimal tilt angle of the prism layer must be determined by accurately measuring the refractive index of the water, combining it with the camera's optical system parameters, and using optical design software to simulate and calculate. At the same time, the range of refractive index variations in different water environments must also be considered to ensure that the prism layer can operate effectively within a certain refractive index fluctuation range.

[0122] In summary, the optical window assembly in the waterproof housing, through the combined design of a waterproof glass layer and a refraction-compensating prism layer, and the inclination angle of the prism layer matching the refractive index of the water body, can effectively compensate for the impact of underwater light refraction on imaging while ensuring the waterproof performance of the camera, thereby improving the camera's working performance in underwater environments.

[0123] In some embodiments, Figure 3 As shown, the structured light projector includes switchable: infrared structured light emission module 9 and blue-green laser emission module 10; wherein,

[0124] The infrared structured light emission module 9 is used for active light projection in an aquatic environment;

[0125] The blue-green laser emission module 10 is used for light penetration in underwater environments;

[0126] The emission angle of the blue-green laser emission module 10 is adjustable to compensate for the light spot diffusion caused by underwater light scattering.

[0127] Specifically, the infrared structured light emission module is mainly based on an infrared light source (such as an infrared LED or infrared laser). It modulates the infrared light through optical elements (such as diffractive optical elements (DOEs) and microlens arrays) to form a specific structured light pattern, such as a Gray code pattern, a stripe pattern, or a speckle pattern. In an aquatic environment, this infrared structured light with coded information is projected onto the surface of the target object. The infrared light reflected by the object surface is received by the first receiver and the second receiver, thereby generating an image for subsequent processing.

[0128] In an aquatic environment, the air's absorption and scattering of infrared light is relatively small, and infrared light can propagate well in the air, ensuring that the structured light pattern can be clearly projected onto the target object, allowing the receiver to receive a sufficiently strong and clear reflected signal, which is beneficial to improving image quality and the accuracy of subsequent depth calculations.

[0129] In aquatic environments, natural light is primarily concentrated in the visible light band, while infrared light accounts for a relatively low proportion of ambient light. Therefore, using infrared structured light can reduce the interference of ambient light on measurements, improving measurement stability and reliability. Furthermore, infrared light is invisible and less disruptive to humans.

[0130] The wavelength range of the blue-green laser transmitter module is 400-550nm, and light in this wavelength range has good penetration underwater. Water has different absorption and scattering properties for light of different wavelengths. Within the visible light range, the blue-green light band has a relatively low absorption coefficient and relatively low scattering. This means that blue-green laser light can maintain high intensity when propagating underwater and travel longer distances. This allows for the effective projection of structured light patterns in underwater environments, providing sufficient illumination conditions for the three-dimensional measurement of underwater objects.

[0131] In underwater environments, light is scattered by water molecules and suspended particles, causing the projected light spot to spread. If the emission angle is fixed, the light spot will become larger and the energy distribution will become more dispersed as the distance increases, reducing the clarity and contrast of the structured light pattern and affecting the receiver's reception and processing of the reflected signal.

[0132] By adjusting the emission angle of the blue-green laser emission module, the spread of the light spot can be compensated according to the specific conditions of the underwater environment (such as water quality and depth) and measurement requirements. For example, when the water is turbid and the scattering is strong, the emission angle can be appropriately reduced to make the laser beam more concentrated, thereby reducing the spread of the light spot and improving the quality of the structured light pattern. In cases of good water quality and weak scattering, the emission angle can be appropriately increased to expand the measurement range.

[0133] The structured light projector utilizes a switchable infrared structured light emission module and a blue-green laser emission module, enabling the camera to automatically or manually select the appropriate emission module based on the operating environment (above or below water). This design enhances the camera's versatility and adaptability, enabling efficient and accurate 3D measurement in both above-water and underwater environments, providing strong support for amphibious applications.

[0134] In summary, the infrared structured light emission module is suitable for above-water environments, and the blue-green laser emission module is suitable for underwater environments. The adjustable emission angle of the blue-green laser emission module can effectively compensate for the impact of underwater light scattering. The switchable design of the two allows the amphibious binocular depth camera to perform well in different environments.

[0135] This specification also provides an electronic device, including any of the aforementioned amphibious binocular depth cameras. It should be noted that the electronic device in this embodiment is exemplary and is provided to help those skilled in the art better understand the role of the amphibious binocular depth camera in an electronic device. It should not constitute any limitation on the electronic device.

[0136] The electronic device includes a camera module, a control and processing unit, a power supply and power supply system, a housing and a mechanical structure, and other auxiliary components.

[0137] Camera module part

[0138] The main body of the amphibious binocular depth camera is the camera described above, consisting of a waterproof housing, a structured light projector, a first receiver, a second receiver, a processor, and a sensor. The waterproof housing not only protects the internal components from water and dust but also provides mechanical support for the other components. The structured light projector switches between an infrared structured light emission module and a blue-green laser emission module depending on the environment to project structured light. The first and second receivers respectively receive reflected signals to generate an image. The processor processes various signals and data, performing operations such as image mapping, parallax calculation, and depth map generation. The sensor monitors the camera's environment in real time and transmits signals to the processor.

[0139] Optical Window Assembly: A crucial component of the waterproof housing, the optical window assembly consists of parallel layers of waterproof glass and a refraction-compensating prism. The waterproof glass prevents water from entering the camera while allowing light to pass through. The refraction-compensating prism, tilted to match the refractive index of water, compensates for underwater light refraction, ensuring high-quality imaging.

[0140] Control and processing unit

[0141] Main Processor: In addition to the camera's own processor, electronic devices may also be equipped with a more powerful main processor for overall system control and management. The main processor exchanges data with the camera's processor, receiving depth information and image information processed by the camera, and performing further analysis, decision-making, and processing based on specific application requirements. For example, in a robot navigation application, the main processor plans the robot's path based on the depth information of the environment acquired by the camera.

[0142] Storage unit: This unit stores camera-collected data, processed results, and various programs and configuration files required for system operation. This unit may include random access memory (RAM) for temporary data storage and non-volatile memory such as flash memory for long-term storage of important data and programs. For amphibious binocular depth cameras, the storage unit also stores above-water and underwater calibration files, allowing the processor to access them in different environments.

[0143] Communication module: This module enables data transmission and communication between the electronic device and external devices. This module can include wireless communication modules (such as Wi-Fi, Bluetooth, 4G / 5G, etc.) for transmitting camera-collected data in real time to a remote server or other device for analysis and processing. It can also include wired communication interfaces (such as USB and Ethernet interfaces) to facilitate data exchange and device control with local devices.

[0144] Power supply and power supply system

[0145] Power modules provide a stable power supply to all components of electronic devices. These modules may include batteries, power adapters, and other components. For electronic devices that need to be mobile above or below water, batteries are the primary power source and must have sufficient capacity and endurance. Furthermore, power modules must include overcharge, over-discharge, and short-circuit protection to ensure safe operation.

[0146] Power Management Unit (PMU): Responsible for managing and distributing the power output of the power module, optimizing device power consumption. The PMU intelligently adjusts power output based on the device's operating status and the power requirements of each component, extending device life. For example, when the camera is in sleep mode, the PMU reduces power output to conserve energy.

[0147] Housing and mechanical structure

[0148] Enclosures protect all components within electronic devices while also providing mechanical strength and protection. Enclosure designs must consider factors such as waterproofing, dustproofing, and shockproofing to adapt to diverse operating environments. For amphibious applications, enclosures may utilize specialized sealing designs and materials to effectively prevent water intrusion both above and below the surface.

[0149] Mounting and fixing structures: These are used to attach cameras and other components to electronic devices and ensure their accurate relative positioning. These mounting and fixing structures must be stable and reliable to prevent loosening or displacement of components during operation, which could affect the camera's imaging and measurement accuracy. For example, a camera might be mounted on the main frame of an electronic device using precise mechanical interfaces and fixings.

[0150] Other auxiliary components

[0151] Display module: This module displays the camera's captured images and processed results in real time, facilitating user operation and monitoring. The display module can be a liquid crystal display (LCD), organic light-emitting diode (OLED), or other display, featuring high resolution and high contrast to clearly display images and data.

[0152] Input devices, such as buttons and touch screens, are used by users to operate and configure electronic devices. Users can use input devices to select camera operating modes, adjust parameters, and start or stop data acquisition.

[0153] This electronic device, which includes an amphibious binocular depth camera, organically combines the camera with other functional modules through a reasonable structural design, realizing efficient data collection, processing and application in both above-water and underwater environments.

[0154] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the various embodiments can be referred to each other. The above description of the disclosed embodiments enables professionals and technicians in this field to implement or use the present invention. Various modifications to these embodiments will be apparent to professionals and technicians in this field, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

[0155] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art may make various variations or modifications within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. An amphibious binocular depth camera, characterized in that: include: Waterproof housing; A structured light projector, used for projecting structured light; A first receiver, configured to receive a reflection signal of the structured light and generate a first image; a second receiver, configured to receive a reflection signal of the structured light and generate a second image; A processor, configured to receive a signal above or below the water surface and read the above-water calibration file or the underwater calibration file accordingly; According to the above-water calibration file or the underwater calibration file, image mapping is performed on the first image and the second image to generate a left image and a right image that satisfy the binocular epipolar constraint; a disparity map is generated based on the left image and the right image; and a depth map is generated based on the intrinsic and extrinsic parameters of the above-water calibration file or the underwater calibration file.

2. The amphibious binocular depth camera according to claim 1, characterized in that: The mapping matrix and internal parameters of the above-water calibration file are different from those of the underwater calibration file.

3. The amphibious binocular depth camera according to claim 2, characterized in that: Compared with the above-water calibration file, the mapping matrix of the underwater calibration file also includes an underwater distortion compensation matrix.

4. The amphibious binocular depth camera according to claim 2, characterized in that: Compared with the above-water calibration file, the internal parameters of the underwater calibration file also include a refraction correction coefficient.

5. The amphibious binocular depth camera according to claim 1, characterized in that: The structured light is a combination of one or more of a Gray code pattern, a stripe pattern, and a speckle pattern.

6. The amphibious binocular depth camera according to claim 1, characterized in that: It also includes a sensor for detecting whether the camera is located above water or underwater. The sensor is electrically connected to the processor and transmits a detection signal to the processor.

7. The amphibious binocular depth camera according to claim 6, characterized in that: The sensor is a water pressure sensor, an optical sensor or a capacitive sensor.

8. The amphibious binocular depth camera according to claim 1, characterized in that: The waterproof housing comprises an optical window assembly, which includes a waterproof glass layer and a refraction compensation prism layer arranged in parallel, wherein the prism layer has an inclination angle matching the refractive index of the water body.

9. The amphibious binocular depth camera according to claim 1, characterized in that: The structured light projector includes switchable: infrared structured light emission module and blue-green laser emission module; wherein, The infrared structured light emission module is used for active light projection in an aquatic environment; The blue-green laser emission module is used for light penetration in underwater environments; The emission angle of the blue-green laser emission module is adjustable to compensate for the light spot diffusion caused by underwater light scattering.

10. An electronic device, characterized in that: An amphibious binocular depth camera comprising any one of claims 1-9.

Images