Switchable amphibious binocular depth camera and electronic equipment

Abstract

The invention discloses a switchable amphibious binocular depth camera and electronic equipment. The switchable amphibious binocular depth camera comprises a waterproof shell; the structured light projector is used for projecting structured light rays; the floodlight projector is used for projecting floodlight; the first receiver is used for receiving a reflection signal of the structured light or the floodlight to generate a first image; the second receiver is used for receiving a reflection signal of the structured light or the floodlight and generating a second image; the processor is used for receiving overwater or underwater signals, adjusting the projection proportion of the structured light rays to the floodlight and correspondingly reading an overwater calibration file or an underwater calibration file; performing image mapping on the first image and the second image according to the overwater calibration file or the underwater calibration file to generate a left image and a right image meeting binocular epipolar constraint; generating a disparity map according to the left image and the right image; and generating a depth map according to the internal reference and the external reference of the overwater calibration file or the underwater calibration file.

Description

Technical Field

[0001] The present invention relates to the technical field of depth cameras, and specifically, to a switchable amphibious binocular depth camera and an electronic device. Background Art

[0002] As an important three-dimensional vision sensor, a binocular depth camera can obtain the depth information of a scene and has wide applications in many fields such as robot navigation, intelligent security, virtual reality, and augmented reality. However, most traditional binocular depth cameras are designed for a single environment (such as an underwater environment), and there are obvious limitations in applications in an amphibious environment (both underwater and above water).

[0003] In an above-water environment, the light propagation is relatively stable. Traditional binocular depth cameras usually use structured light projection technology to assist in obtaining depth information. The structured light projector projects structured light rays with a specific pattern, and the depth is calculated by analyzing the structured light reflection signals received by the first receiver and the second receiver. This method can obtain a high depth measurement accuracy in an above-water environment.

[0004] But in an underwater environment, the situation is completely different. Water has an absorption and scattering effect on light. When light propagates in water, it will undergo severe attenuation and deformation, resulting in difficulty for the first receiver and the second receiver to accurately receive the reflection signal, thus greatly reducing the accuracy of depth measurement or even making it unable to work properly. In addition, the complexity of the underwater environment, such as the changes in water flow and water quality, will also have an adverse impact on the performance of traditional binocular depth cameras.

[0005] Currently, there is a lack of a binocular depth camera on the market that can flexibly switch between underwater and above-water environments and ensure a high depth measurement accuracy in both environments. This makes it difficult to select equipment for some application scenarios that need to work in an amphibious environment, such as underwater archaeology, marine ecological monitoring, and underwater and above-water rescue.

[0006] The disclosure of the above background art content is only used to assist in understanding the inventive concept and technical solution of the present invention, and it does not necessarily belong to the prior art of this patent application. Without clear evidence indicating that the above content was publicly available on the filing date of this patent application, the above background art should not be used to evaluate the novelty and inventiveness of this application. Summary of the Invention

[0007] Therefore, through a unique structural design and image processing method, the present invention ensures that high-quality depth images can be generated in different environments, and can effectively solve the imaging problem of traditional cameras when switching between land and water environments.

[0008] In a first aspect, the present invention provides a switchable amphibious binocular depth camera, which is characterized by comprising:

[0009] Waterproof housing;

[0010] Structured light projector for projecting structured light rays;

[0011] Omnidirectional light projector for projecting omnidirectional light;

[0012] First receiver for receiving the reflected signals of the structured light rays or the omnidirectional light and generating a first image;

[0013] Second receiver for receiving the reflected signals of the structured light rays or the omnidirectional light and generating a second image;

[0014] Processor for receiving signals above or below water, adjusting the projection ratio of the structured light rays and the omnidirectional light, and accordingly reading an above-water calibration file or a below-water calibration file; performing image mapping on the first image and the second image according to the above-water calibration file or the below-water calibration file to generate a left image and a right image that satisfy the binocular epipolar constraint; generating a disparity map based on the left image and the right image; generating a depth map based on the internal and external parameters of the above-water calibration file or the below-water calibration file.

[0015] Optionally, in the described switchable amphibious binocular depth camera, the processor processes the reflected signal of the structured light rays to obtain a first depth map, processes the reflected signal of the omnidirectional light to obtain a second depth map, and fuses the first depth map and the second depth map according to the projection ratio to obtain a fused depth map.

[0016] Optionally, in the described switchable amphibious binocular depth camera, when the processor adjusts the projection ratio of the structured light rays and the omnidirectional light, it also adjusts the projection intensity of the structured light rays and the omnidirectional light.

[0017] Optionally, in the described switchable amphibious binocular depth camera, when adjusting the projection intensity of the structured light rays and the omnidirectional light, if the current is an above-water environment, the projection intensity of the structured light rays is reduced and the projection intensity of the omnidirectional light is increased; if the current is a below-water environment, the projection intensity of the structured light rays is increased and the projection intensity of the omnidirectional light is reduced.

[0018] Optionally, in the described switchable amphibious binocular depth camera, the light-transmitting window of the waterproof housing adopts an anti-water stain coating or a switchable optical medium layer to reduce the influence of underwater scattering on imaging.

[0019] Optionally, the switchable amphibious binocular depth camera is characterized in that it further includes a signal input module, which is electrically connected to the processor and is used to input a signal indicating a water or underwater environment to the processor. The signal input module includes at least one of a pressure sensor and a humidity sensor.

[0020] Optionally, the switchable amphibious binocular depth camera is characterized in that the mapping matrix and internal parameters of the above-water calibration file and the underwater calibration file are different.

[0021] Optionally, the switchable amphibious binocular depth camera is characterized in that, compared with the above-water calibration file, the mapping matrix of the underwater calibration file further includes an underwater distortion compensation matrix.

[0022] Optionally, the switchable amphibious binocular depth camera is characterized in that, compared with the above-water calibration file, the internal parameters of the underwater calibration file further include a refraction correction coefficient.

[0023] In a second aspect, the present invention provides an electronic device, which is characterized in that it includes the switchable amphibious binocular depth camera described in any one of the above.

[0024] Compared with the prior art, the present invention has the following beneficial effects:

[0025] The present invention is equipped with a waterproof housing, which can effectively resist water erosion in water and underwater environments and ensure the normal operation of internal precision components. At the same time, the processor can receive water or underwater signals and intelligently adjust the projection ratio of structured light rays and floodlight according to the environmental characteristics, flexibly switching the working mode, enabling the camera to stably obtain effective data in both water and underwater environments, breaking through the limitation that traditional cameras are only applicable to a single environment, and greatly expanding the application scenarios of the camera. It can play a role in fields such as ship navigation on water, underwater ocean exploration, and water conservancy facility detection.

[0026] In the water environment of the present invention, the structured light projector projects structured light rays. Combining with the above-water calibration file read by the processor, the images generated by the first and second receivers are accurately processed to meet the binocular epipolar constraint, and then a high-quality disparity map and depth map are generated to ensure the accurate acquisition of depth information in the water scene. In the underwater environment, by adjusting the floodlight projection intensity and cooperating with the underwater calibration file, the camera can also accurately generate a depth map under complex underwater light conditions, overcoming the interference caused by the absorption and scattering of water to light, and ensuring high-precision depth imaging in different environments.

[0027] The processor inside the camera of the present invention can automatically identify water or underwater signals and respond quickly. It not only adjusts the light projection ratio but also accurately reads the corresponding calibration file, automatically completing a series of complex operations such as subsequent image mapping, disparity map generation, and depth map generation. This intelligent process requires little manual intervention, greatly improving the working efficiency of the camera in different environments, especially suitable for some dynamic scenarios that require real-time acquisition of depth information, such as underwater robot operations and water sports monitoring.

[0028] The present invention has excellent amphibious adaptability and high-precision imaging characteristics, greatly expanding its application fields. In the scientific research field, it can be used for marine biology research, underwater geological exploration, etc.; in the industrial field, it can assist in underwater engineering construction, water bridge detection, etc.; in the security field, it can achieve water and underwater monitoring and early warning. The rich application scenarios provide new technical support means for the development of various industries.

[0029] The switchable amphibious binocular depth camera of the present invention can meet the requirements of two environments with one set of equipment, effectively reducing the equipment procurement cost. Moreover, its intelligent and efficient working mode reduces the labor input and time cost, and has significant cost-effective advantages from the perspective of long-term use.

[0030] The hardware of the present invention is compatible with water and underwater modes, and the algorithms of the two processing modes of water and underwater are also completely compatible. The difference is only the mapping matrix and internal parameters used for the epipolar constraint, and the process is exactly the same, greatly improving the integration, modularity, and simplicity of the algorithm. It can be used in an soc (system on chip) with DPU (data processor) function and dedicated depth chips, with very good adaptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained according to the provided drawings. By reading the detailed description of the non-limiting embodiments with reference to the following drawings, other features, purposes, and advantages of the present invention will become more obvious:

[0032] Figure 1 It is a schematic structural diagram of a switchable amphibious binocular depth camera in an embodiment of the present invention;

[0033] Figure 2 It is a schematic structural diagram of another switchable amphibious binocular depth camera in an embodiment of the present invention.

[0034] 1 - Waterproof housing;

[0035] 2 - Structured light projector;

[0036] 3 - Floodlight projector;

[0037] 4 - First receiver;

[0038] 5 - Second receiver;

[0039] 6 - Processor;

[0040] 7 - Signal input module; Detailed implementation manners

[0041] The present invention will be described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but do not limit the present invention in any form. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can still be made. These all belong to the protection scope of the present invention.

[0042] The terms "first", "second", "third", "fourth", etc. (if any) in the specification, claims and above - mentioned drawings of the present invention are used to distinguish similar objects, and do not have to be used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances, so that the embodiments of the present invention described here, for example, can be implemented in an order other than those illustrated or described here. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non - exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units does not have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0043] A switchable amphibious binocular depth camera provided by an embodiment of the present invention aims to solve the problems existing in the prior art.

[0044] The technical solutions of the present invention and how the technical solutions of the present application solve the above - mentioned technical problems will be described in detail below with specific embodiments. These several specific embodiments below can be combined with each other, and for the same or similar concepts or processes, they may not be repeated in some embodiments. The embodiments of the present invention will be described below in conjunction with the drawings.

[0045] Since the influence of water and air on the optical path in water - based and underwater scenes is different, the depth cameras in the prior art cannot simultaneously take into account the depth acquisition problems in water - based and underwater scenarios.

[0046] The present invention utilizes structured light and floodlight as active light sources, and utilizes a binocular system consisting of a first receiver and a second receiver to obtain depth data. The present invention automatically adjusts the projection ratio of structured light and floodlight according to underwater or underwater signals, and performs corresponding processing to obtain an accurate above-water or underwater depth map.

[0047] Figure 1 FIG. 1 is a schematic diagram of the structure of a switchable amphibious binocular depth camera according to an embodiment of the present invention. Figure 1 As shown, a switchable amphibious binocular depth camera in an embodiment of the present invention includes:

[0048] Waterproof housing1.

[0049] Specifically, the waterproof housing is the external protective structure of the camera, and its main function is to protect the electronic components and optical parts inside the camera from water erosion, ensuring that the camera can work normally both underwater and above water. It needs to have good sealing and pressure resistance to adapt to the pressure environment of different depths underwater, while ensuring that the camera performance will not be affected by moisture and other reasons when used on water.

[0050] The structured light projector 2 is used to project structured light.

[0051] Specifically, the structured light projector is responsible for projecting structured light. Structured light is a type of light with a specific pattern. By projecting this light onto the surface of an object, the three-dimensional information of the object can be obtained based on the reflection and deformation of the light on the surface of the object. In an amphibious binocular depth camera, the structured light projector can work both above and below water, providing the camera with structured light information for depth perception.

[0052] The floodlight projector 3 is used for projecting floodlight.

[0053] Specifically, the flood light projector is used to project flood light, which is a uniform light. Flood light and structured light can be emitted at the same time or in time-sharing. When the flood light and structured light are emitted at the same time, the main function of the flood light is to provide additional lighting for the camera under low-light conditions or when the structured light cannot work effectively, so that the camera can capture images normally. In an underwater environment, due to the rapid attenuation of light, the flood light projector can help improve the brightness and contrast of the image, allowing the camera to better receive the reflected signal. When the flood light and structured light are emitted in time-sharing, the main function of the flood light is to enable the first receiver and the second receiver to benefit from the flood light image and obtain a depth map using the binocular principle.

[0054] The first receiver 4 is used to receive the reflected signal of the structured light or the floodlight to generate a first image.

[0055] Specifically, the function of the first receiver is to receive the reflected signals of structured light rays or floodlights and convert these signals into a first image. The first receiver is usually an optoelectronic sensor that can sense information such as the intensity and color of light, convert it into an electrical signal, and then convert the electrical signal into a digital image signal through the image processing circuit of the camera. In different environments, such as on water or underwater, the first receiver must be able to accurately receive the reflected signals to ensure that the generated first image has sufficient quality and information.

[0056] The second receiver 5 is used to receive the reflected signals of the structured light rays or the floodlights and generate a second image.

[0057] Specifically, similar to the first receiver, the second receiver is also used to receive the reflected signals of structured light rays or floodlights and generate a second image. It works in cooperation with the first receiver. By using the binocular vision principle, the depth information of an object is calculated based on the two images generated by receiving the reflected signals from different angles of the same object. The positions and angles of the two receivers are precisely designed to meet the binocular epipolar constraint, thereby providing accurate data for subsequent image processing and depth calculation.

[0058] The processor 6 is used to receive signals on water or underwater, adjust the projection ratio of the structured light rays and the floodlights, and accordingly read the calibration file on water or the calibration file underwater; according to the calibration file on water or the calibration file underwater, perform image mapping on the first image and the second image to generate a left image and a right image that meet the binocular epipolar constraint; generate a disparity map based on the left image and the right image; generate a depth map based on the internal and external parameters of the calibration file on water or the calibration file underwater.

[0059] Specifically, the processor is the core component of the camera and has multiple important functions. First, it can receive signals on water or underwater and adjust the projection ratio of structured light rays and floodlights according to different environments. For example, in a bright environment on water, the projection of floodlights may be reduced, and depth measurement mainly relies on structured light; while in a darker environment underwater, the projection of floodlights is increased to improve image quality. Second, the processor will accordingly read the calibration file on water or the calibration file underwater. These calibration files contain parameter information of the camera in different environments, such as lens distortion parameters, internal and external parameters of the camera, etc. According to these parameters, the processor performs image mapping on the first image and the second image, converting them into a left image and a right image that meet the binocular epipolar constraint for subsequent depth calculation. Finally, the processor generates a disparity map based on the left image and the right image, and uses the internal and external parameters in the calibration file to convert the disparity map into a depth map, thereby realizing the acquisition of the depth information of the object by the camera.

[0060] In some embodiments, the processor processes the reflection signal of the structured light rays to obtain a first depth map, processes the reflection signal of the floodlight to obtain a second depth map, and fuses the first depth map and the second depth map according to the projection ratio to obtain a fused depth map.

[0061] The processor processes the reflection signal of the structured light rays to obtain a first depth map. The structured light projector emits specific pattern light rays onto the object surface. Due to factors such as the shape and distance of the object, the reflected structured light pattern will be deformed. The processor uses algorithms to analyze this deformation and, through methods such as the triangulation principle, accurately calculates the distance information between each point on the object surface and the camera, thereby constructing the first depth map. This depth map is mainly based on the depth details of the object obtained by the structured light technology and can provide accurate depth data in scenarios such as on water or in good light conditions where there is a high demand for measuring the depth of fine structures of the object.

[0062] The structured light rays and the floodlight rays are projected alternately. When processing the reflection signal of the floodlight, since the floodlight provides uniform illumination, using the first image and the second image generated by the first receiver and the second receiver, the processor adopts a binocular algorithm to estimate the depth information of the object and generates a second depth map.

[0063] The processor fuses the first depth map and the second depth map according to the projection ratio of the structured light rays and the floodlight. The projection ratio is adjusted in real time according to the water or underwater environment where the camera is located. The fusion process uses fusion algorithms such as weighted average to combine the advantages of the two depth maps and obtain a fused depth map. The fused depth map combines the high-precision details of the structured light depth map and the applicability of the floodlight depth map in complex environments, enabling the camera to provide more comprehensive and accurate depth perception results in amphibious environments.

[0064] In some embodiments, when the processor adjusts the projection ratio of the structured light rays and the floodlight, it also adjusts the projection intensity of the structured light rays and the floodlight.

[0065] The processor's adjustment of the structured light rays is highly adaptable to the environment. In a strong light environment on water, the surrounding light is already sufficient. If the projection intensity of the structured light rays is too high, the reflection signal is prone to overexposure, causing the image information obtained by the camera to be lost or blurred, seriously affecting the accuracy of depth calculation. At this time, the processor will automatically reduce the projection intensity of the structured light to keep the reflection signal within the range that the camera can accurately identify, ensuring the accuracy of the first depth map constructed based on the structured light reflection signal. On the contrary, in a darker water scene, since the ambient light cannot provide good conditions for the structured light reflection, the processor will increase the projection intensity of the structured light to allow more structured light rays to reach the object surface and be reflected back, thereby ensuring that the reflection signal can be clearly captured and providing a stable signal source for constructing a high-precision first depth map to accurately calculate the distance information between each point on the object surface and the camera.

[0066] The floodlight adjustment varies according to different floodlight and structured light projection modes.

[0067] When the floodlight is used as supplementary lighting for the structured light, the floodlight adjustment mainly targets environments with low light or severe interference to the structured light, especially underwater scenes. In extremely dark underwater environments, the light attenuation is extremely severe, and ordinary ambient light can hardly enable objects to be effectively imaged by the camera. At this time, the processor will increase the projection intensity of the floodlight to enhance the illumination level on the object surface. In this way, the camera can obtain clearer reflection signals, facilitating the subsequent processor to analyze information such as the change in floodlight reflected light intensity and image features, and then accurately estimate the depth information of the object to generate a second depth map for supplementation. Through this method, in complex underwater environments where the structured light cannot work effectively, the floodlight illumination and intensity adjustment provide the possibility for the camera to obtain the approximate depth of the object.

[0068] When the floodlight and the structured light are projected alternately, the intensities of the floodlight and the structured light are similar. When above water, the intensity of the floodlight becomes smaller. When underwater, the intensity of the floodlight becomes larger.

[0069] The processor does not adjust the projection intensities of the structured light and the floodlight in isolation, but synchronously adjusts the intensities while adjusting the projection ratio of the two. In an above-water scene, if the projection ratio of the structured light is high, it means that the first depth map contributes more to the final fused depth map because it can provide rich object depth details. At this time, an appropriate structured light intensity ensures that these detail information can be accurately obtained. In an underwater scene, if the projection ratio of the floodlight is high, the weight of the second depth map increases to make up for the deficiency of the structured light, and an appropriate floodlight intensity ensures the underwater imaging quality and provides reliable data for depth estimation. Finally, the processor uses fusion algorithms such as weighted average to integrate the advantages of the two depth maps to obtain a fused depth map, enabling the camera to provide comprehensive and accurate depth perception results in both amphibious environments.

[0070] In some embodiments, when adjusting the projection intensities of the structured light and the floodlight, if the current environment is above water, the projection intensity of the structured light is decreased and the projection intensity of the floodlight is increased; if the current environment is underwater, the projection intensity of the structured light is increased and the projection intensity of the floodlight is decreased.

[0071] Water environment intensity adjustment: In a water environment, the ambient light is usually sufficient, and the structured light itself already has enough lighting basis to achieve depth measurement. At this time, if the projection intensity of the structured light is too high, the light reflected from the object surface is likely to be overexposed, resulting in areas with too high brightness in the image obtained by the camera, losing key details, and seriously affecting the accuracy of generating the first depth map based on the structured light reflection signal. Therefore, the processor will actively reduce the projection intensity of the structured light, control the reflection signal within the dynamic range that the camera sensor can accurately identify, ensure that the obtained image is clear and contains rich details, and provide a reliable basis for accurately calculating the object depth. On the other hand, appropriately increasing the projection intensity of the floodlight can enable the camera to obtain a more comprehensive, uniform, and clear reflection signal, further improving the imaging quality and the accuracy of depth calculation.

[0072] Underwater environment intensity adjustment: The underwater environment is very different from the water environment. When light propagates in water, it will undergo severe attenuation, resulting in extremely weak ambient light. In this case, if only relying on floodlight illumination, due to its uniform but featureless light, it is difficult to accurately obtain the object depth information. And the structured light has a specific pattern, and the object depth can be accurately calculated by analyzing the deformation of the reflection pattern. Therefore, the processor will increase the projection intensity of the structured light, allowing more structured light carrying depth information to reach the object surface and be reflected back to overcome the problem of underwater light attenuation and ensure that the obtained reflection signal intensity is sufficient for constructing a high-precision first depth map. At the same time, the projection intensity of the floodlight is reduced because in the underwater environment, the role of the floodlight is relatively limited. Too high an intensity of the floodlight may not only fail to effectively improve depth measurement, but may also cause light scattering, interfering with the reception and analysis of the structured light reflection signal and reducing the accuracy of depth calculation. Through this adjustment strategy of increasing and decreasing, the camera can also achieve accurate depth perception in the complex underwater environment.

[0073] In some embodiments, the light-transmitting window of the waterproof housing uses an anti-water-stain coating or a switchable optical medium layer to reduce the impact of underwater scattering on imaging.

[0074] Anti-water-stain coating: In an underwater environment, water stains are extremely likely to adhere to the light-transmitting window of the camera. These water stains will disrupt the light propagation path, cause light scattering, resulting in problems such as blurred imaging, reduced contrast, and loss of details. The anti-water-stain coating is attached to the surface of the light-transmitting window through special materials and processes. It has superhydrophobic properties, which can greatly reduce the adhesion force between water stains and the window surface. When water contacts the window treated with the anti-water-stain coating, it will form water droplets and roll off, rather than adhering to form a large-area water film. This effectively reduces the light scattering phenomenon caused by the presence of water stains, allowing more light to pass through the window along the normal path and reach the receiver inside the camera. In this way, the reflection signals obtained by the camera during underwater shooting are clearer and more stable, thereby improving the imaging quality and providing a good front-end guarantee for the processor to generate accurate depth maps based on these signals subsequently.

[0075] Switchable optical medium layer: The scattering of underwater light is not only affected by water stains but also closely related to the optical properties of water itself. The refractive index of water is different from that of air, and when light enters the light-transmitting window of the camera from water, refraction and scattering changes will occur. The switchable optical medium layer is a special design that can be adjusted according to environmental changes. When the camera is in an underwater environment, this medium layer can automatically switch its own optical parameters, such as refractive index, to better match the optical properties of water. Through this matching, the light propagation path is more regular during the process of light entering the window from water and then entering the camera interior, reducing the scattering caused by the refractive index difference. This means that the camera can collect the reflected light from objects more effectively, and the clarity and contrast of the imaging are improved, thus helping to improve the quality of underwater imaging and ensuring that the camera can also obtain high-quality images for depth calculation underwater.

[0076] In some embodiments, as Figure 2 shown, it further includes a signal input module, the signal input module is electrically connected to the processor, and is used to input a signal indicating the on-water or underwater environment to the processor. The signal input module includes at least one of a pressure sensor and a humidity sensor.

[0077] Functional positioning of the signal input module: The signal input module, as the "environmental perception antenna" of the camera system, is electrically connected to the processor. Its core mission is to accurately input a signal indicating the environment (on-water or underwater) where the camera is located to the processor. This signal plays a decisive guiding role in a series of subsequent adjustment operations of the processor, such as adjusting the projection ratio and intensity of structured light and floodlight, and reading the calibration file of the corresponding environment. It is an important start for the camera to achieve the adaptive function in the amphibious environment.

[0078] Working mechanism of the pressure sensor: The pressure sensor is an important part of the signal input module. In an underwater environment, as the depth increases, the water pressure will increase significantly, and there is a clear corresponding relationship between the water pressure and the depth. The pressure sensor can sense the change of the surrounding environmental pressure in real time. When the camera enters the water from above the water surface, the pressure sensor detects that the pressure value rises rapidly, converts this pressure change signal into an electrical signal, and transmits it to the processor. After receiving this signal, the processor analyzes it through a preset algorithm inside, determines that the camera is currently in an underwater environment, and then triggers a series of underwater environment adaptation operations, such as enhancing the projection intensity of structured light and reducing the projection intensity of floodlight, to ensure that the camera can work normally underwater and obtain high-quality image data.

[0079] Working principle of the humidity sensor: The humidity sensor also plays a key role in the signal input module. The air humidity in the above-water environment is usually in a relatively stable range, while when the camera enters the underwater environment, the humidity will rise sharply and approach 100%. The humidity sensor can sensitively capture the change of environmental humidity through its own special sensing material. Once the humidity sensor detects a significant jump in humidity, it will immediately convert the humidity change signal into an electrical signal and send it to the processor. Based on the received humidity signal, the processor determines that the camera is in an underwater environment and then makes corresponding adjustment strategies, similar to the operations of the processor after receiving the signal from the pressure sensor, to make the camera adapt to the underwater environment.

[0080] Advantages of combined use: When the signal input module is equipped with both a pressure sensor and a humidity sensor, it has stronger reliability in environmental judgment. In some special scenarios, a single sensor may be interfered with and produce misjudgments. For example, the pressure sensor may be interfered by non-underwater factors with sudden pressure changes during rapid ascent and descent, or the humidity sensor may produce misjudgments in a high-humidity above-water fog environment. However, with the configuration of at least one of the two, in most cases, it can accurately sense environmental changes, and when both are configured, the processor can comprehensively analyze the signals of the two sensors to further improve the accuracy of environmental judgment and ensure the stable and accurate operation of the camera in a complex amphibious environment.

[0081] In some embodiments, the mapping matrix and internal parameters of the above-water calibration file and the underwater calibration file are different.

[0082] Mapping Matrix Difference: The mapping matrix determines the position mapping relationship of pixel points in the image in three-dimensional space. In the water environment, the light propagation is relatively stable, and the refraction and scattering conditions are significantly different from those underwater. The mapping matrix in the water calibration file is based on the light propagation characteristics in water and is obtained through a large number of accurate measurements and calculations. It can accurately map the pixel points on the two-dimensional image captured by the camera to the corresponding positions in the actual three-dimensional scene, thus laying a foundation for subsequent generation of accurate disparity maps and depth maps. In the underwater environment, when light propagates in water, due to the different refractive indices of water and air, strong refraction occurs, and impurities in the water also cause light scattering. This makes the mapping matrix in the underwater calibration file need to fully consider these complex optical property changes. Compared with that in water, the underwater mapping matrix will be specially adjusted in parameter settings to compensate for the deviation of light during underwater propagation, ensuring that even under complex underwater light conditions, the pixel points of the image captured by the camera can be correctly mapped to the three-dimensional space position and guaranteeing the accuracy of depth calculation.

[0083] Internal Parameter Difference: The camera internal parameters mainly describe the optical and geometric characteristics inside the camera, including parameters such as focal length, principal point position, and pixel scale factor. In the water environment, the camera internal parameters are calibrated based on the normal light propagation and imaging conditions in water. For example, the focal length value is determined according to the light convergence situation of the lens during light propagation in water, and the principal point position is also obtained based on the center characteristics of the imaging plane in water. These internal parameter values enable the camera to accurately convert light into a clear and undistorted image when shooting in water. However, in the underwater environment, due to the influence of the optical properties of water, the actual optical performance of the camera changes. The high refractive index of water changes the angle at which light enters the camera lens, thereby affecting parameters such as the focal length. Therefore, the internal parameter values in the underwater calibration file will be recalibrated. The new focal length value, principal point position, and pixel scale factor, etc., are all optimized for the special situation of light propagation and imaging in the underwater environment, enabling the camera to image normally in the underwater environment and providing basic parameters that conform to the actual underwater situation for subsequent image processing and depth calculation.

[0084] In some embodiments, compared with the water calibration file, the mapping matrix of the underwater calibration file further includes an underwater distortion compensation matrix.

[0085] The optical conditions in the underwater environment are extremely complex compared with those in water. When light propagates in water, in addition to refraction caused by the different refractive indices of water and air, there are various impurities in the water, such as suspended particles, microorganisms, etc., and these impurities will scatter light. In addition, components such as the waterproof housing and light-transmitting window of the camera will also change the light propagation path to a certain extent. The superposition of multiple factors causes the images captured by the camera to be prone to serious distortion. If this distortion is not corrected, it will greatly affect the accuracy of the position mapping of pixel points in the three-dimensional space, and thus lead to a large deviation in depth calculation.

[0086] An underwater distortion compensation matrix came into being, which is specifically used to offset the image distortion caused by these complex factors. This matrix is generated through a large number of precise measurements and complex calculations in the underwater environment. In practical applications, when the camera acquires an underwater image, the processor will call the underwater distortion compensation matrix in the underwater calibration file. Each parameter in the matrix will adjust the position of the image pixel points one by one, correcting the distortion caused by the special underwater optical environment. For example, for pixel points that deviate from their normal positions due to light refraction and scattering, the underwater distortion compensation matrix will remap these pixel points to coordinates that conform to the actual physical positions according to its specific algorithm. After distortion compensation, the distribution of the image pixel points is closer to the object position relationship in the real scene, enabling the mapping matrix in the underwater calibration file to map the image pixel points to the three-dimensional space position more accurately, strongly guaranteeing the accuracy of the depth calculation of the camera in the complex underwater environment, and providing a reliable data basis for applications such as underwater target detection and topographic mapping.

[0087] In some embodiments, compared with the above-water calibration file, the internal parameters of the underwater calibration file further include a refraction correction coefficient.

[0088] The refractive index of water is significantly higher than that of air. This characteristic causes the propagation direction of light to change significantly when it propagates underwater and enters the camera lens, with a huge difference from the propagation path of light in the above-water environment. In the above-water environment, camera internal parameters such as focal length and principal point position are determined based on the relatively stable propagation characteristics of light in air. However, underwater, the refraction effect of light completely changes the actual optical imaging model of the camera.

[0089] The refraction correction coefficient is introduced into the internal parameters of the underwater calibration file to address this thorny problem. Its working principle is based on the precise quantitative analysis of the change in the refraction angle of light. Through complex mathematical models and a large amount of underwater experimental data, the specific value of this coefficient is determined. During the actual operation of the camera, when calculating imaging parameters based on the internal parameters of the underwater calibration file, the refraction correction coefficient plays an indispensable role. For example, when calculating the focal length equivalent value, due to the refraction of light underwater, the actual focal length shown by the camera deviates from the calibrated focal length in the above-water environment. At this time, using the refraction correction coefficient can compensate for this deviation caused by light refraction. Specifically, the refraction correction coefficient will participate in the calculation formula of the focal length equivalent value to adjust the initial focal length value based on the above-water calibration, so that the finally calculated focal length equivalent value is more in line with the actual light propagation and imaging conditions underwater.

[0090] With the addition of the refraction correction coefficient, the images captured by the camera underwater can be processed more accurately. It further improves the depth calculation accuracy and imaging quality of the camera in the underwater environment, ensuring that the camera can still provide high-quality data support for various applications such as underwater target detection and underwater terrain mapping in complex underwater optical environments, greatly expanding the practicality and reliability of the camera in underwater scenarios.

[0091] This specification also provides an electronic device, including the switchable amphibious binocular depth camera described in any of the foregoing items. It should be noted that the electronic device in this embodiment is exemplary and is described to help those skilled in the art better understand the role of the switchable amphibious binocular depth camera in the electronic device, and should not constitute any limitation to the electronic device.

[0092] The electronic device includes a core processing unit, a storage module, a power module, a switchable amphibious binocular depth camera module, and other auxiliary modules.

[0093] Core processing unit: Usually composed of a high-performance central processing unit (CPU), a graphics processing unit (GPU), or a dedicated image signal processor (ISP). The CPU is responsible for the system control and data operation scheduling of the entire device. The GPU focuses on graphics processing tasks and can accelerate the image rendering and depth calculation processes when processing a large amount of image data obtained by the camera. The ISP can preprocess the raw image signals transmitted by the camera, such as noise reduction and color correction, to improve the image quality.

[0094] Storage module: Includes random access memory (RAM) and non-volatile storage, such as a solid-state drive (SSD) or flash memory. RAM is used to temporarily store the image data captured by the camera in real time, the intermediate data during the operation of the processor, and the program code being executed to ensure the efficient operation of the device. Non-volatile storage is used to permanently store the device's operating system, application programs, as well as the depth data and image materials collected and processed by the camera for subsequent calling and analysis.

[0095] Power module: Supplies power to each component of the device and may include a rechargeable battery, a power management chip, etc. The battery provides the electrical energy required for the device to work mobilely. The power management chip is responsible for monitoring the battery status, adjusting the voltage and current output to ensure stable power supply to each component, while optimizing the battery usage efficiency and extending the device's battery life.

[0096] Switchable Amphibious Binocular Depth Camera Module: Installed at a specific location on the device, it has a waterproof housing to protect the internal structured light projector, floodlight projector, first receiver, second receiver, and related circuits. The light-transmitting window of the waterproof housing uses an anti-water stain coating or a switchable optical medium layer to ensure underwater imaging quality. The camera is connected to the core processing unit through a data transmission interface (such as high-speed interfaces like USB3.0, MIPI, etc.) to quickly transmit the collected image data.

[0097] Other Auxiliary Modules: There may be communication modules (such as Wi-Fi, Bluetooth, 4G / 5G modules) for the device to communicate with external devices or networks, enabling data sharing and remote control; display modules (such as liquid crystal displays or organic light-emitting diode displays) for real-time display of information such as images and depth maps captured by the camera; input modules (such as buttons, touchscreens) to facilitate user operation of the device, controlling functions such as camera parameter settings and starting depth measurement.

[0098] Functions of the Electronic Device

[0099] Image Acquisition and Depth Perception: The structured light projector and floodlight projector of the camera emit light, which is reflected by the object and then received by the first receiver and the second receiver to generate an image. The processor adjusts the light projection ratio and intensity according to the signal of the above-water or underwater environment, reads the corresponding calibration file, and processes the image to generate a disparity map and a depth map, achieving accurate three-dimensional scene perception, which can be used for object recognition, distance measurement, etc.

[0100] Environmental Adaptability Function: Through signal input modules such as pressure sensors and humidity sensors, the device can automatically identify the above-water or underwater environment, and the camera and the processor cooperate to adjust the working parameters to ensure high-quality images and depth data can be obtained in different environments, broadening the application scenarios of the device.

[0101] Data Processing and Analysis Function: The core processing unit processes the data collected by the camera, and can use built-in algorithms for target detection and tracking, and combine depth information to achieve more accurate target positioning and behavior analysis. For example, in underwater detection applications, it can identify and track underwater target objects, and measure their size, position changes, etc.

[0102] Data Storage and Transmission Function: The storage module saves the collected image and depth data, and the communication module can transmit the data to other devices or the cloud for remote monitoring, data sharing, and further analysis and processing.

[0103] Role of the Depth Camera in the Electronic Device

[0104] Core perception component: It is the key for an electronic device to obtain three-dimensional environmental information. In scenarios such as underwater archaeology and marine resource exploration, the depth information provided by a camera helps identify the shape, position, and distance of underwater objects, providing an important basis for research and operations; in applications such as water-based security monitoring and intelligent transportation assistance, a depth camera assists in identifying target objects, improving the accuracy of monitoring and early warning.

[0105] Basis for environmental adaptability: The environmental signal perception and adaptive adjustment capabilities of a camera are the basis for an electronic device to operate in an amphibious environment. By automatically adjusting the light projection and imaging parameters, it ensures that the device can stably obtain available data in different environments, guaranteeing the normal operation of other functions.

[0106] Improving data processing accuracy: The depth data provided by a depth camera provides richer information for the data processing algorithms of an electronic device, making algorithms such as target detection, recognition, and tracking more accurate and efficient. For example, in the application of intelligent robot navigation, the depth information obtained by a depth camera helps the robot accurately perceive the surrounding environment, plan the travel route, and avoid collisions.

[0107] In this specification, each embodiment is described in a progressive manner. The key point of each embodiment is to illustrate the differences from other embodiments. For the same or similar parts among the embodiments, reference can be made to each other. The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be obvious to those skilled in the art. The general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown herein, but will conform to the broadest scope consistent with the principles and novel features disclosed herein.

[0108] The specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the above specific implementation manners. Those skilled in the art can make various deformations or modifications within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. A switchable amphibious binocular depth camera, characterized in that, Comprising: A waterproof housing; A structured light projector for projecting structured light rays; A floodlight projector for projecting floodlight; A first receiver for receiving the reflected signal of the structured light rays or the floodlight to generate a first image; A second receiver for receiving the reflected signal of the structured light rays or the floodlight to generate a second image; A processor for receiving signals above or below water, adjusting the projection ratio of the structured light rays and the floodlight, and correspondingly reading an above-water calibration file or a below-water calibration file; According to the above-water calibration file or the below-water calibration file, perform image mapping on the first image and the second image to generate a left image and a right image that satisfy the binocular epipolar constraint; generate a disparity map according to the left image and the right image; generate a depth map according to the internal parameters and external parameters of the above-water calibration file or the below-water calibration file.

2. The switchable amphibious binocular depth camera according to claim 1, wherein, The processor processes the reflected signal of the structured light rays to obtain a first depth map, processes the reflected signal of the floodlight to obtain a second depth map, and fuses the first depth map and the second depth map according to the projection ratio to obtain a fused depth map.

3. The switchable amphibious binocular depth camera according to claim 1, characterized in that, When the processor adjusts the projection ratio of the structured light rays and the floodlight, it also adjusts the projection intensity of the structured light rays and the floodlight.

4. The switchable amphibious binocular depth camera according to claim 3, wherein When adjusting the projection intensity of the structured light rays and the floodlight, if the current is an above-water environment, reduce the projection intensity of the structured light rays and increase the projection intensity of the floodlight; if the current is a below-water environment, increase the projection intensity of the structured light rays and reduce the projection intensity of the floodlight.

5. The switchable amphibious binocular depth camera according to claim 1, characterized in that, The light-transmitting window of the waterproof housing adopts an anti-water-stain coating or a switchable optical medium layer to reduce the impact of underwater scattering on imaging.

6. The switchable amphibious binocular depth camera according to claim 1, characterized in that, It further includes a signal input module, which is electrically connected to the processor and is used to input a signal indicating an above-water or below-water environment to the processor. The signal input module includes at least one of a pressure sensor and a humidity sensor.

7. The switchable amphibious binocular depth camera according to claim 1, characterized in that, The mapping matrix and internal parameters of the above-water calibration file and the below-water calibration file are different.

8. The switchable amphibious binocular depth camera according to claim 7, characterized in that, Compared with the above-water calibration file, the mapping matrix of the below-water calibration file further includes an underwater distortion compensation matrix.

9. The switchable amphibious binocular depth camera according to claim 7, wherein, Compared with the above-water calibration file, the internal parameters of the below-water calibration file further include a refraction correction coefficient.

10. An electronic device, characterized in that, Including a switchable amphibious binocular depth camera according to any one of claims 1-9.

Images