Depth value determination device, method, depth sensing module and electronic device

Abstract

The present disclosure provides a depth value determination device, a depth value determination method, a depth sensing module and an electronic device, and relates to the field of computer technology. The depth value determination device includes: a first histogram determination module, which is used to receive data of photons sensed by a standard pixel area corresponding to a laser emission point, and to perform statistics on the data of photons sensed by the standard pixel area to determine a first histogram; a second histogram determination module, which is used to receive data of photons sensed by pixel areas surrounding the standard pixel area, and to perform statistics on the data of photons sensed by the surrounding pixel areas to determine a second histogram; a depth value calculation module, which is used to calculate the depth value sensed based on the laser emission point according to the first histogram and the second histogram. The present disclosure can improve the sensing accuracy of the depth value.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a depth value determination device, a depth value determination method, a depth sensing module, and an electronic device. Background Art

[0002] Depth sensing technology is a technology that senses the depth of field of objects in space. It is widely used in assisted photography, spatial modeling, AR (Augmented Reality), assisted driving and other fields.

[0003] Currently, the depth value sensed by a depth sensing device may be inaccurate, and the inaccuracy of the depth value will affect the accuracy of subsequent algorithms. Summary of the invention

[0004] The present disclosure provides a depth value determination device, a depth value determination method, a depth sensing module and an electronic device, thereby overcoming the problem of inaccurate depth value sensing at least to a certain extent.

[0005] According to a first aspect of the present disclosure, a depth value determination device is provided, comprising: a first histogram determination module, used to receive data of photons sensed by a standard pixel area corresponding to a laser emission point, and to perform statistics on the data of photons sensed by the standard pixel area to determine a first histogram; a second histogram determination module, used to receive data of photons sensed by pixel areas surrounding the standard pixel area, and to perform statistics on the data of photons sensed by the surrounding pixel areas to determine a second histogram; and a depth value calculation module, used to calculate a depth value sensed based on the laser emission point according to the first histogram and the second histogram.

[0006] According to a second aspect of the present disclosure, a method for determining a depth value is provided, comprising: receiving data of photons sensed by a standard pixel area corresponding to a laser emission point, and performing statistics on the data of photons sensed by the standard pixel area to determine a first histogram; receiving data of photons sensed by pixel areas surrounding the standard pixel area, and performing statistics on the data of photons sensed by the surrounding pixel areas to determine a second histogram; and calculating a depth value sensed based on the laser emission point according to the first histogram and the second histogram.

[0007] According to a third aspect of the present disclosure, a depth sensing module is provided, comprising the above-mentioned depth value determination device.

[0008] According to a fourth aspect of the present disclosure, an electronic device is provided, comprising the above-mentioned depth value determination device.

[0009] According to a fifth aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the above-mentioned depth value determination method is implemented.

[0010] In the technical solutions provided in some embodiments of the present disclosure, two histogram determination modules are used to respectively determine the histogram of photons sensed by the standard pixel area and the histogram of photons sensed by the surrounding pixel area, and then the depth value is calculated based on the two histograms. On the one hand, compared with the calculation method of using only one histogram to calculate the depth value in some technologies, the present disclosure combines the histogram corresponding to the standard pixel area and the histogram of the surrounding pixel area of ​​the standard pixel area in the process of calculating the depth value. The richness of the data can improve the accuracy of the depth value calculation; on the other hand, the depth value determination scheme of the present disclosure is easy to implement and can be applied to portable terminal devices such as smart phones; on the other hand, given that the scheme of the present disclosure can improve the accuracy of the depth value calculation, it can further improve the accuracy of the subsequent algorithm and has a wide range of application scenarios.

[0011] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present disclosure, and together with the specification, are used to explain the principles of the present disclosure. Obviously, the drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0013] Figure 1 A schematic diagram of a scene in which a laser radar is used to perform depth measurement in some embodiments of the present disclosure is shown;

[0014] Figure 2 A schematic diagram showing errors in depth detection results in some technologies;

[0015] Figure 3 Shown with Figure 2 The corresponding simulation histogram;

[0016] Figure 4 Schematically shows the Figure 2 Analysis diagram of the cause of depth value error;

[0017] Figure 5 A schematic diagram showing the phenomenon of diffuse reflection on the surface of some objects;

[0018] Figure 6 A block diagram schematically shows a depth value determination device according to an embodiment of the present disclosure;

[0019] Figure 7A schematic diagram schematically shows a standard pixel area and surrounding pixel areas of a laser emission point in an embodiment of the present disclosure;

[0020] Figure 8 A schematic diagram schematically shows a standard pixel area and surrounding pixel areas of a laser emission point in another embodiment of the present disclosure;

[0021] Fig. 9 A block diagram schematically shows a depth value determination device according to another embodiment of the present disclosure;

[0022] Fig.10 A block diagram of a first histogram determination module according to an embodiment of the present disclosure is schematically shown;

[0023] Fig.11 A block diagram schematically shows a second histogram determination module according to an embodiment of the present disclosure;

[0024] Fig.12 A block diagram of a depth value calculation module according to an embodiment of the present disclosure is schematically shown;

[0025] Fig.13 A schematic diagram showing a depth value determination process according to an embodiment of the present disclosure;

[0026] Fig.14 A flowchart of a method for determining a depth value according to an exemplary embodiment of the present disclosure is schematically shown;

[0027] Fig.15 A block diagram of an electronic device according to an exemplary embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0028] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as being limited to the examples set forth herein; on the contrary, these embodiments are provided so that the present disclosure will be more comprehensive and complete, and the concepts of the example embodiments are fully conveyed to those skilled in the art. The described features, structures, or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known technical solutions are not shown or described in detail to avoid obscuring various aspects of the present disclosure.

[0029] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0030] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the steps. For example, some steps may be decomposed, while some steps may be combined or partially combined, so the actual execution order may change according to the actual situation. In addition, all the terms "first" and "second" below are only for the purpose of distinction and should not be used as limitations of the present disclosure.

[0031] With the development of computer technology, computer vision technology has been widely used in people's daily life and work. As an important part of computer vision technology, depth sensing has broad application scenarios in the fields of imaging, spatial modeling, AR, assisted driving, etc.

[0032] Depth sensing relies on a depth sensing module. The depth sensing module of the present disclosure includes but is not limited to a laser radar (Light Detection and Ranging, LiDAR), which can measure depth using pulse flight time. Figure 1 The following describes the scenarios in which LiDAR is used for depth measurement.

[0033] refer to Figure 1 , the controller 10 is used to control the laser emitting end 11 to emit laser pulses, and the laser pulses are reflected by the object 19 to be measured in the environment and received by the laser receiving end 12. The laser receiving end 12 converts the received photons into electrical signals and sends them to the time data converter 13. The time data converter 13 can determine the flight time of the photons, and then a histogram can be generated using the data recorded in the time data converter 13. Among them, the memory 14 can be used to respond to the control of the controller 10 to save the flight time of the photons.

[0034] The histogram is a histogram of time and photon counts. After obtaining the histogram, the peak value of the count is determined from the histogram. The time of this peak value is the flight time of the laser pulse between the depth sensing module and the object being measured. Therefore, the distance of the object being measured from the depth sensing module can be calculated based on the distance being equal to the product of time and speed. This principle is time-correlated single photon counting (TCSPC).

[0035] In the actual depth sensing process, depth value sensing errors may occur. Figure 2 ,The RGB camera and the LiDAR shoot the same scene. The three measurement points "1", "2", and "3" in the depth map should correspond to the depth of the background (3m). However, the sensed depth is about 60cm.

[0036] Figure 3 for Figure 2 Histogram simulation results for medium depth, Figure 3 The peak value of about 3.5ns corresponds to a distance of about 60cm; the peak value of 20ns corresponds to the signal reflected from the background (3m).

[0037] Figure 4 Schematically shows the Figure 2 The cause of the depth value error is shown in the figure. Figure 4 On the one hand, after the laser transmitting end Tx emits the laser, the reflected light of the near view and the far view coincides with each other on the laser receiving end Rx. That is to say, the incident light 1 is reflected by the far view to obtain the reflected light 1, and the incident light 2 is reflected by the near view. Due to the possible diffuse reflection, the generated reflected light 2 and reflected light 1 are irradiated on the same pixel area of ​​the laser receiving end Rx. Figure 5 The case of incident light and diffuse reflected light is shown as an example. The diffuse reflection phenomenon is widely present on surfaces such as skin and walls.

[0038] On the other hand, theoretical analysis shows that the intensity of the reflected signal received on the laser receiving end Rx sensor is inversely proportional to the square of the distance, as shown in the following formula:

[0039]

[0040] Among them, P Signal.sensor Indicates the signal light intensity received by the laser receiving end Rx sensor, P Signal.out represents the intensity of signal light reflection, ρ object Indicates the reflectivity of the object being measured, D Rxlens represents the equivalent entrance pupil diameter of the laser receiving end Rx optical lens, d represents the distance between the measured object and the depth sensing module, η Rxlens Indicates the optical efficiency of the lens group.

[0041] Therefore, it can be understood that, although it is diffuse reflection, the intensity of the near-view reflected light may be greater than the intensity of the far-view reflected light.

[0042] In order to eliminate or at least reduce the above-mentioned problem of depth sensing error, the present disclosure provides a new depth value determination scheme.

[0043] Figure 6 The block diagram of the depth value determination device according to the embodiment of the present disclosure is schematically shown. Figure 6 The depth value determining device 6 of the embodiment of the present disclosure includes a first histogram determining module 61, a second histogram determining module 62 and a depth value calculating module 63.

[0044] Specifically, the first histogram determination module 61 can be used to receive data on photons sensed by the standard pixel area corresponding to the laser emission point, and to perform statistics on the data on the photons sensed by the standard pixel area to determine the first histogram; the second histogram determination module 62 can be used to receive data on photons sensed by the surrounding pixel areas of the standard pixel area, and to perform statistics on the data on the photons sensed by the surrounding pixel areas to determine the second histogram; the depth value calculation module 63 can be used to calculate the depth value based on the first histogram and the second histogram, and the depth value is the depth value sensed based on the laser emission point.

[0045] In the exemplary embodiment of the present disclosure, the laser emission point is any laser emission point on the laser emission end of the depth sensing module. It should be understood that the laser emission end includes multiple laser emission points, each of which emits laser pulses to different positions in the scene. Figure 2 In the example shown, the depth sensing module can emit 720 sensing points per frame, and each sensing point corresponds to a laser emission point, that is, the number of depth values ​​in the depth map is consistent with the number of laser emission points.

[0046] The laser emitting end is a chip array, with a size of, for example, 2mm×2mm, on which holes are opened, each hole corresponding to a laser emitting point. In these embodiments, the number of holes is the same as the number of laser emitting points described in the present disclosure, for example, 720.

[0047] In some other embodiments, the chip array is also matched with an optical diffraction device, in which case the number of holes and the number of laser emission points may not be the same. For example, there are 80 holes, and through the action of the optical diffraction device, each hole can generate 9 laser beams, which is equivalent to 720 laser emission points.

[0048] The standard pixel area corresponding to the laser emission point is: the pixel area determined on the sensor at the laser receiving end after the laser emission point is calibrated. The sensor can be understood as a pixel matrix (size is, for example, 180*210), and the standard pixel area corresponding to a laser emission point can be a pixel point or a region composed of multiple pixel points. In addition, the sensor at the laser receiving end mentioned in the present disclosure can be, for example, a SPAD (Single Photon Avalanche Diode) pixel array.

[0049] The surrounding pixel region of the standard pixel region may be a pixel region immediately adjacent to the standard pixel region, or may be a pixel region including pixels immediately adjacent to the standard pixel region.

[0050] In addition, the range of the standard pixel area and / or the surrounding pixel area is related to the size of the returned laser spot. The more concentrated the laser spot is, the smaller the pixel area can be, and the more dispersed the laser spot is, the larger the pixel area can be.

[0051] Figure 7 The schematic diagram schematically shows a standard pixel area and surrounding pixel areas of a laser emission point of the present disclosure. Figure 7 , the pixel point 70 corresponds to the standard pixel area, and the eight pixel points 71 adjacent to the pixel point 70 are the surrounding pixel areas of the standard pixel area.

[0052] Figure 8 Schematic diagram of another standard pixel area and surrounding pixel area of ​​a laser emission point of the present disclosure is shown. Figure 8 , a standard pixel area is composed of four pixel points 80, and 12 pixel points 81 adjacent to the four pixel points 80 are the surrounding pixel areas of the standard pixel area.

[0053] The standard pixel area corresponding to the laser emission point and the surrounding pixel area of ​​the standard pixel area are obtained through a pre-calibration process.

[0054] refer to Fig. 9 Compared with the depth value determining device 6 , the depth value determining device 9 may further include a calibration module 91 .

[0055] Specifically, the calibration module 91 can be used to predetermine the pixel area on the laser receiving end sensor where the light spot obtained by the laser emitted from the laser emission point after being reflected by the standard scene arrives as the standard pixel area corresponding to the laser emission point.

[0056] Standard scenes can be built in advance by developers. For example, place a piece of white paper about 1m in front of the depth sensing module and turn on the depth sensing module for sensing. The position of the light spot emitted by each laser emission point and reflected back by the white paper on the laser receiving end sensor is the standard pixel area corresponding to the laser emission point.

[0057] It should be understood that the surface of an object reflecting laser light in a standard environment is a surface with no diffuse reflection or slight diffuse reflection, and the present disclosure does not limit the specific construction method of the standard environment.

[0058] After obtaining the standard pixel area corresponding to each laser emission point, the mapping relationship between the laser emission point and the standard pixel area can be stored so that when the user subsequently determines the depth value of the scene object, the electronic device can directly use the mapping relationship to determine the standard pixel area corresponding to any laser emission point.

[0059] When the standard pixel area corresponding to the laser emission point is a pixel on the sensor, the first histogram determination module can count the photons sensed by the pixel to obtain a first histogram. The first histogram represents the relationship between the flight time and the photon count.

[0060] When the standard pixel area corresponding to the laser emission point is more than two pixels on the sensor, refer to Fig.10 The first histogram determination module 61 of the embodiment of the present disclosure may include a first OR logic unit 101 and a first data statistics unit 103 .

[0061] The first OR logic unit 101 can be used to perform OR logic processing on the data of photons sensed by each pixel in the standard pixel area to generate first intermediate data. The data of photons sensed by the pixel can be a digital signal, such as 1 for sensing photons and 0 for not sensing photons.

[0062] The first data statistics unit 103 may be configured to perform statistics on the first intermediate data to determine a first histogram.

[0063] It can be understood that, through the first OR logic unit 101, as long as one pixel point in the standard pixel area senses a photon, it is considered that the standard pixel area senses a photon.

[0064] refer to Fig.11 The second histogram determination module 62 of the embodiment of the present disclosure may include a second OR logic unit 111 and a second data statistics unit 113 .

[0065] The second OR logic unit 111 may be used to perform OR logic on the data of photons sensed by each pixel point in the surrounding pixel area to generate second intermediate data.

[0066] The second data statistics unit 113 may be configured to perform statistics on the second intermediate data to determine a second histogram.

[0067] In addition, any one of the first OR logic unit 101 and the second OR logic unit 111 may include at least one OR gate.

[0068] refer to Fig.12 The depth calculation module 63 of the embodiment of the present disclosure may include a histogram processing unit 121 and a data conversion unit 123 .

[0069] The histogram processing unit 121 can be used to subtract the first histogram from the second histogram to obtain a third histogram. It should be understood that the subtraction of the histograms in the present disclosure refers to subtracting the values ​​of photon counts (vertical axis) for the same time (horizontal axis).

[0070] Taking into account device errors and / or special environmental interference, there may be negative photon counts in the result of subtracting the first histogram from the second histogram. In some embodiments of the present disclosure, the negative photon counts can be set to 0 to unify the form of the histogram and facilitate subsequent processing.

[0071] In addition, in the case where the scale bar of the first histogram is inconsistent with the scale bar of the second histogram, before performing the subtraction process, the embodiment of the present disclosure can also first adjust the scale bar of the first histogram to be consistent with the scale bar of the second histogram, and then subtract the first histogram with consistent scales from the second histogram to obtain a third histogram.

[0072] For example, the scale of the first histogram is adjusted to be consistent with the scale of the second histogram; or, the scale of the second histogram is adjusted to be consistent with the scale of the first histogram.

[0073] The data conversion unit 123 may be configured to calculate a depth value sensed based on the laser emission point using the time corresponding to the peak value in the third histogram.

[0074] Specifically, the depth value can be calculated by the time corresponding to the laser speed and the peak value, and the depth value is based on the depth value sensed at the laser emission point. In the depth map, the depth value is only the value of a point, such as Figure 2 That is to say, for each standard pixel area on the depth sensing module, the above depth value determination process is performed to obtain a depth map.

[0075] Fig.13 The embodiment of the present disclosure is shown Figure 7 Schematic diagram of the corresponding depth value determination process.

[0076] refer to Fig.13 On the one hand, the standard pixel area corresponding to the laser emission point outputs a digital signal of 1 or 0, which is counted to generate the first histogram.

[0077] On the other hand, each pixel point in the surrounding pixel area of ​​the standard pixel area can output a digital signal of 1 or 0, and they are subjected to an OR logic process to generate a second histogram.

[0078] For pixel a and pixel b in the surrounding pixel area, where pixel a and pixel b may be adjacent pixels in the surrounding pixel area or two pixels at random positions, pixel a outputs signal A and pixel b outputs signal B. Signal A and signal B are used as inputs of an OR gate for OR processing, and then processed by two levels of OR gates to obtain an OR-processed digital signal, which is statistically processed to generate a second histogram.

[0079] It should be understood that Fig.13 The OR gate structure shown is only an exemplary description, and other OR gate configurations may also be used to implement OR logic processing, and the present disclosure does not limit this.

[0080] In addition, for the statistical method of the first histogram and the second histogram of the present disclosure, in addition to the processing method of OR logic, logical structures such as AND gates can also be added therein, and the present disclosure does not impose any limitation on this.

[0081] For example, in the process of generating the second histogram of the present disclosure, when more than a predetermined proportion of pixels in the surrounding pixel area output signals of detecting photons, it is considered that photons are detected in the surrounding pixel area at this moment. The present disclosure does not limit the predetermined proportion. For example, in a surrounding pixel area with a total of 8 pixels, if more than 5 pixels detect photons (output digital signal 1), the photon count is increased by 1. The present disclosure does not limit the circuit configuration methods similar to this situation.

[0082] Furthermore, the present disclosure also provides a method for determining a depth value.

[0083] refer to Fig.14 , the depth value determination method of the embodiment of the present disclosure may include:

[0084] S142. Receive data of photons sensed by the standard pixel area corresponding to the laser emission point, and perform statistics on the data of photons sensed by the standard pixel area to determine a first histogram.

[0085] S144. Receive data of photons sensed by pixel regions surrounding the standard pixel region, and perform statistics on the data of photons sensed by the pixel regions surrounding the standard pixel region to determine a second histogram.

[0086] S146. Calculate the depth value sensed based on the laser emission point according to the first histogram and the second histogram.

[0087] According to an exemplary embodiment of the present disclosure, a depth value determination method may include: predetermining a pixel area on a laser receiving end sensor where a light spot obtained by reflecting a laser emitted from a laser emission point through a standard scene reaches the pixel area as a standard pixel area corresponding to the laser emission point.

[0088] According to an exemplary embodiment of the present disclosure, the standard pixel area includes more than two pixel points. In this case, the process of determining the first histogram can be configured to execute: performing OR logic processing on the data of photons sensed by each pixel point in the standard pixel area to generate first intermediate data, and performing statistics on the first intermediate data to determine the first histogram.

[0089] According to an exemplary embodiment of the present disclosure, the process of determining the second histogram can be configured to perform: performing OR logic processing on the data of photons sensed by each pixel point in the surrounding pixel area to generate second intermediate data, and performing statistics on the second intermediate data to determine the second histogram.

[0090] According to an exemplary embodiment of the present disclosure, the process of calculating the depth value sensed based on the laser emission point according to the first histogram and the second histogram can be configured to perform: subtracting the first histogram from the second histogram, and using the time corresponding to the peak in the third histogram to calculate the depth value sensed based on the laser emission point.

[0091] According to an exemplary embodiment of the present disclosure, when the scale of the first histogram is inconsistent with the scale of the second histogram, the scale of the first histogram is adjusted to be consistent with the scale of the second histogram, and the first histogram with consistent scales is subtracted from the second histogram to obtain a third histogram.

[0092] Since the various processing procedures of the depth value determination method according to the embodiment of the present disclosure are the same as those described in the above-mentioned depth value determination device, they will not be described in detail here.

[0093] Furthermore, the present disclosure also provides a depth sensing module, which may include the depth value determination device of the present disclosure. That is, the first histogram determination module, the second histogram determination module and the depth value calculation module may all be configured in the depth sensing module, and the depth sensing module as a whole implements the depth value determination process of the present disclosure.

[0094] In addition, it should be noted that the above-mentioned depth value determination device can also be configured independently of the depth sensing module, that is, in terms of hardware structure, the depth sensing module and the depth determination device are two independent devices, which are usually connected by wire to transmit data. In this case, the depth value determination device of the present disclosure is configured separately and can be combined with various types or models of depth sensing modules to improve the accuracy of the depth sensing module in sensing depth.

[0095] Fig.15 Schematic diagram of an electronic device suitable for implementing the exemplary embodiment of the present disclosure is shown. It should be noted that, Fig.15 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0096] The electronic device of the present disclosure includes at least a processor and a memory, and the memory is used to store one or more programs. When the one or more programs are executed by the processor, the processor can implement the depth value determination method of the exemplary embodiment of the present disclosure. Further, the electronic device of the embodiment of the present disclosure may include the above-mentioned depth value determination device.

[0097] Specifically, Fig.15 As shown, the electronic device 150 may include: a processor 1510, an internal memory 1520, an external memory interface 1530, a communication module 1540, an audio module 1550, a display screen 1560, a camera module 1570, a depth sensing module 1580, a sensor module 1590, etc. The sensor module 1590 may include a pressure sensor, a gyroscope sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, etc.

[0098] It is understood that the structure illustrated in the embodiment of the present disclosure does not constitute a specific limitation on the electronic device 150. In other embodiments of the present disclosure, the electronic device 150 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0099] The processor 1510 may include one or more processing units, for example, the processor 1510 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor and / or a neural network processor (NPU), etc. Among them, different processing units may be independent devices or integrated into one or more processors. In addition, a memory may be provided in the processor 1510 for storing instructions and data.

[0100] The electronic device 150 can realize the shooting function through the ISP, the camera module 1570, the video codec, the GPU, the display screen 1560 and the application processor. In some embodiments, the electronic device 150 may include 1 or N camera modules 1570, where N is a positive integer greater than 1. If the electronic device 150 includes N cameras, one of the N cameras is a main camera.

[0101] The electronic device 150 can sense depth information in the scene through the depth sensing module 1580, for example, sense the depth value of the target object in the scene, and the target object includes the object that the user pays attention to or is interested in. The depth sensing module 1580 may include the depth value determination device of the embodiment of the present disclosure.

[0102] The electronic device 150 can combine the depth value sensed by the depth sensing module 1580 of the present disclosure to implement subsequent processing processes. The subsequent processing processes mentioned in the present disclosure include but are not limited to spatial modeling, AR interactive interaction, assisted driving, and image optimization by cooperating with the images taken by the camera module 1570.

[0103] The internal memory 1520 may be used to store computer executable program codes, which may include instructions. The internal memory 1520 may include a program storage area and a data storage area. The external memory interface 1530 may be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 150.

[0104] The present disclosure also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiments; or may exist independently without being assembled into the electronic device.

[0105] The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0106] Computer-readable storage media can send, propagate or transmit programs for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable storage medium can be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0107] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by an electronic device, the electronic device implements the method described in the embodiments of the present disclosure.

[0108] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0109] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, and the units described may also be arranged in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0110] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0111] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0112] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0113] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing what is disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and embodiments are to be considered merely as exemplary, and the true scope and spirit of the present disclosure are indicated by the claims.

[0114] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A depth value determination device, characterized in that: include: A first histogram determination module is used to receive data of photons sensed by a standard pixel area corresponding to a laser emission point, and to perform statistics on the data of photons sensed by the standard pixel area to determine a first histogram; the standard pixel area corresponding to the laser emission point is a pixel area determined on a sensor at a laser receiving end after calibration of the laser emission point; A second histogram determination module is used to receive data of photons sensed by pixel regions surrounding the standard pixel region, and to perform statistics on the data of photons sensed by the pixel regions surrounding the standard pixel region to determine a second histogram; a depth value calculation module, configured to calculate a depth value sensed based on the laser emission point according to the first histogram and the second histogram; Wherein, the depth value calculation module includes: a histogram processing unit, configured to subtract the first histogram from the second histogram to obtain a third histogram; The data conversion unit is used to calculate the depth value sensed based on the laser emission point by using the time corresponding to the peak in the third histogram.

2. The depth value determination device according to claim 1, characterized in that: The depth value determination device also includes: The calibration module is used to predetermine the pixel area on the laser receiving end sensor where the light spot obtained by the laser emitted from the laser emission point after being reflected by the standard scene arrives as the standard pixel area corresponding to the laser emission point.

3. The depth value determination device according to claim 1 or 2, characterized in that: The standard pixel area includes more than two pixel points, and the first histogram determination module includes: A first OR logic unit, configured to perform OR logic processing on data of photons sensed by each pixel point in the standard pixel area to generate first intermediate data; The first data statistics unit is used to perform statistics on the first intermediate data to determine the first histogram.

4. The depth value determination device according to claim 1, characterized in that: The second histogram determination module comprises: A second OR logic unit, used for performing OR logic processing on the data of photons sensed by each pixel point in the surrounding pixel area to generate second intermediate data; The second data statistics unit is used to perform statistics on the second intermediate data to determine the second histogram.

5. The depth value determination device according to claim 1, characterized in that: The histogram processing unit is also used to adjust the scale of the first histogram to be consistent with the scale of the second histogram when the scale of the first histogram is inconsistent with the scale of the second histogram, and subtract the first histogram and the second histogram with consistent scales to obtain the third histogram.

6. A method for determining a depth value, characterized in that: include: Receiving data of photons sensed by a standard pixel area corresponding to a laser emission point, and performing statistics on the data of photons sensed by the standard pixel area to determine a first histogram; The standard pixel area corresponding to the laser emission point is a pixel area determined on the sensor at the laser receiving end after the laser emission point is calibrated; Receiving data of photons sensed by pixel regions surrounding the standard pixel region, and performing statistics on the data of photons sensed by the pixel regions surrounding the standard pixel region to determine a second histogram; subtracting the first histogram from the second histogram to obtain a third histogram; The depth value sensed based on the laser emission point is calculated using the time corresponding to the peak in the third histogram.

7. The method for determining a depth value according to claim 6, wherein: The depth value determination method further includes: It is predetermined that the light spot obtained by the laser emitted from the laser emission point after being reflected by the standard scene reaches the pixel area on the laser receiving end sensor, which is used as the standard pixel area corresponding to the laser emission point.

8. A depth sensing module, characterized in that: The invention comprises the depth value determining device according to any one of claims 1 to 5.

9. An electronic device, characterized in that: The invention comprises the depth value determining device according to any one of claims 1 to 5.

Images