Image processing method, device, storage medium and electronic device

Abstract

The embodiments of the present application disclose an image processing method, device, storage medium and electronic device, wherein the method comprises: obtaining a reflection intensity map, determining at least one light spot in the reflection intensity map, obtaining pixel characteristic parameters corresponding to light spot pixels in the light spot, and determining an interfering light spot from the at least one light spot based on the pixel characteristic parameters. By using the embodiments of the present application, interfering light spots can be accurately identified.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an image processing method, device, storage medium and electronic device. Background Art

[0002] At present, in the three-dimensional imaging scene, it is mainly based on binocular cameras, result light method and time of flight method. Taking the indirect time of flight method (iTOF method) in the time of flight method as an example, the iTOF method is a method of determining the distance to the target by measuring the phase relationship between the modulated emitted light and the received light. It is widely used in gesture recognition, face recognition, three-dimensional modeling, somatosensory games, machine vision, auxiliary focus, security, automatic driving and other related application scenarios related to three-dimensional depth vision. Summary of the invention

[0003] The present application provides an image processing method, device, storage medium and electronic device, and the technical solution is as follows:

[0004] In a first aspect, an embodiment of the present application provides an image processing method, the method comprising:

[0005] Acquire a reflection intensity map, and determine at least one light spot in the reflection intensity map;

[0006] Obtaining pixel characteristic parameters corresponding to the light spot pixel points in the light spot;

[0007] Based on the pixel characteristic parameter, an interfering light spot is determined from the at least one light spot.

[0008] In a second aspect, an embodiment of the present application provides an image processing device, the device comprising:

[0009] a light spot determination module, configured to obtain a reflection intensity map and determine at least one light spot in the reflection intensity map;

[0010] A parameter acquisition module, used to acquire pixel characteristic parameters corresponding to the light spot pixel points in the light spot;

[0011] The interference determination module is used to determine an interference light spot from the at least one light spot based on the pixel characteristic parameter.

[0012] In a third aspect, an embodiment of the present application provides a computer storage medium, wherein the computer storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor and executing the above-mentioned method steps.

[0013] In a fourth aspect, an embodiment of the present application provides an electronic device, which may include: a processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the above-mentioned method steps.

[0014] The beneficial effects brought about by the technical solutions provided by some embodiments of the present application include at least:

[0015] In one or more embodiments of the present application, by acquiring a reflection intensity map, determining at least one light spot in the reflection intensity map, and then acquiring pixel characteristic parameters corresponding to the light spot pixels in the light spot, based on the difference in pixel characteristic parameters between the interfering light spot and the effective light spot, the interfering light spot can be accurately determined from at least one light spot, which can assist in eliminating light spot interference caused by the superposition of edge signals of multiple light spots; the entire image interference processing process reduces dependence on equipment hardware and complex algorithms, realizes accurate identification of light spot interference, can assist in outputting high-precision depth measurement results, and improves the robustness in depth measurement scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 It is a flowchart of an image processing method provided in an embodiment of the present application;

[0018] Figure 2 is a schematic diagram of a partial area in a reflection intensity map involved in the image processing method provided in an embodiment of the present application;

[0019] Figure 3 It is a local schematic diagram of a scene involving light spot interference points involved in the image processing method provided in an embodiment of the present application;

[0020] Figure 4 is a flowchart of another image processing method provided in an embodiment of the present application;

[0021] Figure 5 is a flowchart of another image processing method provided in an embodiment of the present application;

[0022] Figure 6 is a scene schematic diagram of a light spot provided in an embodiment of the present application;

[0023] Figure 7 is a schematic diagram of a scene involving an interference spot provided in an embodiment of the present application;

[0024] Figure 8 is a structural schematic diagram of an image processing device provided in an embodiment of the present application;

[0025] Fig. 9 is a structural diagram of an interference determination module provided in an embodiment of the present application;

[0026] Fig.10 is a structural schematic diagram of another image processing device provided in an embodiment of the present application;

[0027] Fig.11 is a structural diagram of an interference verification module provided in an embodiment of the present application;

[0028] Fig.12 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application;

[0029] Fig.13 It is a schematic diagram of the structure of the operating system and user space provided in the embodiment of the present application;

[0030] Fig.14 yes Fig.13 The architecture diagram of the Android operating system;

[0031] Fig.15 yes Fig.13 Architecture diagram of the IOS operating system. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0033] In the description of the present application, it should be understood that the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In the description of the present application, it should be noted that, unless otherwise clearly specified and limited, "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood in specific circumstances. In addition, in the description of the present application, unless otherwise specified, "multiple" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B, which can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are an "or" relationship.

[0034] The present application is described in detail below with reference to specific embodiments.

[0035] In one embodiment, Figure 1 As shown, an image processing method is proposed, which can be implemented by a computer program and can be run on an image processing device based on the von Neumann system. The computer program can be integrated in an application or run as an independent tool application. The image processing device can be a terminal, including but not limited to: a personal computer, a tablet computer, a handheld device, a vehicle-mounted device, a wearable device, a computing device or other processing device connected to a wireless modem. In different networks, terminal devices can be called different names, such as: user equipment, access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication device, user agent or user device, cellular phone, cordless phone, 5G network or terminal device in future evolution network, etc.

[0036] Specifically, the image processing method includes:

[0037] S101: Acquire a reflection intensity map, and determine at least one light spot in the reflection intensity map;

[0038] The reflection intensity map is obtained based on the TOF camera carried by the terminal. The TOF camera is a camera that uses the time-of-flight method for 3D imaging. The terminal emits a modulated light beam through the TOF camera, which is reflected after encountering an object. The terminal can receive the reflected light signal after being reflected by the object through a photosensitive element (Sensor) such as a TOF camera to generate a reflection intensity map. The reflection intensity map can be used to feedback the pixel intensity corresponding to at least one pixel point. The light energy corresponding to at least one pixel point is proportional to the pixel intensity of the pixel point in the reflection intensity map, that is, the stronger the light energy corresponding to the pixel point is, the higher the pixel intensity of the pixel point is. In actual application scenarios, the terminal uses a TOF camera to perform scene imaging, and can obtain a depth map (target scene depth distance map, also called Depth map) and a reflection intensity map in the current scene.

[0039] It can be understood that TOF cameras are based on time-of-flight 3D imaging. The time-of-flight method can be divided into direct time-of-flight (dTOF) and indirect time-of-flight (iTOF) according to the different signal results obtained. The signal result obtained by the direct time-of-flight method is the time difference, and the signal result obtained by the indirect time-of-flight method is the phase difference.

[0040] It can be understood that a TOF camera can be understood as a TOF module integrated in a terminal, and a TOF module generally includes a light receiving end (RX) and a light transmitting end (TX);

[0041] Schematically, the light receiving end usually includes a lens, a narrowband filter (such as a 940nm narrowband filter), a TOF sensor, etc., and the light emitting end usually includes a vertical cavity surface emitting laser (Vertical-Cavity Surface-Emitting Laser, VCSEL) array, a collimating lens, an optical diffractive element (Diffractive Optical Elements, DOE), etc., and the DOE can be a diffuser. The VCSELs in the VCSEL array are evenly arranged on a plane, and are used to emit a light beam with a two-dimensional regular pattern; the collimating lens is used to receive and converge the light beam to ensure that the light beam has a parallel light path; the DOE is used to scatter the light beam processed by the collimating lens, and project it onto the object in the scene according to a certain field of view. The bright spot formed after being projected onto the object and reflected and received by the light receiving end is the speckle. For example, the VCSEL array usually includes at least two VCSELs, and the at least two VCSELs are evenly arranged. When the evenly arranged VCSELs are used to emit a light beam, a regular and uniform speckle can be obtained.

[0042] In practical applications, the light intensity value of each pixel received by the optical receiver (RX) is usually represented as the confidence of "pixel (point) resolution to generate depth value", i.e. Confidence. The reflection intensity map can be understood as a map used to feedback the light intensity value received by each pixel. In some embodiments, the reflection intensity map can also be called a confidence map (Confidence map). The parameter value of each pixel point in the confidence map is represented by the intensity of the received signal light, which is then used as the confidence of the target distance measurement value of the point.

[0043] Optionally, the TOF camera carried by the terminal can be a device that supports the speckle iTOF mode, that is, it can be used as a speckle iTOF camera. The modulated infrared light emitted by the light transmitting end (TX) of the terminal through the TOF camera in the speckle iTOF mode is a plurality of beams of light. After being reflected by the target object in the scene, a plurality of light spots are received on the RX end, that is, only the area illuminated by the light spots on the depth map will have valid depth values, and the remaining areas are invalid values. If valid depth values ​​appear in the remaining areas, the depth value pixels appearing in this area are interference points, which can be removed accordingly. In the present application, the image processing method can be executed to accurately identify the light spot interference based on the reflection intensity map.

[0044] It is understandable that after being reflected by the target object in the scene, the several light spots received on the RX end are actually several speckles. Usually at least one speckle will cover several pixels, but each speckle only generates one valid depth value in application scenarios such as depth imaging and ranging.

[0045] In a feasible implementation, after acquiring the reflection intensity map, the terminal may determine at least one light spot based on intensity distribution information of at least one pixel point in the reflection intensity map.

[0046] Schematically, the light spot shape in the reflection intensity map usually presents a two-dimensional Gaussian shape, that is, the intensity at the center of the light spot is high and gradually weakens around the light spot; that is, at least one light spot can be identified based on the light spot shape in the reflection intensity map.

[0047] S102: Obtain pixel characteristic parameters corresponding to the light spot pixel points in the light spot;

[0048] Schematically, in the reflection intensity map, at least one area where a light spot is located may cover multiple light spot pixels. Figure 2 As shown, Figure 2 is a schematic diagram of a portion of the reflection intensity map. Figure 2 In the figure, the dotted circular frame can be regarded as a light spot, and the light spot corresponding to the dotted circular frame presents a two-dimensional Gaussian shape, that is, the intensity of the light spot is high in the center and gradually weakens around the light spot. Figure 2Each pixel grid in the image can be regarded as a pixel point, and the area covered by the dotted circular frame usually contains multiple pixel points, which are also light spot pixel points. The pixel characteristic parameters can be pixel intensity parameters, pixel position parameters, etc. of the light spot pixel points. The pixel characteristic parameters can be directly obtained based on the reflection intensity map.

[0049] S103: Determine an interfering light spot from the at least one light spot based on the pixel characteristic parameter.

[0050] It is understandable that since the reflected light beam will diverge (that is, one light spot covers multiple pixels in the reflection intensity map), the light spot shape in the reflection intensity map usually presents a two-dimensional Gaussian shape, and the central light energy is the strongest, that is, Figure 2 The interfering light spot is generated by the superposition of the divergent edge signals of at least two light spots. For example, if the distance between multiple light spots is close, the divergent edge signals of these multiple light spots will be superimposed on each other, forming a "pseudo light spot" between these light spots. This "pseudo light spot" also conforms to the two-dimensional Gaussian shape, but the received light energy is much smaller than the theoretical design value, and the signal-to-noise ratio is poor in subsequent application scenarios. In actual application scenarios, these "pseudo light spots" can also generate a rough depth value and will be mistakenly identified as a light spot, but most of them have poor accuracy and will interfere with the back-end application, which is the interference point.

[0051] For example, due to objective factors such as lens distortion and light signal divergence, when the distance between at least two light spots is close (for example, the distance between two light spots is about twice the distance between the center areas of the light spots), a "pseudo light spot" or a light spot interference point will appear between the light spots that affect each other. Figure 3 As shown, Figure 3 It is a local schematic diagram of a scene involving light spot interference points. In a certain actual application scenario, there are actually four light spots on the surface of an object in the scene, and the light spot intensities of the four generated light spots are very close (the values ​​in the depth map (Depth map) or the reflection intensity map are equivalent). The back-end application scenarios such as virtual reality and ranging will think that the surface of the area corresponding to these four light spots is a plane. However, due to signal divergence between the four light spots (it can be understood that due to reasons such as hardware design at the factory stage, at least one light spot may actually cover multiple pixels in the reflection intensity map) and there is a signal superposition phenomenon at least between the edge signals of two or more light spots. That is to say, due to the signal superposition phenomenon, there will be an interference light spot, such as Figure 3 As shown, Figure 3The middle light spot shown is actually an interference light spot generated by the superposition of edge signals between multiple light spots. Usually, the pixel points with high pixel intensity in the area covered by the interference light spot can be used as interference light spots. It is understandable that due to the large difference between the parameters of the interference light spot and the actual light spot (such as the depth value in the depth map and the pixel intensity value in the reflection intensity map), the surface of these five points may be fitted as a curved surface in the back-end application scenario, causing serious errors. Based on this interference light spot, the depth pixel points at the corresponding position in the depth map can be eliminated to reduce the light spot interference.

[0052] It can be understood that by performing interference light spot detection on at least one light spot based on the pixel characteristic parameters, so as to detect the difference in pixel characteristic parameters (such as pixel intensity parameters and pixel position parameters) corresponding to pixel points between the actual light spot and the interference light spot generated by the superposition of edge light signals of multiple actual light spots, the terminal can determine the interference light spot from at least one light spot based on this; for example, interference light spot intensity detection can be performed on at least one light spot based on the pixel intensity parameter, and the terminal can determine the interference light spot from at least one light spot; for example, interference light spot intensity detection can be performed on at least one light spot based on the pixel position parameter, and the terminal can determine the interference light spot from at least one light spot; in actual application scenarios, the terminal can usually determine an interference light spot from at least one light spot, and based on the actual application scenario, at least one pixel point covered by the interference light spot is used as the interference light spot.

[0053] In one or more embodiments involved in the present application, by obtaining a reflection intensity map, determining at least one light spot in the reflection intensity map, and then obtaining pixel characteristic parameters corresponding to the light spot pixels in the light spot, based on the difference in pixel characteristic parameters between the interfering light spot and the effective light spot, the interfering light spot can be accurately determined from at least one light spot, which can assist in eliminating the light spot interference caused by the superposition of edge signals of multiple light spots; the entire image interference processing process reduces the dependence on device hardware and complex algorithms, realizes accurate identification of light spot interference, can assist in outputting high-precision depth measurement results, and improves the robustness in depth measurement scenarios.

[0054] See also Figure 4 , Figure 4 This is a flow chart of another embodiment of an image processing method proposed in this application. Specifically:

[0055] S201: Acquire a reflection intensity map, and determine at least one light spot in the reflection intensity map;

[0056] For details, please refer to the method steps involved in one or more embodiments of the present application, which will not be repeated here.

[0057] S202: Obtain the number of light spots corresponding to the light spot; if the number of light spots is greater than a number threshold, adjust the camera exposure time.

[0058] According to some embodiments, after acquiring the reflection intensity map, the terminal may determine at least one light spot based on intensity distribution information of at least one pixel point in the reflection intensity map.

[0059] Schematically, the light spot shape in the reflection intensity map is usually a two-dimensional Gaussian shape, that is, the intensity of the light spot center is high and gradually weakens around the light spot; that is, at least one light spot can be identified based on the light spot shape in the reflection intensity map, and then the number of all light spots can be calculated to obtain the number of light spots corresponding to the light spot, that is, the number of light spots in the reflection intensity map. It is understandable that the number of light spots counted may have interference spots.

[0060] In this application specification, the quantity threshold is a threshold value used to measure the impact of the number of interfering light spots. If the number of detected light spots is greater than the quantity threshold, it can be considered that the number of interfering light spots is huge, and the energy of these interfering light spots is usually large, and the depth value of the interfering light spots corresponding to the interfering light spots in the depth map is also high.

[0061] It is understandable that in actual application scenarios, the TOF camera carried by the terminal can be a device that supports the speckle iTOF mode, such as a speckle iTOF camera; the TOF camera carried by the terminal is usually customized and calibrated according to the actual use scenario (target detection distance, detection accuracy, target object reflectivity, whether it is used outdoors, etc.) at the factory stage, including sensor selection, light source design (including light source power, light point arrangement), lens design and other software and hardware aspects. In the terminal use stage after leaving the factory, the terminal works based on a certain speckle iTOF mode, and the theoretical number of spots theoretically received by the light receiving end in a certain speckle iTOF mode, the theoretical number of pixels in the area covered by the spot, etc. have been determined. For example, after the actual application stage of the speckle iTOF camera, how many spots will there be in the field of view theoretically, how many pixels will each spot theoretically cover, and the theoretical number threshold of the spot corresponding to the theory is usually set, and the terminal can obtain it accordingly. Optionally, the threshold for the number of spots is usually greater than the theoretical number of spots (that is, the actual number of spots).

[0062] In a specific implementation scenario, if the number of detected light spots is greater than the number threshold, it can be considered that the number of interfering light spots is huge, and the energy of these interfering light spots is usually large, and the depth value of the interfering light spots corresponding to the interfering light spots in the depth map is also high. At this time, a possible scenario is that the entire TOF camera field of view of the terminal is a close-range target object. Even the area that is originally not a light spot in the depth map or the reflection intensity map can receive the scattered light due to the light signal of the close-range target object, and the number of interfering light spots or interfering light spots generated between each other is large, and the energy and intensity are high. The terminal can adjust the camera exposure time. There is usually a camera module in the terminal, such as an automatic exposure (Auto Focus, AE) algorithm module. The terminal can send an indication signal to the AE (Auto Focus automatic exposure) algorithm module based on the number of light spots fed back by the reflection intensity map, assist the overexposure judgment process of the AE (Auto Focus automatic exposure) algorithm module, and reduce the exposure time when judging overexposure to ensure the measurement of the depth distance of the close-range target object. Thereby, a light spot number is realized as an overexposure judgment condition based on the number of light spots in the reflection intensity map, and the automatic exposure process is optimized, which further increases the practicality of terminals such as iTOF cameras.

[0063] Optionally, after the terminal adjusts the camera exposure time, it can execute the step of obtaining the reflection intensity map and determining at least one light spot in the reflection intensity map, that is, after completing the camera exposure time, the reflection intensity map is re-acquired to achieve interference optimization and optimize the optimization effect in the actual application scenario.

[0064] S203: Obtain pixel characteristic parameters corresponding to the light spot pixel points in the light spot;

[0065] For details, please refer to the method steps involved in one or more embodiments of the present application, which will not be repeated here.

[0066] S204: Determine at least one light spot interference detection method for the reflection intensity map.

[0067] In a specific implementation scenario, the current application scenario can be obtained, and different application scenarios correspond to different combinations of light spot interference detection methods (the combination includes at least one light spot interference detection method). In different application scenarios, the requirements for measurement accuracy, interference adaptability, etc. are different. Different reference light spot interference detection method combinations can be set in advance based on different reference application scenarios, that is, an interference detection combination mapping relationship between at least one reference application scenario and its corresponding reference light spot interference detection method combination is established. In actual applications, after determining the current application scenario, the light spot interference detection method combination in the current application scenario can be determined based on the aforementioned interference detection combination mapping relationship, so as to determine at least one light spot interference detection method for the reflection intensity map based on the light spot interference detection method combination.

[0068] Illustratively, the interference detection combination mapping relationship may be represented in the form of a mapping combination, a mapping list, a mapping array, etc., which is not specifically limited here.

[0069] Illustratively, different reference application scenarios may be application scenarios of the type of terminal involving long-distance depth measurement, virtual reality application, visual environment reconstruction, virtual reality game, plane detection, etc.

[0070] Optionally, the light spot interference detection method includes but is not limited to the fitting of one or more detection methods such as pixel intensity detection method, energy ratio detection method, pixel mean detection method, total energy intensity detection method, and light spot position detection method.

[0071] Optionally, when it is determined that there are at least two light spot interference detection modes in the current application scenario, the detection timing between at least one light spot interference detection mode is not restricted, and at least one light spot interference detection mode can be executed synchronously or asynchronously.

[0072] Optionally, when it is determined that there is at least one light spot interference detection method in the current application scenario, a corresponding interference light spot can be obtained based on each light spot interference detection method, and whether the interference light spot or the interference light spot corresponding to the interference light spot is removed can be removed in combination with the interference light spot removal strategy set in the actual scenario. For example, multiple interference light spots (such as the first interference light spot, the second interference light spot, the third interference light spot, the fourth interference light spot, the fifth interference light spot, etc.) can be obtained by combining multiple light spot interference detection methods, and the intersection of these interference light spots can be taken. For example, based on the interference light spot removal strategy set in the actual application scenario, if a certain light spot is detected as belonging to "the first interference light spot, the second interference light spot, the third interference light spot, the fourth interference light spot, and the fifth interference light spot" at the same time, then the certain light spot is removed.

[0073] S205: performing light spot interference detection processing on at least one of the light spots using the light spot interference detection method based on the pixel characteristic parameters to obtain an interference light spot.

[0074] In a feasible implementation, the light spot interference detection method may be a pixel intensity detection method, by detecting the pixel intensity of the light spot pixel point, so as to distinguish the interference light spot from the identified multiple light spots. In practical application scenarios, for example, the reflection intensity map under the current field of view is obtained by the light receiving end of the speckle iTOF camera, wherein the proportion of the number of pixels covered by all actual light spots in the reflection intensity map to the number of pixels in the entire reflection intensity map is usually relatively small, for example, about a few percent to more than ten percent. In the area not covered by the actual light spot, the value of the pixel point in the reflection intensity map or the depth map, or the pixel intensity value (that is, the Confidence value) is not 0, which is caused by the presence of noise such as ambient light, circuit, dark circuit, etc. However, the pixel intensity value (that is, the Confidence value) of the area not covered by these actual light spots is usually very low (even if there is an interference light spot, the pixel intensity value of the interference light spot is usually much lower than the pixel intensity value of the actual light spot). In some scenarios, the pixel intensity value (that is, the Confidence value) of the interference light spot or edge area is one to several orders of magnitude lower than the pixel intensity value of the actual light spot signal light. Based on this, the light spot intensity parameters corresponding to the light spot pixels covered by all the light spots identified in the reflection intensity map can be detected, so as to determine the interference light spots from the identified light spots, that is, determine the interference light spots.

[0075] It can be understood that after the terminal determines the light spot interference detection method, if the light spot interference detection method is a pixel point intensity detection method, the terminal can perform pixel point intensity detection processing on at least one light spot based on the pixel intensity parameter to obtain a first interference light spot; the first interference light spot is also the interference light spot detected by the pixel point intensity detection method.

[0076] Optionally, the pixel intensity detection method can specifically be to draw an intensity statistical graph, such as an intensity histogram that characterizes the pixel intensity. The intensity histogram can be used to characterize the pixel intensity of all light spot pixels covered by the light spots. For example, light spot pixels with the same pixel intensity value can be fed back, for example, the intensity priority of each light spot pixel can be fed back.

[0077] It is understandable that in the terminal use stage after leaving the factory, the terminal works based on a certain speckle iTOF mode, and the theoretical number of light spots theoretically received by the light receiving end under a certain speckle iTOF mode, the theoretical number of pixels in the light spot coverage area, etc. have been determined. For example, after the actual application stage of the speckle iTOF camera, how many light spots will there be in the field of view theoretically, and how many pixels will the light spots theoretically cover; further, since the number of target light spots of theoretical light spot pixels (excluding interference points) can be determined in advance, or the target light spot ratio of theoretical light spot pixels (excluding interference points) (the ratio of theoretical light spot pixels to the total number of pixels in the image) can be determined. Based on this, when detecting the pixel intensity, it is only necessary to determine the reference light spot (some embodiments may be referred to as the first light spot or the second light spot) indicated by the target light spot ratio or the target light spot number from all the light spot pixels, and use the light spot intensity parameter corresponding to the reference light spot as a threshold to screen the interference points.

[0078] In a feasible implementation, the light spot interference detection method may be an energy ratio detection method, which detects the energy ratio of the pixels contained in the light spot to distinguish the interference light spot from the identified multiple light spots. Indicatively, the energy ratio detection method may traverse at least one light spot, calculate the energy ratio of the pixel intensity parameter (i.e., the Confidence value) of the center pixel of the light spot and the Confidence value of other pixels of the light spot except the center pixel of the light spot. Usually, the signal light energy of the actual light spot is concentrated, and its energy ratio will be greater than that of the interference light spot.

[0079] It is understandable that after the terminal determines the light spot interference detection mode, if the light spot interference detection mode is the energy ratio detection mode, the terminal can perform energy ratio detection processing on at least one light spot based on the pixel intensity parameter to obtain a second interference light spot; the second interference light spot is also the interference light spot detected by the energy ratio detection mode. Schematically, it is usually possible to calculate the energy ratio of two adjacent light spots, and then compare the energy ratios of the two light spots. Usually, if a light spot is an interference light spot, the energy ratio of the actual light spot is much greater than that of the adjacent interference light spot.

[0080] In a feasible implementation manner, the light spot interference detection method may be a pixel mean detection method, and the pixel mean detection method may be to detect the ratio of "the sum of pixel intensity parameters (such as Confidence value) of all pixels in the reflection intensity map covered by a certain light spot" and "the sum of pixel intensity parameters (such as Confidence value) of all pixels within a certain neighborhood of the light spot", so as to use the ratio as the pixel mean;

[0081] Optionally, the pixel mean detection method can be the ratio of "the sum of the spot pixel intensity parameters (such as Confidence value) divided by the average number of pixels covered" and "the spot pixel intensity parameters (such as Confidence value) of all pixels within a certain neighborhood of the spot divided by the average number of pixels covered", so as to use the ratio as the pixel mean; generally, the smaller the ratio, the greater the possibility that the spot is an interference spot.

[0082] It can be understood that after the terminal determines the light spot interference detection method, if the light spot interference detection method is a pixel mean detection method, the terminal performs pixel mean detection processing on at least one light spot based on the pixel intensity parameter, so as to obtain a third interference light spot from multiple light spots; for example, after the pixel mean detection processing, the pixel mean of the light spot is obtained, and then the pixel mean is compared with the mean threshold, so as to determine the interference light spot from multiple light spots, that is, to determine the interference light spot.

[0083] In a feasible implementation, the light spot interference detection method can be a total energy intensity detection method. Usually, in at least one small area corresponding to the reflection intensity map, the number of theoretical light spots in the small area can be set in advance, and the total energy of multiple actual light spots (excluding interference light spots) in the small area should be equivalent to each other. For example, the small area corresponds to a two-dimensional plane of the target object in the visual scene, and the total energy of the generated interference light spots is much smaller than the actual light spots in the small area. Schematically, a small area may not be determined. Usually, there are more light spots in the reflection energy map. After processing, a light spot usually only corresponds to a parameter value of a pixel point in the depth map, that is, it is possible to set any n (n is usually a small value) adjacent light spots for comparison.

[0084] It can be understood that after the terminal determines the light spot interference detection method, if the light spot interference detection method is the total energy intensity detection method, the terminal performs total energy intensity detection processing on at least one light spot based on the pixel intensity parameter to obtain a fourth interference light spot; for example, after the total energy intensity detection processing, the total energy intensity of multiple light spots is obtained, and then the total energy intensity of at least one light spot is compared. When the total energy intensity of a certain light spot is small, the light spot indicated by the smaller total energy intensity can be used as an interference light spot, that is, at least one of all the pixel points covered by the interference light spot can be determined as an interference light spot, for example, the pixel point indicated by the maximum pixel intensity parameter covered by the interference light spot is used as the interference light spot.

[0085] In a feasible implementation manner, the light spot interference detection method may be a light spot position detection method, which may also be understood as fixed position detection. According to some embodiments, when the speckle iTOF camera is in the factory stage, the position of the light spot on the reflection intensity diagram obtained at the light receiving end is usually relatively fixed, but may be offset by several pixels due to objective reasons such as parallax between long and short distances and signal divergence. Based on this, the position range of the light spot may be set in the factory stage, and the position range of at least one light spot may be recorded. If the light spot appears outside these positions, it is likely to be an interference light spot.

[0086] It is understandable that after the terminal determines the light spot interference detection mode, if the light spot interference detection mode is a light spot position detection mode, the terminal can perform light spot position detection processing on at least one light spot based on the pixel position parameter to obtain a fifth interference light spot. For example, when the light spot position detection processing is performed, the light spot positions of multiple light spots are obtained, and the light spot positions of the light spots are matched with their corresponding position ranges, so that the interference light spot can be determined from the multiple light spots.

[0087] In some embodiments, after determining the interference light spot, the terminal performs interference elimination processing on the interference light spot. Schematically, after the terminal determines the interference light spot based on the Confidence information in the reflection intensity map, it can remove at least one pixel point contained in the interference light spot in the reflection intensity map, such as removing the entire interference light spot. Schematically, after determining the interference light spot, the terminal can determine the interference position of the interference light spot in the depth map based on the position of the interference light spot to eliminate the interference light spot in the depth map.

[0088] In a feasible implementation scenario, after the interfering light spot is determined, the interfering light spot may be verified by combining a color image such as RGB and a depth image to determine whether the interfering light spot is a correct interfering point.

[0089] It is understandable that the terminal can obtain a color image and a depth image, and the reflection intensity map, the depth image and the color image are different types of images for the same target object; that is, when the terminal is imaging the target object in the current scene, the terminal can simultaneously capture the reflection intensity map, the depth map and the color image for the same target object, and the color image can generally be an RGB image. After determining the interference spot, the captured color image is then acquired.

[0090] It can be understood that the terminal can determine the first position of the interfering light spot in the depth image, and can determine the second position of at least one adjacent light spot corresponding to the interfering light spot in the depth image; based on the first position and the second position, obtain the depth pixel verification result for the interfering light spot.

[0091] It can be understood that the terminal further determines a third position of the interfering light spot in the color image, and determines a fourth position of at least one adjacent light spot corresponding to the interfering light spot in the color image; based on the third position and the fourth position, a color pixel verification result for the interfering light spot is obtained;

[0092] Schematically, in the reflection intensity image, at least one light spot usually covers multiple light spot pixels, while in the depth image and the color image, it usually corresponds to only one pixel;

[0093] Indicatively, a light spot may cover multiple pixels, but the light spot is only solved to generate a depth value at a certain pixel position of the light spot (such as the pixel at the center of gravity of the light spot). The depth value represents the depth distance value of the small area near the light spot in the depth image; for example: it can be combined with the original information of all pixels covered by the light spot in the original image (RAW image, the original image output by the TOF camera), and the pixel at the center of gravity of the light spot corresponding to the center of gravity position of the light spot is taken to generate an effective depth value. The effective depth value will calculate a depth value based on the original information of all pixels covered by the light spot.

[0094] Illustratively, after determining the interference spot, its position in the depth image, that is, the first position, can be determined. That is, the position of the interference spot corresponding to the effective depth value is taken as the first position (usually the position in the depth image corresponding to the pixel point indicated by the maximum pixel intensity of the interference spot).

[0095] Furthermore, the depth image is usually obtained by taking a TOF camera of the device, and the color image such as the RGB image is usually obtained by taking a RGB camera of the device. By calibrating the RGB camera and the TOF camera, the coordinate transformation relationship between the depth image and the color image can be determined (which may be in the form of an image coordinate transformation formula). After the terminal determines the interference light spot, the interference light spot corresponding to the interference light spot in the depth image is obtained, and the position of the interference light spot in the depth image is used as the first position; then based on the first position in the aforementioned depth image, the third position of the interference light spot in the color image is determined according to the coordinate transformation relationship; similarly, at least one adjacent light spot corresponding to the interference light spot is obtained, and then the adjacent light spot corresponding to at least one adjacent light spot in the depth image can be obtained with reference to the aforementioned method, and the position of the adjacent light spot in the depth image is used as the second position; then based on the second position, the fourth position of the adjacent light spot in the color image is determined according to the coordinate transformation relationship; it can be understood that when the number of adjacent light spots corresponding to the interference light spot is multiple, the second position of at least one adjacent light spot in the depth image and the fourth position in the color image are obtained in the aforementioned manner.

[0096] Optionally, the positional relationship between at least one adjacent light spot and the interfering light spot is an adjacent relationship, and the number of adjacent light spots may be a preset number. A second position of an adjacent pixel point corresponding to the adjacent light spot may be obtained in the depth image, and a fourth position of the adjacent light spot in the color image may be determined based on the second position of the adjacent pixel point according to the image coordinate system transformation relationship;

[0097] In practical applications: color images such as RGB images have high resolution and can provide color information; depth images can provide depth information. If the above steps determine that it is an interference light spot, the interference light spot can be verified. In specific implementations: the first position where the interference light spot corresponds to the interference light spot can be found in the depth image, and the second position where the adjacent light spot corresponds to the adjacent light spot can be found in the depth image, and then the third position where the interference light spot corresponds to the interference light spot can be found on the RGB image, and the fourth position where the adjacent light spot corresponding to the interference light spot is located in the RGB image;

[0098] Schematically, the depth value indicated by the interference light spot at the first position is significantly different from the depth value indicated by other adjacent light spots at the second position, such as the depth difference between the “depth value at the first position” and the “depth value at the second position” is greater than a certain depth difference threshold, and in the color information given by the RGB image, the color information corresponding to the points “the interference light spot and the adjacent light spot” in the RGB image of the interference light spot is similar, such as the color pixel difference (such as the RGB difference) between the “color pixel value (such as the RGB value) at the third position” and the “color pixel value (such as the RGB value) at the third position” is less than a certain color difference threshold (such as the threshold value or critical value set for the RGB value); this indicates that these points may be on the surface of the same object, and the interference light spot is more likely to be correctly judged; otherwise, the interference light spot corresponding to the interference light spot may come from a long distance or other low-reflectivity targets or smooth mirror-reflected targets, that is, it is possible that the interference light spot does not belong to the interference light spot type but may belong to the effective light spot type.

[0099] In a specific implementation, based on the first position and the second position, obtaining a depth pixel verification result for the interference light spot may be: obtaining a first depth value corresponding to the first position and obtaining a second depth value corresponding to the second position, calculating a depth value difference between the first depth value and the second depth value, if the depth value difference is greater than a depth threshold, the depth pixel verification result is a first result, such as True, otherwise it is a second result, such as False;

[0100] Further, based on the third position and the fourth position, a color pixel verification result for the interfering light spot is obtained, which may be: obtaining a first color value (such as an RGB value) corresponding to the third position and obtaining a second color value corresponding to the fourth position, and calculating a color value difference between the first color value and the second color value. If the color value difference is less than a color threshold, the color information of the interfering light spot is usually similar to that of the adjacent light spot, and the color pixel verification result is a first result, such as True; otherwise, the color pixel verification result is a second result, such as False.

[0101] Then, the interference verification result for the interference light spot is determined by combining the depth pixel verification result and the color pixel verification result. In an exemplary manner, if the depth pixel verification result and the color pixel verification result are both the first result, such as True, then the interference verification result for the interference light spot is: the interference verification is passed; otherwise, the interference verification result for the interference light spot is: the interference verification is not passed.

[0102] In an embodiment of the present application, by obtaining a reflection intensity map, determining at least one light spot in the reflection intensity map, and then obtaining pixel characteristic parameters corresponding to the light spot pixels in the light spot, based on the difference in pixel characteristic parameters between the interference light spot and the effective light spot, the interference light spot can be accurately determined from at least one light spot, which can assist in eliminating the light spot interference caused by the superposition of edge signals of multiple light spots; the entire image interference processing process reduces the dependence on device hardware and complex algorithms, realizes accurate identification of light spot interference, can assist in outputting high-precision depth measurement results, and improves the robustness in depth measurement scenarios; and, the interference light spot can be verified in combination with the depth map and the color map, further improving the accuracy of interference light spot identification; the detection process of the interference light spot is optimized by combining different detection methods or different timing detection methods determined based on actual application scenarios.

[0103] See also Figure 5 , Figure 5 This is a flow chart of another embodiment of an image processing method proposed in this application. Specifically:

[0104] S301: Acquire a reflection intensity map, determine at least one light spot in the reflection intensity map, and acquire pixel characteristic parameters corresponding to light spot pixels in the light spot;

[0105] S302: Determine at least one light spot interference detection method for the reflection intensity map.

[0106] For details, please refer to the method steps involved in one or more embodiments of the present application, which will not be repeated here.

[0107] S303: If the light spot interference detection method is a pixel intensity detection method, determining a first intensity priority corresponding to the light spot pixel based on the pixel intensity parameter, and obtaining a target light spot ratio for a reflection intensity map;

[0108] According to some embodiments, the light spot interference detection method may be a pixel intensity detection method, which detects the pixel intensity of the light spot pixel to distinguish the interference light spot from the identified multiple light spots. In actual application scenarios, for example, the reflection intensity map under the current field of view is obtained through the light receiving end of the speckle iTOF camera, wherein the proportion of the number of pixels covered by all actual light spots in the reflection intensity map to the number of pixels in the entire reflection intensity map is usually small, for example, about a few percent to more than ten percent. In areas not covered by actual light spots, the value of the pixel point or the pixel intensity value (that is, the Confidence value) in the reflection intensity map or depth map is not 0, which is caused by the presence of noise such as ambient light, circuits, and dark circuits. However, the pixel intensity values ​​of the areas not covered by these actual light spots (which can also be understood as effective light spots other than interference light spots) (that is, the Confidence values ​​of the pixels in the areas not covered) are usually very low (even if there are interference light spots, the pixel intensity values ​​in the interference light spots are usually much lower than the pixel intensity values ​​in the actual light spots). In some scenarios, the pixel intensity values ​​of the interference light spots or edge areas are one to several orders of magnitude lower than the pixel intensity values ​​of the actual light spot signal light (that is, the Confidence values). Based on this, the light spot intensity parameters corresponding to the light spot pixels covered by all the light spots identified in the reflection intensity map (such as the Confidence values ​​corresponding to the light spot pixels) can be detected to determine the interference light spots from the multiple identified light spots, that is, to determine the interference light spots.

[0109] It can be understood that after the terminal determines the light spot interference detection method, if the light spot interference detection method is a pixel intensity detection method, the terminal can perform pixel intensity detection processing on at least one light spot based on the pixel intensity parameter. In a specific implementation, it can be based on the pixel intensity parameters of the light spot pixels covered by the light spot in the reflection intensity map (also referred to as the Confidence map) (such as the Confidence value of the light spot pixels covered by the light spot), and prioritized according to the numerical values ​​of the pixel intensity parameters of all the light spot covered pixels (the number of light spot pixels covered by the light spot can be multiple), determine the first intensity priority corresponding to at least one light spot pixel (that is, the light spot pixel covered by the light spot), it can be understood that the pixel intensity parameter is positively correlated with the first intensity priority, and the target light spot ratio for the reflection intensity map is obtained.

[0110] Optionally, determining the first intensity priority corresponding to at least one light spot pixel (that is, the pixel covered by the light spot) can be determined by drawing an intensity statistical graph, such as the intensity statistical graph can be an intensity histogram that characterizes the pixel intensity parameters of the light spot pixel (the Confidence value corresponding to the light spot pixel), and the intensity histogram can be used to characterize the pixel intensity of the light spot pixel points covered by all light spots. For example, light spot pixels with the same pixel intensity parameters (such as the same Confidence value) can be fed back, for example, the intensity priority of at least one light spot pixel point can be fed back.

[0111] It is understandable that in the terminal use stage after leaving the factory, the terminal works based on a certain speckle iTOF mode, and the theoretical number of light spots received by the light receiving end in a certain speckle iTOF mode, the theoretical number of light spot pixels covered by the light spots, etc. have been determined. For example, after the actual application stage of the speckle iTOF camera, how many light spots will there be in the field of view theoretically, and how many light spot pixels will at least one light spot theoretically cover; further, since the theoretical number of light spot pixels covered by the light spot (which may also be referred to as the number of reference pixels in some embodiments) can be determined, or the target light spot ratio of the theoretical light spot pixels (excluding interference points) (the ratio of theoretical light spot pixels to the total number of image pixels) can be determined. Based on this, when detecting the pixel intensity, it is only necessary to determine the reference light spot (which may be referred to as the first light spot or the second light spot in some embodiments) from all the light spot pixels, and use the pixel intensity parameter corresponding to the reference light spot as a threshold for interference screening.

[0112] Schematically, histogram statistics are the statistics of the histogram of the entire Confidence image. According to prior knowledge, after the speckle iTOF camera is designed, the number of spots in the field of view and the number of pixels covered by at least one spot can be theoretically derived.

[0113] S304: Determine an intensity threshold based on the target light spot ratio, the first intensity priority of the light spot pixels, and the pixel intensity parameters of the light spot pixels; and determine a first interference light spot from at least one of the light spots based on the intensity threshold.

[0114] In a feasible implementation manner, a first light spot can be determined based on the target light spot ratio and the first intensity priority of the light spot pixel point, and a first intensity threshold corresponding to the first light spot can be obtained, and a first interference light spot can be determined from at least one of the light spots based on the first intensity threshold;

[0115] Illustratively, assuming that the target light spot proportion (that is, the proportion of the number of light spot pixels) is x%, the first intensity priority of the light spot pixels can be represented in the form of a histogram corresponding to a Confidence graph. The terminal can determine the first x% number of points in the histogram from high to low to determine the first light spot (the x% number of points can be used as the first light spot), and the pixel intensity parameter of the first light spot is used as the first intensity threshold. For example, the Confidence value corresponding to the first light spot is Cx, that is, theoretically, the Confidence values ​​of all light spot pixels covered by the light spots should be greater than Cx. That is, Cx can be used as a threshold to filter out interfering light spots, that is, the Confidence values ​​of all light spot pixels can be compared with Cx. If the Confidence value of a light spot pixel is greater than Cx, the light spot pixel is a non-interfering light spot. If the Confidence value of a light spot pixel is less than or equal to Cx, the light spot pixel can be used as an interfering light spot, or the light spot pixel can be used as a suspected interfering light spot, and then further determined in combination with other light spot interference detection methods. For example, other light spot interference methods are used to determine that the light spot pixel meets the interference point judgment condition, and then the light spot pixel is used as an interfering light spot. After the interfering light spots are screened out, the first interfering light spot is determined. For example, the light spot corresponding to the interfering light spot can be used as the first interfering light spot. In actual light spot interference detection, most of the light spot pixels covered by the first interfering light spot may be interfering light spots.

[0116] In a feasible implementation manner, the number of reference pixels corresponding to the light spot is obtained, the number of reference pixels is the theoretical number of light spot pixels covered by the light spot, a target ratio corresponding to the target light spot proportion and the number of reference pixels is determined, a second light spot is determined based on the target ratio and the first intensity priority of the light spot pixels, and a second intensity threshold corresponding to the second light spot is obtained, and a first interference light spot is determined from at least one of the light spots based on the second intensity threshold;

[0117] Indicatively, in actual application scenarios, the Confidence value of the pixel point in the intersection area of ​​the spot edge of the actual spot (which can be understood as the pixel intensity of the pixel point) is usually greater than the Confidence value of the pixel point such as ambient light noise or circuit noise; and in general scenarios, there may be distant or low-reflectivity objects beyond the design range. At this time, the signal light reflected back by these distant objects presents the spot energy on the light receiving end Sensor very weak, that is, the value of the pixel intensity parameter is low, which may be lower than the Confidence value of the pixel point in the interference spot at the intersection of the close-range spot edge. In other words: the number of effective spots on the current reflection intensity map (which can be understood as the number of actual spots other than the interference spot) may be lower than the set threshold, such as lower than the number of pixels corresponding to the target spot ratio. In order to improve the recognition accuracy of spot interference. The number of reference pixels corresponding to the spot can be used to detect interference spots.

[0118] The number of reference pixels corresponding to the light spot can be understood as the number of light spot pixels theoretically covered by at least one light spot. It can be understood that, for example, in the terminal use stage after leaving the factory, the terminal works based on a certain speckle iTOF mode, and the theoretical number of light spots theoretically received by the light receiving end in a certain speckle iTOF mode, the target light spot ratio, the theoretical number of pixels in the area covered by at least one light spot (that is, the number of reference pixels), etc. have been determined or can be calculated.

[0119] It is understandable that the second intensity threshold may be determined based on a combination of the number of reference pixels corresponding to at least one light spot and the target light spot ratio, so as to improve the recognition accuracy of the light spot interference.

[0120] Schematically, if a light spot covers n light spot pixels (that is, the number of reference pixels is n), taking the aforementioned histogram as an example, the first intensity priority of the light spot pixel can be obtained in the form of a histogram corresponding to the Confidence graph, then the terminal can determine the first (x / n)% number of points from the top to the bottom of the histogram to determine the second light spot (the (x / n)% number of points can be used as the second light spot), and the pixel intensity parameter of the second light spot is used as the second intensity threshold, such as the Confidence value corresponding to the second light spot as the second intensity threshold Cxna. Further, theoretically, the Confidence value of all light spot-covered pixels should be greater than Cxna. That is, Cxna can be used as a threshold to filter out interfering light spots. If the Confidence value of the light spot pixel is greater than Cxna, the light spot pixel is not an interfering light spot. If the Confidence value of the light spot pixel is less than or equal to Cxna, the light spot pixel can be used as an interfering light spot. The first interfering light spot can be determined based on the light spot corresponding to the interfering light spot.

[0121] In a feasible implementation, the first intensity threshold and the second intensity threshold can also be combined to achieve a balance, so as to avoid errors in the light spot interference judgment, achieve accuracy in the interference judgment, and avoid missing the interfering light spot. Indicatively, from the first intensity threshold and the second intensity threshold, that is, from Cx to Cxna, the Confidence threshold increases from small to large, corresponding to fewer and fewer interference points. The first intensity threshold and the second intensity threshold can be combined with the actual application scenario to determine a suitable threshold for the light spot interference judgment.

[0122] It can be understood that the terminal can determine the first light spot based on the target light spot proportion and the first intensity priority of the light spot pixel and obtain the first intensity threshold corresponding to the first light spot; determine the first light spot based on the target light spot proportion and the first intensity priority of the light spot pixel and obtain the first intensity threshold corresponding to the first light spot, obtain the number of reference pixels corresponding to the light spot, determine the target ratio corresponding to the target light spot proportion and the number of reference pixels, determine the second light spot based on the target ratio and the first intensity priority of the light spot pixel and obtain the second intensity threshold corresponding to the second light spot, and determine the first interference light spot from at least one of the light spots based on the first intensity threshold and the second intensity threshold.

[0123] It can be understood that from the first intensity threshold and the second intensity threshold, that is, from Cx to Cxna, the Confidence threshold increases from small to large, corresponding to fewer and fewer interference points, and a threshold can be determined by combining the first intensity threshold and the second intensity threshold to filter out interference; in a specific implementation, the terminal determines a threshold reference range based on the first intensity threshold and the second intensity threshold, such as the threshold reference range is: [Cx, Cxna], and then obtains the target intensity threshold from the threshold reference range, and determines the first interference light spot from at least one of the light spots based on the target intensity threshold.

[0124] Optionally, the target intensity threshold may be a value randomly selected from the threshold reference range as the target intensity threshold.

[0125] Optionally, a threshold may be determined based on the first intensity threshold and the second intensity threshold in combination with the current actual application scenario to perform interference filtering.

[0126] Schematically, in different application scenarios, the requirements for measurement accuracy, interference adaptability, etc. are different, and an adjustment factor can be set. The reference adjustment factor is set for each of the multiple reference application scenarios. The reference factor is used to determine the target intensity threshold from the threshold reference range. In the actual application stage, an adjustment factor mapping relationship between multiple reference application scenarios and their corresponding reference adjustment factors is established. After determining the current application scenario, the adjustment factor corresponding to the application scenario can be determined based on the adjustment factor mapping relationship. Based on the adjustment factor, a preset calculation formula is used to determine the target intensity threshold from the threshold reference range. For example, the threshold reference range is: [Cx, Cxna]. Assuming the adjustment factor is a, the target intensity threshold b = (Cxna-Cx)*a+Cx;

[0127] S305: If the light spot interference detection method is an energy ratio detection method, obtaining a center point intensity value and an edge point intensity value corresponding to at least one of the light spots;

[0128] According to some embodiments, the energy ratio detection method detects the energy ratio of the pixels contained in the light spot to distinguish the interference light spot from the identified multiple light spots. Schematically, the energy ratio detection method can traverse at least one light spot, calculate the energy ratio of the pixel intensity parameter (that is, the Confidence value) of the center pixel of the light spot and the Confidence value of other pixels of the light spot except the center pixel of the light spot. Usually, the signal light energy of the actual light spot is concentrated, and its energy ratio will be greater than that of the interference light spot.

[0129] The center point intensity value can be understood as the pixel intensity value of the center pixel of the light spot (equivalent to the Confidence value). In some embodiments, the centroid pixel of the light spot can be used as the center pixel of the light spot, and the pixel intensity value of the centroid pixel of the light spot can be used as the center point intensity value. The edge point intensity value can be understood as the pixel intensity value of the edge pixel in the light spot except the center pixel (that is, the Confidence value). Both the center point intensity value and the edge point intensity value can be directly obtained based on the reflection intensity map.

[0130] In schematic form, assuming that the light spot covers m light spot pixels, theoretically, the intensity of the light spot pixels gradually decays from the center to the edge of the light spot, that is, the corresponding Confidence value gradually decays, and the decay law basically conforms to the Gaussian surface. The Confidence value of the light spot pixels in a row or column of the area covered by the light spot conforms to the Gaussian curve. Figure 6 As shown, Figure 6 is a schematic diagram of a scene involving light spots. Figure 6 The relationship between the confidence value of a certain column of light spot pixels and the position of the light spot pixels is as follows: Figure 6 As shown, in Figure 6 The light spots on the two sides can be regarded as actual light spots or effective signal light spots. The light spot in the middle is generated by the superposition of the edge signal lights of the light spots on both sides. The effective signal light spot is as follows: Figure 6 As shown in (a), the energy intensity of the effective signal spot is concentrated, the confidence value of the central pixel is high, and the energy of the spots around the effective signal spot decays rapidly. In other words, the ratio of the confidence value of the central pixel of the effective spot to the confidence value of the spots around is large. Figure 6 As shown in (b), the distribution of the interference signal spot also presents a Gaussian surface, but the energy of the interference signal spot is dispersed, and the ratio of the confidence value of the central pixel point of the interference signal spot to the confidence value of the edge is much smaller than the ratio corresponding to the two adjacent effective spots.

[0131] S306: Perform energy ratio detection processing on at least one light spot based on the center point intensity value and the edge point intensity value, and determine a second interference light spot from the at least one light spot.

[0132] According to some embodiments, the terminal may perform energy ratio detection processing on at least one light spot based on the pixel intensity parameter to obtain a second interference light spot; the second interference light spot is also the interference light spot detected by the energy ratio detection method. Schematically, the energy ratio of two adjacent light spots is usually calculated, and then the energy ratio of the two light spots is compared. Usually, if a light spot is an interference light spot, the energy ratio of the actual light spot is much greater than that of the adjacent interference light spot.

[0133] It can be understood that, 1. the terminal may determine the first intensity ratio of the light spot based on the center point intensity value and the edge point intensity value;

[0134] The first intensity ratio is the ratio of the intensity value at the center of the light spot to the intensity value at the edge of the light spot.

[0135] Illustratively, the edge distance can be set, with the center point of the light spot as a reference, and the edge points of the same light spot indicated by the edge distance are taken. For example, the edge distance can be 2 pixels, and the pixel points 2 pixels apart from the center point of the light spot are taken as edge points.

[0136] 2. The terminal obtains first light spots corresponding to the light spots respectively, and determines a second intensity ratio of the first light spots;

[0137] Indicatively, it can be to obtain a neighboring light spot adjacent to the light spot, and use the neighboring light spot as the first light spot corresponding to the light spot; considering that the ratio of the neighboring light spots is comparable. Adjacent can be understood as the light spot and its corresponding first light spot are very close. In a real scene, the two light spots may come from adjacent positions on the surface of the same object, that is, the energy of the two light spots and the calculated distance are similar, that is, the two light spots are comparable. If the light spot is far away from the first light spot corresponding to it, it is possible that one comes from the surface of a close object and the other comes from a distant background. The reflected light of the distant background itself has a long optical path, and the light spot hitting the sensor at the light receiving end of the TOF camera may have weak energy or unconcentrated energy. Therefore, the comparability between the light spots at a longer distance is poor. Based on this, the neighboring light spot adjacent to the light spot can be obtained, and the neighboring light spot can be used as the first light spot corresponding to the light spot.

[0138] Illustratively, a fixed distance may be set, and a light spot within the fixed distance from at least one of the light spots is acquired as the first light spot.

[0139] In an illustrative manner, a distance threshold may be set, and the terminal may obtain the target distance between the light spot and at least one second light spot (the second light spot is a light spot other than the current light spot), and determine the first light spot corresponding to the light spot from the at least one second light spot based on the target distance and the distance threshold, that is, take the first light spot corresponding to the target distance between the second light spot and the light spot when it is less than or equal to the distance threshold.

[0140] It can be understood that the second intensity ratio is determined in a similar manner to the first intensity ratio, that is, the ratio of the intensity value of the center point of the first light spot to the intensity value of the edge point. Schematically, the "selection of the center point and the edge point" of the first light spot should be consistent with the "selection of the center point and the edge point" of the light spot, such as selecting the edge point corresponding to the center point of the light spot and the edge point corresponding to the center point of the first light spot based on the same edge distance.

[0141] 3. The terminal determines a second interfering light spot from at least one of the light spots based on the first intensity ratio and the second intensity ratio of the light spots.

[0142] In a feasible implementation manner, a target difference may be determined based on the first intensity ratio and the second intensity ratio; a second interfering light spot is determined from at least one of the light spots based on the target difference and a difference threshold;

[0143] Illustratively, the target difference is the difference between the first intensity ratio and the second intensity ratio. The difference threshold is a threshold value for the target difference determined based on the pre-set hardware requirements and application scenarios. If the target difference is greater than the difference threshold, it can be determined that the first light spot is an interference light spot. If the target difference is less than or equal to the difference threshold, it can be generally determined that the first light spot is not an interference light spot.

[0144] In a feasible implementation, the first intensity ratio may be used as a reference, and an intensity ratio fluctuation range for the first light spot may be determined based on at least one of the second intensity ratios; and a second interfering light spot may be determined from at least one of the light spots based on the intensity ratio fluctuation range and the reference fluctuation range.

[0145] For example, there are k×k adjacent first light spots in a small area including the current light spot. The fluctuation range of the ratio of the central pixel Confidence value to the edge Confidence value of these k×k first light spots (that is, the intensity ratio fluctuation range corresponding to the multiple second intensity ratios and the current first intensity ratio) should not exceed the reference fluctuation range. The reference fluctuation range can be set in advance. If it deviates from the reference fluctuation range, the light spot is identified as a second interference light spot.

[0146] In one or more embodiments, the first intensity ratio of the light spot can be determined based on the center point intensity value and the edge point intensity value; illustratively, a threshold value for the first intensity ratio, that is, an intensity ratio threshold, is determined based on predetermined hardware requirements and application scenarios; and then based on the first intensity ratio and the intensity ratio threshold, a second interfering light spot is determined from at least one of the light spots. For example, if the first intensity ratio is greater than the intensity ratio threshold, the light spot is a valid light spot; if the first intensity ratio is less than or equal to the intensity ratio threshold, the light spot is an interfering light spot.

[0147] S307: If the light spot interference detection method is a pixel mean detection method, determining a target pixel region corresponding to at least one of the light spots, wherein the light spot is located in the target pixel region;

[0148] According to some embodiments, the light spot interference detection method may be a pixel mean detection method, and the pixel mean detection method may be to detect the ratio of "the sum of pixel intensity parameters (such as Confidence value) of all light spot pixels in the reflection intensity map covered by a certain light spot" and "the sum of pixel intensity parameters (such as Confidence value) of all regional pixels within a certain neighborhood of the light spot", so as to use the ratio as the pixel mean;

[0149] Among them, the certain neighborhood range of the light spot is the aforementioned target pixel area, and the target pixel area can be a rectangular area, which contains one or more pixels covered by the light spot. The target pixel area can be a circular area. The target pixel area can be all pixel points that are greater than a certain proportion of the maximum intensity in a certain rectangular area. For example, the maximum Confidence value in a certain rectangular area is 100, and a certain proportion is set to a (such as a is 40%), then it is determined that all pixel points in the certain rectangular area whose Confidence value is greater than (a*100) meet the requirements.

[0150] S308: Determine a spot pixel point average for at least one of the light spots based on the pixel intensity parameter, and determine a regional pixel point average corresponding to at least one of the target pixel regions;

[0151] Illustratively, a first total intensity corresponding to all the light spot pixels and a total number of light spot pixels of at least one of the light spots may be obtained, and a quotient of the first total intensity and the total number of light spot pixels is taken as a mean value of the light spot pixels of the light spot;

[0152] The first total intensity is the sum of the pixel intensity parameters (Confidenc values) of all the light spot pixels covered by a light spot. The total number of light spot pixels is the total number of all the light spot pixels of the light spot. Assuming that light spot a covers 6 light spot pixels, the first total intensity is the sum of the Confidenc values ​​of the 6 light spot pixels, and the total number of light spot pixels is 6.

[0153] Illustratively, the second total intensity and the total number of regional pixels corresponding to all regional pixels of at least one of the target pixel areas can be obtained, and the quotient of the second total intensity and the total number of regional pixels can be used as the mean value of the regional pixels corresponding to the target pixel area.

[0154] The second total intensity is the sum of the pixel intensity parameters of all regional pixels in the target pixel area. The total number of regional pixels is the number of all light spot pixels in the area. Assuming that the target pixel area b covers 8 regional pixels, the second total intensity is the sum of the Confidenc values ​​of the 8 regional pixels, and the total number of regional pixels is 8.

[0155] S309: Performing pixel point mean value detection processing on at least one light spot based on the pixel point mean value of the light spot and the pixel point mean value of the region, and determining a third interference light spot from the at least one light spot.

[0156] Optionally, the pixel mean detection method can be: for a certain light spot, calculate "the sum of the light spot pixel intensity parameters (such as Confidence value) (i.e., the first total intensity) divided by the average value C1 of the number of pixels covered (i.e., the total number of light spot pixels)", and the average value C1 can be used as the light spot pixel mean; and calculate "the pixel intensity parameters of all pixels in the target pixel area (i.e., the second total intensity) divided by the average value C2 of the number of pixels covered (i.e., the total number of regional pixels)", and the average value C2 can be used as the regional pixel mean; the ratio of the average value C1 to the average value C2 (i.e., C1 / C2), generally, the smaller the ratio is, the greater the possibility that the certain light spot is an interference light spot. Among them, the ratio (i.e., C1 / C2) can be called a reference ratio.

[0157] It can be understood that if the light spot interference detection method is a pixel mean detection method, the terminal performs pixel mean detection processing on at least one light spot based on the pixel intensity parameter, so as to obtain a third interference light spot from multiple light spots; for example, after the pixel mean detection processing, a ratio (that is, C1 / C2) for at least one light spot is obtained, and then the ratio (that is, C1 / C2) is compared with a certain ratio threshold. If the ratio (that is, C1 / C2) is less than a certain ratio threshold, the light spot is used as an interference light spot, otherwise, the light spot is usually a valid light spot; in the case where there are multiple light spots, the multiple light spots can be compared in turn with reference to the aforementioned method, so as to determine the interference light spot from the multiple light spots, and the interference light spot can be determined based on the interference light spot.

[0158] The ratio threshold is a threshold value or critical value set for the reference ratio (ie, C1 / C2).

[0159] Illustratively, a reference ratio corresponding to the mean pixel value of the light spot and the mean pixel value of the area is determined; the terminal can perform pixel mean detection processing on at least one light spot based on the reference ratio and the ratio threshold, and determine a third interfering light spot from at least one of the light spots.

[0160] It is understandable that the basis of the pixel average detection method is that the effective light spot light energy is very strong and concentrated, and the energy and confidence decay rapidly from the center of the light spot to the edge. For the effective light spot, the ratio of C1 / C2, that is, the reference ratio, is much larger than the ratio corresponding to the interference point. This is because the center of the light spot of the interference point itself is the superposition of at least two effective light spot edges, and the energy is very weak. In schematic form, the judgment of the interference light spot can be achieved by setting a threshold value for the reference ratio.

[0161] S310: If the light spot interference detection method is the total energy intensity detection method, then based on the pixel intensity parameter, determine the light spot intensity of the third light spot in the reference area and the average light spot intensity corresponding to all the fourth light spots;

[0162] According to some embodiments, the light spot interference detection method can be a total energy intensity detection method. Usually, in at least one reference area corresponding to the reflection intensity map (the reference area can usually be a small area with a preset size specification), the theoretical number of light spots in the reference area can be set in advance or calculated and derived, and the total energy of the light spots (also understood as the light spot energy) between multiple actual light spots (excluding interference light spots) in the reference area should be equivalent. For example, the reference area corresponds to a two-dimensional plane of the target object in the visual scene, and the light spot energy of the interference light spot generated at this time is usually much smaller than the light spot energy of the actual light spot in the reference area.

[0163] The spot energy can be understood as the sum of pixel intensity parameters of all spot pixels covered by the spot, that is, the sum of Confidence values ​​of all spot pixels covered by the spot. In some embodiments, the spot energy can also be called spot intensity or spot energy intensity.

[0164] The reference area can generally be understood as a small area range of a preset size specification; schematically: the reference area can be an area of ​​r×c pixels, which can be understood as r×c pixels being the preset size specification.

[0165] In a specific implementation scenario, it is assumed that the reference area should theoretically include 2×2 light spots.

[0166] A feasible implementation may be: the third light spot may be any light spot among all the light spots in the reference area; the third light spot may also be the light spot with the smallest light spot intensity among all the light spots in the reference area;

[0167] Optionally, the relationship between the third light spot and the fourth light spot may be: all the light spots included in the reference area are fourth light spots, and the third light spot is one of all the fourth light spots;

[0168] For example, assuming that there are 4 light spots in the reference area: spot A1, spot A2, spot A3, and spot A4; the fourth light spot is also spot A1, spot A2, spot A3, and spot A4; the third light spot is also one of “spot A1, spot A2, spot A3, and spot A4”, for example, the third light spot can be the spot with the smallest spot intensity among “spot A1, spot A2, spot A3, and spot A4”, assuming that: the spot intensity of spot A1 is: 45, the spot intensity of spot A2 is: 99, the spot intensity of spot A3 is: 100, and the spot intensity of spot A4 is: 101, then spot A1 in the reference area can be used as the third light spot.

[0169] Optionally, the relationship between the third light spot and the fourth light spot may be: the reference area includes the third light spot and all the fourth light spots

[0170] For example, assuming that there are 4 light spots in the reference area: spot B1, spot B2, spot B3, and spot B4; the third light spot may be one of “spot B1, spot B2, spot B3, and spot B4”; the fourth light spot is the light spot other than the third light spot in “spot B1, spot B2, spot B3, and spot B4”: for example, the third light spot may be spot B3 in “spot B1, spot B2, spot B3, and spot B4”, then the fourth light spot is: “spot B1, spot B2, and spot B4” other than spot B3; schematically, assuming that: the spot intensity of spot B1 is: 45, the spot intensity of spot B2 is: 99, the spot intensity of spot B3 is: 100, and the spot intensity of spot B4 is: 101, then the spot B1 with the smallest spot intensity in the reference area can be taken as the third light spot.

[0171] The spot intensity is the sum of the pixel intensity parameters (Confidence values) of all the spot pixels covered by a spot. For example, if there are three spot pixels, and the Confidence values ​​of at least one spot pixel are spot pixel 1: x1, spot pixel 2: x2, and spot pixel 1: x3, respectively, the spot intensity is: (x 1+ x 2+ x3) / 3.

[0172] The “average spot intensity” in the average spot intensity corresponding to all fourth light spots may be understood as: the ratio of the sum of the spot intensities of all fourth light spots (ie, the sum of the Confidence values ​​of all fourth light spots) to the number of fourth light spots.

[0173] The average light spot intensity corresponding to all fourth light spots is the ratio of the sum of the light spot intensities of all fourth light spots to the total number of fourth light spots. For example, there are 4 fourth light spots, namely, light spot A1, light spot A2, light spot A3, and light spot A4. Assume that the light spot intensity of light spot A1 is 45, the light spot intensity of light spot A2 is 99, the light spot intensity of light spot A3 is 100, and the light spot intensity of light spot A4 is 101; then the total number of fourth light spots is 4, and the average light spot intensity corresponding to all fourth light spots is: (45+99+100+101) / 4.

[0174] In a feasible implementation manner, the total energy intensity detection method may also be a non-maximum suppression detection method. According to some embodiments, such as the speckle iTOF camera in the actual application stage, how many effective light spots will theoretically be in a certain reference area in the field of view, how many pixels will theoretically be covered by at least one effective light spot, and the threshold value of the number of light spots corresponding to the theory is usually determined; non-maximum suppression detection may be performed based on the actual number of light spots in the reference area: when the total number of light spots corresponding to all light spots in the reference area is less than or equal to the threshold value of the number of light spots corresponding to the reference area (in some implementation manners, the threshold value of the number of light spots may be determined based on the theoretical number of light spots), there is no need to perform total energy intensity detection, and at this time, the actual light spots in the reference area are usually all effective light spots; when the actual number of light spots is greater than the theoretical number of light spots corresponding to the reference area, it can usually be considered that there are interfering light spots.

[0175] Illustratively, the reference area may be an area of ​​r×c pixels, which may be understood as a preset size specification. Assume that the reference area should theoretically contain 2×2 light spots, that is, the threshold value of the number of light spots in the reference area is: 4, and in practice, the number of light spots detected in the reference area in the reflection intensity map is: 5, that is, the total number of light spots corresponding to the reference area is 5; at this time: the total number of light spots is greater than the threshold value of the number of light spots, then the total energy intensity detection is performed, that is, executing S310 "based on the pixel intensity parameter, determine the spot intensity of the third light spot in the reference area and the average of the spot intensities corresponding to all fourth light spots" and S311.

[0176] Optionally, the light spot with the smallest light spot intensity among all the light spots in the reference area may be used as the interference light spot.

[0177] Optionally, the spot number threshold may be determined based on the theoretical spot number, such as the spot number threshold may be equal to the theoretical spot number; the spot number threshold may also be slightly larger than the theoretical spot number to take into account the spot error in actual scenes and improve robustness.

[0178] In a feasible implementation manner, the number of effective light spots that should theoretically appear in the reference area has been predetermined, and therefore, the spacing between at least one theoretical light spot or the range of spacing variation can also be determined, that is, non-maximum suppression detection can be performed based on the light spot spacing distance between two light spots in the reference area for further judgment, by setting a threshold value for the light spot spacing distance, that is, the light spot spacing distance threshold, after obtaining the light spot spacing distance between two light spots in the reference area, if the light spot spacing distance is less than the spacing distance threshold, then interfering light spots may appear in the field of view reference area, and "determine the light spot intensity of the third light spot in the reference area and the average light spot intensity corresponding to all fourth light spots based on the pixel intensity parameter" and S311 in S310 can be executed.

[0179] Optionally, if the light spot spacing distance is greater than or equal to the spacing distance threshold, it can be ignored.

[0180] Optionally, the positional relationship between the two light spots may be an adjacent positional relationship.

[0181] Indicatively, if the spot spacing distance between two adjacent spots of three spots is less than the spot spacing distance threshold, and the spot intensity of the middle spot is less than the average of the spot intensities of the spots on both sides, or the spot intensity of the middle spot is less than the spot intensity of the spots on both sides, then the energy intensity of the middle spot is weak, and the middle spot can be used as an interference spot, such as Figure 7 As shown, Figure 7 is a schematic diagram of a scene involving interfering light spots. Figure 7 The interval between two of the three adjacent light spots is less than the interval distance threshold, and the spot intensity of the middle light spot is less than the spot intensity of the light spots on both sides, that is, the energy intensity of the middle light spot is weaker. The middle light spot can be used as an interference light spot, and at least one of all the pixel points covered by the interference light spot can be used as an interference light spot. For example, the point indicated by the maximum value of the pixel intensity parameter (Confidence value) among all the pixel points covered by the interference light spot is used as the interference point.

[0182] S311: Performing total energy intensity detection processing on the third light spot based on the light spot intensity of the third light spot and the light spot intensity average to determine a fourth interference light spot.

[0183] In a feasible implementation manner, an intensity difference between the spot intensity of the third light spot and the average of the spot intensities, that is, a difference between the spot intensity and the average of the spot intensities, is determined; and a first difference threshold for the intensity difference is set to achieve the judgment;

[0184] If the intensity difference is less than or equal to the first difference threshold, the third light spot whose intensity difference is less than or equal to the first difference threshold is used as the fourth interfering light spot.

[0185] If the intensity difference is greater than the first difference threshold, it is ignored.

[0186] In a feasible implementation manner, determining an intensity ratio of the spot intensity of the third light spot to the average of the spot intensities, that is, a ratio of the spot intensity to the average of the spot intensities;

[0187] If the intensity ratio is less than or equal to a first ratio threshold, the third light spot is used as the fourth interfering light spot, wherein the first ratio threshold is a threshold value set for the intensity ratio.

[0188] If the intensity ratio is greater than the first ratio threshold, then an ignoring process is performed.

[0189] In a specific implementation scenario, it can be understood that if the reference area includes multiple light spots, the light spot intensities corresponding to the multiple light spots can be obtained, and the third light spot indicated by the minimum light spot intensity is selected based on the light spot intensities of the multiple light spots, and then all the fourth light spots in the reference area are determined. One way is: all the light spots in the reference area are used as the fourth light spots; another way is: all the light spots in the reference area except the third light spot are used as the fourth light spots; then the average light spot intensity corresponding to all the fourth light spots is calculated, and then the quotient of the light spot intensity of the third light spot and the average light spot intensity is calculated as the intensity ratio. If the intensity ratio is less than the first ratio threshold, the third light spot is used as the fourth interference light spot.

[0190] S312: If the light spot interference detection method is a light spot position detection method, determine the light spot position for the at least one light spot based on the pixel position parameter; and obtain a reference position range corresponding to the at least one light spot;

[0191] It can be understood that the light spot interference detection method can be a light spot position detection method, and can also be understood as a fixed position detection. According to some embodiments, when the speckle iTOF camera is in the factory stage, the position of the light spot on the reflection intensity diagram obtained at the light receiving end is usually relatively fixed, but it may be offset by several pixels due to objective reasons such as far-distance parallax and signal divergence. Based on this, the position range of the light spot can be set in the factory stage, and the position range of at least one light spot can be recorded. If the light spot appears outside these positions, it is likely to be an interference spot.

[0192] S313: If the spot position of the light spot does not match the reference position range, determining the light spot as a fifth interference light spot.

[0193] It is understandable that after the terminal determines the light spot interference detection mode, if the light spot interference detection mode is the light spot position detection mode, the terminal can perform light spot position detection processing on at least one light spot based on the pixel position parameter to obtain the fifth interference light spot. For example, during the light spot position detection processing, the light spot positions of multiple light spots are obtained, and the light spot positions of the light spots are matched with the position ranges corresponding to the light spots to detect whether the light spot positions fall within the reference position range, and the light spot is determined as the fifth interference light spot; and by analogy, the fifth interference light spot can be determined from the multiple light spots.

[0194] In an embodiment of the present application, by obtaining a reflection intensity map, determining at least one light spot in the reflection intensity map, and then obtaining pixel characteristic parameters corresponding to the light spot pixels in the light spot, based on the difference in pixel characteristic parameters between the interference light spot and the effective light spot, the interference light spot can be accurately determined from at least one light spot, which can assist in eliminating the light spot interference caused by the superposition of edge signals of multiple light spots; the entire image interference processing process reduces the dependence on device hardware and complex algorithms, realizes accurate identification of light spot interference, can assist in outputting high-precision depth measurement results, and improves the robustness in depth measurement scenarios; and, different light spot interference detection methods or light spot interference detection methods with different time sequences determined based on actual application scenarios can be combined to optimize the detection process of interference light spots.

[0195] The following will be combined Figure 8 , the image processing device provided in the embodiment of the present application is introduced in detail. It should be noted that, Figure 8 The image processing device shown is used to execute the present application Figure 1 to Figure 6 For the convenience of explanation, only the part related to the embodiment of the present application is shown. For the specific technical details not disclosed, please refer to the present application. Figure 1 to Figure 7 The embodiment shown.

[0196] See also Figure 8 , which shows a schematic diagram of the structure of the image processing device of an embodiment of the present application. The image processing device 1 can be implemented as all or part of the user terminal through software, hardware or a combination of both. According to some embodiments, the image processing device 1 includes a light spot determination module 11, a parameter acquisition module 12 and an interference determination module 13, which are specifically used to:

[0197] A light spot determination module 11 is used to obtain a reflection intensity map and determine at least one light spot in the reflection intensity map;

[0198] A parameter acquisition module 12 is used to acquire pixel characteristic parameters corresponding to the light spot pixels in the light spot;

[0199] The interference determination module 13 is configured to determine an interference light spot from the at least one light spot based on the pixel characteristic parameter.

[0200] Optional, such as Fig. 9 As shown, the interference determination module 13 includes:

[0201] A detection and determination unit 131, used to determine at least one light spot interference detection method for the reflection intensity map;

[0202] The interference detection unit 132 is used to perform light spot interference detection processing on at least one of the light spots based on the pixel characteristic parameters and adopt the light spot interference detection method to obtain an interference light spot.

[0203] Optionally, the interference detection unit 132 is specifically configured to:

[0204] If the light spot interference detection method is a pixel point intensity detection method, a pixel point intensity detection process is performed on at least one light spot based on a pixel intensity parameter to obtain a first interference light spot;

[0205] If the light spot interference detection method is an energy ratio detection method, performing energy ratio detection processing on at least one light spot based on the pixel intensity parameter to obtain a second interference light spot;

[0206] If the light spot interference detection method is a pixel point average detection method, a pixel point average detection process is performed on at least one light spot based on the pixel intensity parameter to obtain a third interference light spot;

[0207] If the light spot interference detection mode is the energy total intensity detection mode, performing energy total intensity detection processing on at least one light spot based on the pixel intensity parameter to obtain a fourth interference light spot;

[0208] If the light spot interference detection method is a light spot position detection method, a light spot position detection process is performed on at least one light spot based on a pixel position parameter to obtain a fifth interference light spot.

[0209] Optionally, the interference detection unit 132 is specifically configured to:

[0210] Determine a first intensity priority corresponding to the light spot pixel point based on the pixel intensity parameter, and obtain a target light spot ratio for the reflection intensity map;

[0211] An intensity threshold is determined based on the target light spot ratio, the first intensity priority of the light spot pixel, and the pixel intensity parameter of the light spot pixel.

[0212] Based on the intensity threshold, a first interfering light spot is determined from at least one of the light spots.

[0213] Optionally, the interference detection unit 132 is specifically configured to:

[0214] Determine a first light spot based on the target light spot ratio and the first intensity priority of the light spot pixel point and obtain a first intensity threshold corresponding to the first light spot, and determine a first interference light spot from at least one of the light spots based on the first intensity threshold; or,

[0215] Obtaining the number of reference pixels corresponding to the light spot, determining a target ratio corresponding to the target light spot ratio and the number of reference pixels, determining a second light spot based on the target ratio and the first intensity priority of the light spot pixels, and obtaining a second intensity threshold corresponding to the second light spot, and determining a first interfering light spot from at least one of the light spots based on the second intensity threshold; or,

[0216] Based on the target light spot proportion and the first intensity priority of the light spot pixels, the first light spot is determined and the first intensity threshold corresponding to the first light spot is obtained, the number of reference pixels corresponding to the light spot is obtained, and the target ratio corresponding to the target light spot proportion and the number of reference pixels is determined; based on the target ratio and the first intensity priority of the light spot pixels, the second light spot is determined and the second intensity threshold corresponding to the second light spot is obtained; and based on the first intensity threshold and the second intensity threshold, a first interference light spot is determined from at least one of the light spots.

[0217] Optionally, the interference detection unit 132 is specifically configured to:

[0218] determining a threshold reference range based on the first intensity threshold and the second intensity threshold;

[0219] A target intensity threshold is acquired from the threshold reference range, and a first interfering light spot is determined from at least one of the light spots based on the target intensity threshold.

[0220] Optionally, the interference detection unit 132 is specifically configured to:

[0221] Obtaining a center point intensity value and an edge point intensity value corresponding to at least one of the light spots;

[0222] An energy ratio detection process is performed on at least one light spot based on the center point intensity value and the edge point intensity value, and a second interfering light spot is determined from the at least one light spot.

[0223] Optionally, the interference detection unit 132 is specifically configured to:

[0224] Determine a first intensity ratio of the light spot based on the center point intensity value and the edge point intensity value; determine a second interfering light spot from at least one of the light spots based on the first intensity ratio and an intensity ratio threshold; and / or,

[0225] Determine the first intensity ratio of the light spot based on the center point intensity value and the edge point intensity value; respectively obtain the first light spots corresponding to the light spots to determine the second intensity ratio of the first light spots; and determine a second interfering light spot from at least one of the light spots based on the first intensity ratio and the second intensity ratio of the light spots.

[0226] Optionally, the interference detection unit 132 is specifically configured to:

[0227] Determine a target difference based on the first intensity ratio and the second intensity ratio; determine a second interfering light spot from at least one of the light spots based on the target difference and a difference threshold; or,

[0228] Taking the first intensity ratio as a reference, an intensity ratio fluctuation range for the first light spot is determined based on at least one of the second intensity ratios; and a second interfering light spot is determined from at least one of the light spots based on the intensity ratio fluctuation range and a reference fluctuation range.

[0229] Optionally, the interference detection unit 132 is specifically configured to:

[0230] Acquire a neighboring light spot adjacent to the light spot, and use the neighboring light spot as a first light spot corresponding to the light spot; or,

[0231] A target distance between the light spot and at least one second light spot is acquired, and a first light spot corresponding to the light spot is determined from the at least one second light spot based on the target distance and a distance threshold.

[0232] Optionally, the interference detection unit 132 is specifically configured to:

[0233] Determine a target pixel region corresponding to at least one of the light spots, wherein the light spot is located in the target pixel region;

[0234] Determine a spot pixel point mean for at least one of the light spots based on the pixel intensity parameter, and determine a regional pixel point mean corresponding to at least one of the target pixel regions;

[0235] A pixel point mean value detection process is performed on at least one light spot based on the light spot pixel point mean value and the regional pixel point mean value of the light spot, and a third interference light spot is determined from the at least one light spot.

[0236] Optionally, the interference detection unit 132 is specifically configured to:

[0237] Obtaining a first total intensity corresponding to all light spot pixels of at least one of the light spots and a total number of light spot pixels, and taking the quotient of the first total intensity and the total number of light spot pixels as a light spot pixel mean of the light spot;

[0238] Obtain a second total intensity and a total number of regional pixels corresponding to all regional pixels of at least one of the target pixel regions, and use the quotient of the second total intensity and the total number of regional pixels as a mean value of the regional pixels corresponding to the target pixel region.

[0239] Optionally, the interference detection unit 132 is specifically configured to:

[0240] Determine a reference ratio corresponding to the mean value of the light spot pixels and the mean value of the regional pixels;

[0241] A pixel point mean value detection process is performed on at least one light spot based on the reference ratio and the ratio threshold, and a third interfering light spot is determined from the at least one light spot.

[0242] Optionally, the interference detection unit 132 is specifically configured to:

[0243] Based on the pixel intensity parameter, determining the spot intensity of the third light spot in the reference area and the average of the spot intensities corresponding to all the fourth light spots;

[0244] The third light spot is subjected to total energy intensity detection processing based on the light spot intensity of the third light spot and the light spot intensity average to determine a fourth interfering light spot.

[0245] Optionally, all the light spots included in the reference area are fourth light spots and the third light spot is one of all the fourth light spots; or, the reference area includes the third light spot and all the fourth light spots.

[0246] Optionally, the interference detection unit 132 is specifically configured to:

[0247] Obtaining the total number of light spots corresponding to the reference area; if the total number of light spots is greater than the light spot number threshold, executing the step of determining the light spot intensity of the third light spot in the reference area and the average light spot intensity corresponding to all fourth light spots based on the pixel intensity parameter; or,

[0248] The spot spacing distance between the two spots in the reference area is obtained. If the spot spacing distance is less than the spacing distance threshold, the step of determining the spot intensity of the third spot in the reference area and the average of the spot intensities corresponding to all the fourth spots based on the pixel intensity parameter is performed.

[0249] Optionally, the interference detection unit 132 is specifically configured to:

[0250] Determine an intensity ratio of the spot intensity of the third light spot to the average of the spot intensities;

[0251] If the intensity ratio is less than or equal to the first ratio threshold, the third light spot is used as the fourth interfering light spot.

[0252] Optionally, the interference detection unit 132 is specifically configured to:

[0253] Determine the light spot position for the at least one light spot based on the pixel position parameter; obtain a reference position range corresponding to the at least one light spot;

[0254] If the spot position of the light spot does not match the reference position range, the light spot is determined as a fifth interference light spot.

[0255] Optional, such as Fig.10 As shown, the device 1 further includes:

[0256] An image acquisition module 14 is used to acquire a color image and a depth image, wherein the reflection intensity map, the depth image and the color image are all different types of images for the same target object;

[0257] The interference verification module 15 is used to perform interference verification on the interference light spot based on the color image and the depth image.

[0258] Optional, such as Fig.11 As shown, the interference verification module 15 includes:

[0259] The color verification unit 151 is used to determine a third position of the interference light spot in the color image, and to determine a fourth position of at least one adjacent light spot corresponding to the interference light spot in the color image; and to obtain a color pixel verification result for the interference light spot based on the third position and the fourth position;

[0260] A depth verification unit 152 is used to determine a first position of the interfering light spot in the depth image, and to determine a second position of at least one adjacent light spot corresponding to the interfering light spot in the depth image; and to obtain a depth pixel verification result for the interfering light spot based on the first position and the second position;

[0261] The result generating unit 153 is configured to obtain an interference verification result for the interference light spot based on the color pixel verification result and the depth pixel verification result.

[0262] Optionally, the device 1 further includes:

[0263] The exposure adjustment module is used to obtain the number of light spots corresponding to the light spots, and if the number of light spots is greater than a number threshold, adjust the camera exposure time.

[0264] It should be noted that, when the image processing device provided in the above embodiment executes the image processing method, only the division of the above at least one functional module is used as an example. In actual application, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the image processing device provided in the above embodiment and the image processing method embodiment belong to the same concept, and the implementation process thereof is detailed in the method embodiment, which will not be repeated here.

[0265] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0266] In an embodiment of the present application, by obtaining a reflection intensity map, determining at least one light spot in the reflection intensity map, and then obtaining pixel characteristic parameters corresponding to the light spot pixels in the light spot, based on the difference in pixel characteristic parameters between the interference light spot and the effective light spot, the interference light spot can be accurately determined from at least one light spot, which can assist in eliminating the light spot interference caused by the superposition of edge signals of multiple light spots; the entire image interference processing process reduces the dependence on device hardware and complex algorithms, realizes accurate identification of light spot interference, can assist in outputting high-precision depth measurement results, and improves the robustness in depth measurement scenarios; and, different light spot interference detection methods or light spot interference detection methods with different time sequences determined based on actual application scenarios can be combined to optimize the detection process of interference light spots.

[0267] The present application also provides a computer storage medium, which can store multiple instructions, and the instructions are suitable for being loaded and executed by a processor as described above. Figure 1 to Figure 7 The image processing method of the embodiment shown in the figure can be specifically executed by referring to Figure 1 to Figure 7 The specific description of the illustrated embodiment will not be repeated here.

[0268] The present application also provides a computer program product, which stores at least one instruction, and the at least one instruction is loaded and executed by the processor as described above. Figure 1 to Figure 7 The image processing method of the embodiment shown in the figure can be specifically executed by referring to Figure 1 to Figure 7 The specific description of the illustrated embodiment will not be repeated here.

[0269] Please refer to Fig.12 , which shows a block diagram of the structure of an electronic device provided by an exemplary embodiment of the present application. The electronic device in the present application may include one or more of the following components: a processor 110, a memory 120, an input device 130, an output device 140, and a bus 150. The processor 110, the memory 120, the input device 130, and the output device 140 may be connected via the bus 150.

[0270] The processor 110 may include one or more processing cores. The processor 110 uses at least one interface and line to connect at least one part of the entire electronic device, and executes at least one function of the electronic device 100 and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 120, and calling data stored in the memory 120. Optionally, the processor 110 can be implemented in at least one hardware form of digital signal processing (digital signal processing, DSP), field-programmable gate array (field-programmable gate array, FPGA), and programmable logic array (programmable logic Array, PLA). The processor 110 can integrate one or a combination of a central processing unit (central processing unit, CPU), a graphics processing unit (graphics processing unit, GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 110, but may be implemented separately through a communication chip.

[0271] The memory 120 may include a random access memory (RAM) or a read-only memory (ROM). Optionally, the memory 120 includes a non-transitory computer-readable storage medium. The memory 120 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 120 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing at least one of the following method embodiments, etc. The operating system may be an Android system, including a system deeply developed based on the Android system, an IOS system developed by Apple, including a system deeply developed based on the IOS system or other systems. The data storage area may also store data created by the electronic device during use, such as a phone book, audio and video data, chat record data, etc.

[0272] See also Fig.13As shown, the memory 120 can be divided into an operating system space and a user space. The operating system runs in the operating system space, and native and third-party applications run in the user space. In order to ensure that different third-party applications can achieve good operating results, the operating system allocates corresponding system resources to different third-party applications. However, different application scenarios in the same third-party application also have different requirements for system resources. For example, in the local resource loading scenario, the third-party application has higher requirements for disk reading speed; in the animation rendering scenario, the third-party application has higher requirements for GPU performance. The operating system and third-party applications are independent of each other, and the operating system often cannot perceive the current application scenario of the third-party application in a timely manner, resulting in the operating system being unable to perform targeted system resource adaptation according to the specific application scenario of the third-party application.

[0273] In order for the operating system to distinguish the specific application scenarios of third-party applications, it is necessary to open up data communication between third-party applications and the operating system so that the operating system can obtain the current scenario information of third-party applications at any time, and then perform targeted system resource adaptation based on the current scenario.

[0274] Taking the Android operating system as an example, the programs and data stored in the memory 120 are as follows: Fig.14As shown, the memory 120 may store a Linux kernel layer 320, a system runtime library layer 340, an application framework layer 360 and an application layer 380, wherein the Linux kernel layer 320, the system runtime library layer 340 and the application framework layer 360 belong to the operating system space, and the application layer 380 belongs to the user space. The Linux kernel layer 320 provides the underlying driver for at least one hardware of the electronic device, such as display driver, audio driver, camera driver, Bluetooth driver, Wi-Fi driver, power management, etc. The system runtime library layer 340 provides the main feature support for the Android system through some C / C++ libraries. For example, the SQLite library provides database support, the OpenGL / ES library provides 3D drawing support, and the Webkit library provides browser kernel support, etc. The Android runtime library (Android runtime) is also provided in the system runtime library layer 340, which mainly provides some core libraries, allowing developers to use the Java language to write Android applications. The application framework layer 360 provides at least one API that may be used when building an application. Developers can also use these APIs to build their own applications, such as activity management, window management, view management, notification management, content provider, package management, call management, resource management, and location management. At least one application runs in the application layer 380. These applications can be native applications that come with the operating system, such as contact applications, text messaging applications, clock applications, camera applications, etc.; they can also be third-party applications developed by third-party developers, such as game applications, instant messaging applications, photo beautification applications, etc.

[0275] Taking the operating system as an IOS system as an example, the programs and data stored in the memory 120 are as follows: Fig.15As shown, the IOS system includes: a core operating system layer 420 (Core OS layer), a core service layer 440 (Core Services layer), a media layer 460 (Media layer), and a touchable layer 480 (Cocoa Touch Layer). The core operating system layer 420 includes an operating system kernel, a driver, and an underlying program framework, which provide functions closer to the hardware for use by the program framework located in the core service layer 440. The core service layer 440 provides system services and / or program frameworks required by the application, such as a foundation framework, an account framework, an advertising framework, a data storage framework, a network connection framework, a geographic location framework, a motion framework, and the like. The media layer 460 provides the application with interfaces related to audio and video, such as interfaces related to graphics and images, interfaces related to audio technology, interfaces related to video technology, and wireless playback (AirPlay) interfaces for audio and video transmission technology. The touchable layer 480 provides at least one commonly used interface-related framework for application development, and the touchable layer 480 is responsible for the user's touch interaction operations on the electronic device. For example, local notification service, remote push service, advertising framework, game tool framework, message user interface (UI) framework, user interface UIKit framework, map framework, etc.

[0276] exist Fig.15 Among the frameworks shown, the frameworks related to most applications include but are not limited to: the basic framework in the core service layer 440 and the UIKit framework in the touchable layer 480. The basic framework provides many basic object classes and data types, provides the most basic system services for all applications, and has nothing to do with UI. The classes provided by the UIKit framework are basic UI class libraries for creating touch-based user interfaces. iOS applications can provide UIs based on the UIKit framework, so it provides the basic architecture of applications for building user interfaces, drawing, processing and user interaction events, responding to gestures, etc.

[0277] Among them, the method and principle of implementing data communication between third-party applications and the operating system in the IOS system can be referred to the Android system, and this application will not go into details here.

[0278] Among them, the input device 130 is used to receive input instructions or data, and the input device 130 includes but is not limited to a keyboard, a mouse, a camera, a microphone or a touch device. The output device 140 is used to output instructions or data, and the output device 140 includes but is not limited to a display device and a speaker. In one example, the input device 130 and the output device 140 can be combined, and the input device 130 and the output device 140 are touch screen displays, which are used to receive touch operations on or near the user using any suitable object such as a finger or a touch pen, and to display the user interface of at least one application. The touch screen display is usually set on the front panel of the electronic device. The touch screen display can be designed as a full screen, a curved screen or a special-shaped screen. The touch screen display can also be designed as a combination of a full screen and a curved screen, or a combination of a special-shaped screen and a curved screen, which is not limited in the embodiments of the present application.

[0279] In addition, those skilled in the art will appreciate that the structure of the electronic device shown in the above drawings does not constitute a limitation on the electronic device, and the electronic device may include more or fewer components than shown, or combine certain components, or arrange the components differently. For example, the electronic device also includes a radio frequency circuit, an input unit, a sensor, an audio circuit, a wireless fidelity (WiFi) module, a power supply, a Bluetooth module and other components, which will not be described in detail here.

[0280] In the embodiment of the present application, the execution subject of at least one step may be the electronic device described above. Optionally, the execution subject of at least one step is the operating system of the electronic device. The operating system may be an Android system, an IOS system, or other operating systems, which is not limited in the embodiment of the present application.

[0281] The electronic device of the embodiment of the present application may also be equipped with a display device, which may be at least one device capable of realizing a display function, such as a cathode ray tube display (CR), a light-emitting diode display (LED), an electronic ink screen, a liquid crystal display (LCD), a plasma display panel (PDP), etc. The user may use the display device on the electronic device 101 to view displayed text, images, videos and other information. The electronic device may be a smart phone, a tablet computer, a gaming device, an AR (Augmented Reality) device, a car, a data storage device, an audio playback device, a video playback device, a notebook, a desktop computing device,

[0282] exist Fig.12 In the electronic device shown, where the electronic device may be a terminal, the processor 110 may be used to call the network optimization application stored in the memory 120, and specifically perform the following operations:

[0283] Acquire a reflection intensity map, and determine at least one light spot in the reflection intensity map;

[0284] Obtaining pixel characteristic parameters corresponding to the light spot pixel points in the light spot;

[0285] Based on the pixel characteristic parameter, an interfering light spot is determined from the at least one light spot.

[0286] In an embodiment, when the processor 110 determines the interfering light spot from the at least one light spot based on the pixel characteristic parameter, the processor 110 specifically performs the following operations:

[0287] At least one light spot interference detection method for the reflection intensity map is determined, and based on the pixel characteristic parameters, the light spot interference detection method is used to perform light spot interference detection processing on at least one of the light spots to obtain an interference light spot.

[0288] In an embodiment, when the processor 110 performs the light spot interference detection process on at least one of the light spots based on the pixel characteristic parameters and using the light spot interference detection method to obtain an interference light spot, the processor 110 specifically performs the following operations:

[0289] If the light spot interference detection method is a pixel point intensity detection method, a pixel point intensity detection process is performed on at least one light spot based on a pixel intensity parameter to obtain a first interference light spot;

[0290] If the light spot interference detection method is an energy ratio detection method, performing energy ratio detection processing on at least one light spot based on the pixel intensity parameter to obtain a second interference light spot;

[0291] If the light spot interference detection method is a pixel point average detection method, a pixel point average detection process is performed on at least one light spot based on the pixel intensity parameter to obtain a third interference light spot;

[0292] If the light spot interference detection mode is the energy total intensity detection mode, performing energy total intensity detection processing on at least one light spot based on the pixel intensity parameter to obtain a fourth interference light spot;

[0293] If the light spot interference detection method is a light spot position detection method, a light spot position detection process is performed on at least one light spot based on a pixel position parameter to obtain a fifth interference light spot.

[0294] In one embodiment, when the processor 110 performs the pixel intensity detection process on at least one light spot based on the pixel intensity parameter to obtain the first interference light spot, the processor 110 specifically performs the following steps:

[0295] Determine a first intensity priority corresponding to the light spot pixel point based on the pixel intensity parameter, and obtain a target light spot ratio for the reflection intensity map;

[0296] An intensity threshold is determined based on the target light spot ratio, the first intensity priority of the light spot pixel, and the pixel intensity parameter of the light spot pixel.

[0297] Based on the intensity threshold, a first interfering light spot is determined from at least one of the light spots.

[0298] In one embodiment, the processor 110 performs the following steps when determining the intensity threshold based on the target light spot ratio, the first intensity priority of the light spot pixel point, and the pixel intensity parameter of the light spot pixel point, and determining the first interference light spot from at least one of the light spots based on the intensity threshold value:

[0299] Determine a first light spot based on the target light spot ratio and the first intensity priority of the light spot pixel point and obtain a first intensity threshold corresponding to the first light spot, and determine a first interference light spot from at least one of the light spots based on the first intensity threshold; or,

[0300] Obtaining the number of reference pixels corresponding to the light spot, determining a target ratio corresponding to the target light spot ratio and the number of reference pixels, determining a second light spot based on the target ratio and the first intensity priority of the light spot pixels, and obtaining a second intensity threshold corresponding to the second light spot, and determining a first interfering light spot from at least one of the light spots based on the second intensity threshold; or,

[0301] Based on the target light spot proportion and the first intensity priority of the light spot pixels, the first light spot is determined and the first intensity threshold corresponding to the first light spot is obtained, the number of reference pixels corresponding to the light spot is obtained, and the target ratio corresponding to the target light spot proportion and the number of reference pixels is determined; based on the target ratio and the first intensity priority of the light spot pixels, the second light spot is determined and the second intensity threshold corresponding to the second light spot is obtained; and based on the first intensity threshold and the second intensity threshold, a first interference light spot is determined from at least one of the light spots.

[0302] In one embodiment, when the processor 110 determines the first interfering light spot from at least one light spot based on the first intensity threshold and the second intensity threshold, the processor 110 specifically performs the following steps:

[0303] determining a threshold reference range based on the first intensity threshold and the second intensity threshold;

[0304] A target intensity threshold is acquired from the threshold reference range, and a first interfering light spot is determined from at least one of the light spots based on the target intensity threshold.

[0305] In one embodiment, when the processor 110 performs the energy ratio detection process on at least one light spot based on the pixel intensity parameter to obtain the second interference light spot, the processor 110 specifically performs the following steps:

[0306] Obtaining a center point intensity value and an edge point intensity value corresponding to at least one of the light spots;

[0307] An energy ratio detection process is performed on at least one light spot based on the center point intensity value and the edge point intensity value, and a second interfering light spot is determined from the at least one light spot.

[0308] In one embodiment, when the processor 110 performs the energy ratio detection processing on at least one light spot based on the center point intensity value and the edge point intensity value to determine the second interference light spot from at least one light spot, the processor 110 specifically performs the following steps:

[0309] Determine a first intensity ratio of the light spot based on the center point intensity value and the edge point intensity value; determine a second interfering light spot from at least one of the light spots based on the first intensity ratio and an intensity ratio threshold; and / or,

[0310] Determine the first intensity ratio of the light spot based on the center point intensity value and the edge point intensity value; respectively obtain the first light spots corresponding to the light spots to determine the second intensity ratio of the first light spots; and determine a second interfering light spot from at least one of the light spots based on the first intensity ratio and the second intensity ratio of the light spots.

[0311] In one embodiment, when the processor 110 determines the second interfering light spot from at least one of the light spots based on the first intensity ratio and the second intensity ratio of the light spots, the processor 110 specifically performs the following steps:

[0312] Determine a target difference based on the first intensity ratio and the second intensity ratio; determine a second interfering light spot from at least one of the light spots based on the target difference and a difference threshold; or,

[0313] Taking the first intensity ratio as a reference, an intensity ratio fluctuation range for the first light spot is determined based on at least one of the second intensity ratios; and a second interfering light spot is determined from at least one of the light spots based on the intensity ratio fluctuation range and a reference fluctuation range.

[0314] In one embodiment, when the processor 110 executes the step of acquiring the first light spot corresponding to the light spot, the processor 110 specifically performs the following steps: acquiring an adjacent light spot adjacent to the light spot, and using the adjacent light spot as the first light spot corresponding to the light spot; or,

[0315] A target distance between the light spot and at least one second light spot is acquired, and a first light spot corresponding to the light spot is determined from the at least one second light spot based on the target distance and a distance threshold.

[0316] In one embodiment, when the processor 110 performs the pixel mean value detection processing on at least one light spot based on the pixel intensity parameter to obtain the third interference light spot, the processor 110 specifically performs the following steps: determining a target pixel area corresponding to at least one of the light spots, where the light spot is located in the target pixel area;

[0317] Determine a spot pixel point mean for at least one of the light spots based on the pixel intensity parameter, and determine a regional pixel point mean corresponding to at least one of the target pixel regions;

[0318] A pixel point mean value detection process is performed on at least one light spot based on the pixel point mean value of the light spot and the pixel point mean value of the region, and a third interfering light spot is determined from the at least one light spot.

[0319] In one embodiment, when the processor 110 determines the spot pixel point mean for at least one of the light spots based on the pixel intensity parameter and determines the regional pixel point mean corresponding to at least one of the target pixel areas, the processor 110 specifically performs the following steps:

[0320] Obtaining a first total intensity corresponding to all light spot pixels of at least one of the light spots and a total number of light spot pixels, and taking the quotient of the first total intensity and the total number of light spot pixels as a light spot pixel mean of the light spot;

[0321] Obtain a second total intensity and a total number of regional pixels corresponding to all regional pixels of at least one of the target pixel regions, and use the quotient of the second total intensity and the total number of regional pixels as a mean value of the regional pixels corresponding to the target pixel region.

[0322] In one embodiment, when the processor 110 performs the pixel mean detection processing on at least one light spot based on the pixel mean of the light spot and the pixel mean of the region to determine the third interference light spot from the at least one light spot, the processor 110 specifically performs the following steps: determining a reference ratio corresponding to the pixel mean of the light spot and the pixel mean of the region;

[0323] A pixel point mean value detection process is performed on at least one light spot based on the reference ratio and the ratio threshold, and a third interfering light spot is determined from the at least one light spot.

[0324] In one embodiment, when the processor 110 performs the total energy intensity detection processing on at least one light spot based on the pixel intensity parameter to obtain the fourth interference light spot, the processor 110 specifically performs the following steps: determining the light spot intensity of the third light spot in the reference area and the average light spot intensity corresponding to all fourth light spots based on the pixel intensity parameter; the fourth light spot is the light spot in the reference area except the third light spot;

[0325] The third light spot is subjected to total energy intensity detection processing based on the light spot intensity of the third light spot and the light spot intensity average to determine a fourth interfering light spot.

[0326] In one embodiment, all the light spots included in the reference area are fourth light spots and the third light spot is one of all the fourth light spots; or, the reference area includes the third light spot and all the fourth light spots.

[0327] In one embodiment, before determining the spot intensity of the third light spot in the reference area and the average of the spot intensities corresponding to all the fourth light spots based on the pixel intensity parameter, the processor 110 further performs the following steps: obtaining the total number of light spots corresponding to the reference area; if the total number of light spots is greater than the spot number threshold, performing the step of determining the spot intensity of the third light spot in the reference area and the average of the spot intensities corresponding to all the fourth light spots based on the pixel intensity parameter; or,

[0328] The spot spacing distance between the two spots in the reference area is obtained. If the spot spacing distance is less than the spacing distance threshold, the step of determining the spot intensity of the third spot in the reference area and the average of the spot intensities corresponding to all the fourth spots based on the pixel intensity parameter is performed.

[0329] In one embodiment, when the processor 110 performs the total energy intensity detection processing on the third light spot based on the light spot intensity of the third light spot and the light spot intensity mean to determine the fourth interference light spot, the processor 110 specifically performs the following steps:

[0330] Determine an intensity ratio of the spot intensity of the third light spot to the average of the spot intensities;

[0331] If the intensity ratio is less than or equal to the first ratio threshold, the third light spot is used as the fourth interfering light spot.

[0332] In one embodiment, when the processor 110 performs the light spot position detection process on at least one light spot based on the pixel position parameter to obtain the fifth interference light spot, the processor 110 specifically performs the following steps:

[0333] Determine the light spot position for the at least one light spot based on the pixel position parameter; obtain a reference position range corresponding to the at least one light spot;

[0334] If the spot position of the light spot does not match the reference position range, the light spot is determined as a fifth interference light spot.

[0335] In one embodiment, after determining the interfering light spot from the at least one light spot, the processor 110 further performs the following steps:

[0336] Acquire a color image and a depth image, wherein the reflection intensity map, the depth image, and the color image are all different types of images for the same target object;

[0337] Based on the color image and the depth image, interference verification is performed on the interference light spot.

[0338] In one embodiment, when the processor 110 performs the interference verification on the interference spot based on the color image and the depth image, the processor 110 specifically performs the following steps:

[0339] Determine a first position of the interfering light spot in the depth image, and determine a second position of at least one adjacent light spot corresponding to the interfering light spot in the depth image; based on the first position and the second position, obtain a depth pixel verification result for the interfering light spot;

[0340] Determine a third position of the interfering light spot in the color image, and determine a fourth position of at least one adjacent light spot corresponding to the interfering light spot in the color image; based on the third position and the fourth position, obtain a color pixel verification result for the interfering light spot;

[0341] Based on the color pixel verification result and the depth pixel verification result, an interference verification result for the interference light spot is obtained.

[0342] In one embodiment, after executing the steps of acquiring the reflection intensity map and determining at least one light spot in the reflection intensity map, the processor 110 further performs the following steps:

[0343] Obtaining the number of light spots corresponding to the light spot;

[0344] If the number of light spots is greater than the threshold, the camera exposure time is adjusted.

[0345] In an embodiment of the present application, by obtaining a reflection intensity map, determining at least one light spot in the reflection intensity map, and then obtaining pixel characteristic parameters corresponding to the light spot pixels in the light spot, based on the difference in pixel characteristic parameters between the interference light spot and the effective light spot, the interference light spot can be accurately determined from at least one light spot, which can assist in eliminating the light spot interference caused by the superposition of edge signals of multiple light spots; the entire image interference processing process reduces the dependence on device hardware and complex algorithms, realizes accurate identification of light spot interference, can assist in outputting high-precision depth measurement results, and improves the robustness in depth measurement scenarios; and, different light spot interference detection methods or light spot interference detection methods with different time sequences determined based on actual application scenarios can be combined to optimize the detection process of interference light spots.

[0346] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the process of at least one of the above-mentioned embodiments of the method. The storage medium can be a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0347] The above disclosure is only the preferred embodiment of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. An image processing method, characterized in that: The method comprises: Acquire a reflection intensity map, and determine at least one light spot in the reflection intensity map; Obtaining pixel characteristic parameters corresponding to the light spot pixel points in the light spot, wherein the pixel characteristic parameters include pixel intensity parameters and pixel position parameters; Determining an interfering light spot from the at least one light spot based on the pixel characteristic parameter; The step of determining an interfering light spot from the at least one light spot based on the pixel characteristic parameter comprises: determining at least one light spot interference detection mode for the reflection intensity map; If the light spot interference detection method is a pixel point intensity detection method, a pixel point intensity detection process is performed on at least one light spot based on a pixel intensity parameter to obtain a first interference light spot; If the light spot interference detection method is an energy ratio detection method, performing energy ratio detection processing on at least one light spot based on the pixel intensity parameter to obtain a second interference light spot; If the light spot interference detection method is a pixel point average detection method, a pixel point average detection process is performed on at least one light spot based on the pixel intensity parameter to obtain a third interference light spot; If the light spot interference detection mode is the energy total intensity detection mode, performing energy total intensity detection processing on at least one light spot based on the pixel intensity parameter to obtain a fourth interference light spot; If the light spot interference detection method is a light spot position detection method, a light spot position detection process is performed on at least one light spot based on a pixel position parameter to obtain a fifth interference light spot.

2. The method according to claim 1, characterized in that The step of performing pixel point intensity detection processing on at least one light spot based on the pixel intensity parameter to obtain a first interference light spot includes: Determine a first intensity priority corresponding to the light spot pixel point based on the pixel intensity parameter, and obtain a target light spot ratio for the reflection intensity map; Determining an intensity threshold based on the target light spot ratio, the first intensity priority of the light spot pixel, and the pixel intensity parameter of the light spot pixel; Based on the intensity threshold, a first interfering light spot is determined from at least one of the light spots.

3. The method according to claim 2, characterized in that The step of determining an intensity threshold based on the target light spot ratio, the first intensity priority of the light spot pixel point, and the pixel intensity parameter of the light spot pixel point, and determining a first interference light spot from at least one of the light spots based on the intensity threshold, comprises: Determine a first light spot based on the target light spot ratio and the first intensity priority of the light spot pixel point and obtain a first intensity threshold corresponding to the first light spot, and determine a first interference light spot from at least one of the light spots based on the first intensity threshold; or, Obtaining the number of reference pixels corresponding to the light spot, determining a target ratio corresponding to the target light spot ratio and the number of reference pixels, determining a second light spot based on the target ratio and the first intensity priority of the light spot pixels, and obtaining a second intensity threshold corresponding to the second light spot, and determining a first interfering light spot from at least one of the light spots based on the second intensity threshold; or, Based on the target light spot proportion and the first intensity priority of the light spot pixels, the first light spot is determined and the first intensity threshold corresponding to the first light spot is obtained, the number of reference pixels corresponding to the light spot is obtained, and the target ratio corresponding to the target light spot proportion and the number of reference pixels is determined; based on the target ratio and the first intensity priority of the light spot pixels, the second light spot is determined and the second intensity threshold corresponding to the second light spot is obtained; and based on the first intensity threshold and the second intensity threshold, a first interference light spot is determined from at least one of the light spots.

4. The method according to claim 3, characterized in that The determining a first interfering light spot from at least one of the light spots based on the first intensity threshold and the second intensity threshold comprises: determining a threshold reference range based on the first intensity threshold and the second intensity threshold; A target intensity threshold is acquired from the threshold reference range, and a first interfering light spot is determined from at least one of the light spots based on the target intensity threshold.

5. The method according to claim 1, characterized in that The performing energy ratio detection processing on at least one light spot based on the pixel intensity parameter to obtain a second interference light spot includes: Obtaining a center point intensity value and an edge point intensity value corresponding to at least one of the light spots; An energy ratio detection process is performed on at least one light spot based on the center point intensity value and the edge point intensity value, and a second interfering light spot is determined from the at least one light spot.

6. The method according to claim 5, characterized in that The performing energy ratio detection processing on at least one light spot based on the center point intensity value and the edge point intensity value to determine a second interference light spot from the at least one light spot includes: Determine a first intensity ratio of the light spot based on the center point intensity value and the edge point intensity value; determine a second interfering light spot from at least one of the light spots based on the first intensity ratio and an intensity ratio threshold; and / or, Determine the first intensity ratio of the light spot based on the center point intensity value and the edge point intensity value; respectively obtain the first light spots corresponding to the light spots to determine the second intensity ratio of the first light spots; and determine a second interfering light spot from at least one of the light spots based on the first intensity ratio and the second intensity ratio of the light spots.

7. The method according to claim 6, characterized in that The step of determining a second interfering light spot from at least one of the light spots based on the first intensity ratio and the second intensity ratio of the light spots comprises: Determine a target difference based on the first intensity ratio and the second intensity ratio; determine a second interfering light spot from at least one of the light spots based on the target difference and a difference threshold; or, Taking the first intensity ratio as a reference, an intensity ratio fluctuation range for the first light spot is determined based on at least one of the second intensity ratios; and a second interfering light spot is determined from at least one of the light spots based on the intensity ratio fluctuation range and a reference fluctuation range.

8. The method according to claim 6, characterized in that The obtaining a first light spot corresponding to the light spot includes: Acquire a neighboring light spot adjacent to the light spot, and use the neighboring light spot as a first light spot corresponding to the light spot; or, A target distance between the light spot and at least one second light spot is acquired, and a first light spot corresponding to the light spot is determined from the at least one second light spot based on the target distance and a distance threshold.

9. The method according to claim 1, characterized in that: The performing pixel point mean value detection processing on at least one light spot based on the pixel intensity parameter to obtain a third interference light spot includes: Determine a target pixel region corresponding to at least one of the light spots, wherein the light spot is located in the target pixel region; Determine a spot pixel point mean for at least one of the light spots based on the pixel intensity parameter, and determine a regional pixel point mean corresponding to at least one of the target pixel regions; Based on the light spot pixel mean value and the regional pixel mean value of the light spot, pixel mean value detection processing is performed on at least one light spot, and a third interfering light spot is determined from the at least one light spot.

10. The method according to claim 9, characterized in that The step of determining a spot pixel point mean value for at least one of the spot based on the pixel intensity parameter and determining a regional pixel point mean value corresponding to at least one of the target pixel regions comprises: Obtaining a first total intensity corresponding to all light spot pixels of at least one of the light spots and a total number of light spot pixels, and taking the quotient of the first total intensity and the total number of light spot pixels as a light spot pixel mean of the light spot; Obtain a second total intensity and a total number of regional pixels corresponding to all regional pixels of at least one of the target pixel regions, and use the quotient of the second total intensity and the total number of regional pixels as a mean value of the regional pixels corresponding to the target pixel region.

11. The method according to claim 9, characterized in that The performing pixel point mean value detection processing on at least one light spot based on the pixel point mean value of the light spot and the pixel point mean value of the region, and determining a third interference light spot from the at least one light spot, comprises: Determine a reference ratio corresponding to the mean value of the light spot pixels and the mean value of the regional pixels; A pixel point mean value detection process is performed on at least one light spot based on the reference ratio and the ratio threshold, and a third interfering light spot is determined from the at least one light spot.

12. The method according to claim 1, characterized in that The performing total energy intensity detection processing on at least one light spot based on the pixel intensity parameter to obtain a fourth interference light spot includes: Based on the pixel intensity parameter, determining the spot intensity of the third light spot in the reference area and the average of the spot intensities corresponding to all the fourth light spots; The third light spot is subjected to total energy intensity detection processing based on the light spot intensity of the third light spot and the light spot intensity average to determine a fourth interfering light spot.

13. The method according to claim 12, characterized in that All the light spots included in the reference area are fourth light spots and the third light spot is one of all the fourth light spots; or, the reference area includes the third light spot and all the fourth light spots.

14. The method according to claim 12, characterized in that Before determining the spot intensity of the third light spot in the reference area and the average of the spot intensities corresponding to all the fourth light spots based on the pixel intensity parameter, the method further includes: Obtaining the total number of light spots corresponding to the reference area; if the total number of light spots is greater than the light spot number threshold, executing the step of determining the light spot intensity of the third light spot in the reference area and the average light spot intensity corresponding to all fourth light spots based on the pixel intensity parameter; or, The spot spacing distance between the two spots in the reference area is obtained. If the spot spacing distance is less than the spacing distance threshold, the step of determining the spot intensity of the third spot in the reference area and the average of the spot intensities corresponding to all the fourth spots based on the pixel intensity parameter is performed.

15. The method according to claim 12, characterized in that The performing total energy intensity detection processing on the third light spot based on the light spot intensity of the third light spot and the light spot intensity mean to determine the fourth interference light spot includes: Determine an intensity ratio of the spot intensity of the third light spot to the average of the spot intensities; If the intensity ratio is less than or equal to the first ratio threshold, the third light spot is used as the fourth interfering light spot.

16. The method according to claim 1, characterized in that The step of performing a light spot position detection process on at least one light spot based on a pixel position parameter to obtain a fifth interference light spot includes: Determine the light spot position for the at least one light spot based on the pixel position parameter; obtain a reference position range corresponding to the at least one light spot; If the spot position of the light spot does not match the reference position range, the light spot is determined as a fifth interference light spot.

17. The method according to claim 1, characterized in that After determining the interfering light spot from the at least one light spot, the method further includes: Acquire a color image and a depth image, wherein the reflection intensity map, the depth image, and the color image are all different types of images for the same target object; Based on the color image and the depth image, interference verification is performed on the interference light spot.

18. The method according to claim 17, characterized in that The performing interference verification on the interference light spot based on the color image and the depth image includes: Determine a first position of the interfering light spot in the depth image, and determine a second position of at least one adjacent light spot corresponding to the interfering light spot in the depth image; based on the first position and the second position, obtain a depth pixel verification result for the interfering light spot; Determine a third position of the interfering light spot in the color image, and determine a fourth position of at least one adjacent light spot corresponding to the interfering light spot in the color image; based on the third position and the fourth position, obtain a color pixel verification result for the interfering light spot; Based on the color pixel verification result and the depth pixel verification result, an interference verification result for the interference light spot is obtained.

19. The method according to claim 1, characterized in that After acquiring the reflection intensity map and determining at least one light spot in the reflection intensity map, the method further includes: Obtaining the number of light spots corresponding to the light spot; If the number of light spots is greater than the number threshold, the camera exposure time is adjusted.

20. An image processing device, characterized in that: The device comprises: a light spot determination module, configured to obtain a reflection intensity map and determine at least one light spot in the reflection intensity map; A parameter acquisition module, used to acquire pixel characteristic parameters corresponding to the light spot pixel points in the light spot, wherein the pixel characteristic parameters include pixel intensity parameters and pixel position parameters; An interference determination module, configured to determine an interference light spot from the at least one light spot based on the pixel characteristic parameter; The step of determining an interfering light spot from the at least one light spot based on the pixel characteristic parameter comprises: determining at least one light spot interference detection mode for the reflection intensity map; If the light spot interference detection method is a pixel point intensity detection method, a pixel point intensity detection process is performed on at least one light spot based on a pixel intensity parameter to obtain a first interference light spot; If the light spot interference detection method is an energy ratio detection method, performing energy ratio detection processing on at least one light spot based on the pixel intensity parameter to obtain a second interference light spot; If the light spot interference detection method is a pixel point average detection method, a pixel point average detection process is performed on at least one light spot based on the pixel intensity parameter to obtain a third interference light spot; If the light spot interference detection mode is the energy total intensity detection mode, performing energy total intensity detection processing on at least one light spot based on the pixel intensity parameter to obtain a fourth interference light spot; If the light spot interference detection method is a light spot position detection method, a light spot position detection process is performed on at least one light spot based on a pixel position parameter to obtain a fifth interference light spot.

21. A computer storage medium, characterized in that The computer storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the method steps as claimed in any one of claims 1 to 19.

22. An electronic device, characterized in that: include: A processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the method steps as claimed in any one of claims 1 to 19.

Images