Depth information determination method, device and storage medium

Abstract

The present application discloses a method, device and storage medium for determining depth information, wherein the method comprises: obtaining a scene picture of an initial scene; photographing a target scene using at least one photographing device to obtain a first scene picture of the target scene; determining that there are objects of a preset type in the target scene when there is a preset first category change between the scene picture of the initial scene and the first scene picture of the target scene; determining a depth parameter corresponding to the target scene based on the objects of the preset type in the target scene; and determining the depth information of at least one object in the target scene based on the depth parameter corresponding to the target scene. Thus, the embodiments of the present application can adjust the depth parameters that may exist in different scenes and affect the accuracy of the depth information of the detected object to improve the accuracy of the detected depth information.

Description

Technical Field

[0001] The present application relates to the field of measurement, and in particular to a method, device and storage medium for determining depth information. Background Art

[0002] 3D depth sensing Time of Flight (ToF) technology is often used in logistics, navigation, robotics, and security to solve distance detection problems that traditional 2D technology cannot solve. ToF depth sensors detect the distance of objects in the target scene by emitting electromagnetic wave signals to different objects and receiving reflected signals. Summary of the invention

[0003] Embodiments of the present application provide a method, device, and storage medium for determining depth information.

[0004] In a first aspect, an embodiment of the present application provides a method for determining depth information, the method for determining depth information being used for a device that needs to determine depth information, the device comprising at least one shooting device;

[0005] Acquire a scene image of an initial scene; wherein the initial scene represents a scene whose light intensity is within a preset range and does not contain objects of a preset type;

[0006] Using the at least one shooting device to shoot a target scene to obtain a first scene picture of the target scene;

[0007] In a case where there is a preset first category change between the scene picture of the initial scene and the first scene picture of the target scene, determining that there is an object of the preset type in the target scene; wherein the first category change indicates that a light intensity difference between the outgoing light emitted by the at least one shooting device and the reflected light received by the at least one shooting device is less than a first preset threshold or greater than a second preset threshold;

[0008] Determining a depth parameter corresponding to the target scene based on the preset type of object in the target scene; wherein the depth parameter represents a parameter for adjusting the depth information;

[0009] Determine depth information of at least one object in the target scene based on the depth parameter corresponding to the target scene.

[0010] In a second aspect, an embodiment of the present application provides a depth information determination device, wherein the depth information determination method is used for a device that needs to determine depth information, wherein the device includes at least one shooting device;

[0011] An acquisition module, used to acquire a scene image of an initial scene; wherein the initial scene represents a scene whose light intensity is within a preset range and does not contain objects of a preset type;

[0012] An obtaining module, configured to use the at least one photographing device to photograph a target scene to obtain a first scene image of the target scene;

[0013] A first determination module is configured to determine that an object of the preset type exists in the target scene when there is a preset first category change between the scene image of the initial scene and the first scene image of the target scene; wherein the first category change indicates that a light intensity difference between the outgoing light emitted by the at least one camera and the reflected light received by the at least one camera is greater than a first preset threshold or less than a second preset threshold;

[0014] A second determination module is used to determine a depth parameter corresponding to the target scene based on the preset type of object in the target scene; wherein the depth parameter represents a parameter for adjusting the depth information;

[0015] The third determination module is used to determine the depth information of at least one object in the target scene based on the depth parameter corresponding to the target scene.

[0016] In a third aspect, an embodiment of the present application provides a computer storage medium, wherein the computer storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor and executing the above-mentioned method steps.

[0017] In a fourth aspect, an embodiment of the present application provides an electronic device, which may include: a processor and a memory;

[0018] The memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the above method steps.

[0019] The beneficial effects brought about by the technical solutions provided by some embodiments of the present application include at least:

[0020] In an embodiment of the present application, a scene picture of an initial scene can be obtained; a first scene picture of the target scene can be obtained by photographing the target scene using at least one photographing device; when there is a preset first category change between the scene picture of the initial scene and the first scene picture of the target scene, it is determined that there is a preset type of object in the target scene; a depth parameter corresponding to the target scene is determined based on the preset type of object in the target scene; and depth information of at least one object in the target scene is determined based on the depth parameter corresponding to the target scene. Thus, an embodiment of the present application can photograph the target scene using a photographing device, and determine the depth parameter corresponding to the target scene based on the preset type of object in the first scene picture of the target scene. Thereby, the depth parameters that may exist in different scenes and affect the accuracy of the depth information of the detected object are adjusted to solve the problem of inaccurate depth information detected by the depth sensor in the related art when using a unified depth parameter in different scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0022] Figure 1a A schematic diagram of an application scenario of a depth information determination method in the related art is exemplified;

[0023] Figure 1b Another application scenario schematic diagram of the depth information determination method in the related art is exemplarily shown;

[0024] Figure 2a A schematic diagram of a flow chart of a method for determining depth information provided in an embodiment of the present application;

[0025] Figure 2b A schematic diagram of waveform information in a device provided in an embodiment of the present application;

[0026] Figure 3 A schematic diagram of a flow chart of another method for determining depth information provided in an embodiment of the present application;

[0027] Figure 4 A flowchart of another method for determining depth information provided in an embodiment of the present application;

[0028] Figure 5 A schematic diagram of a flow chart of another method for determining depth information provided in an embodiment of the present application;

[0029] Figure 6A schematic diagram of the structure of a depth information determination method and device provided in an embodiment of the present application;

[0030] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Instead, they are only examples of devices and methods consistent with some aspects of the present application as detailed in the attached claims.

[0032] In the description of the present application, it should be understood that the terms "first", "second", etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to the specific circumstances. In addition, in the description of the present application, unless otherwise specified, "multiple" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects associated before and after are in an "or" relationship.

[0033] Figure 1a A schematic diagram of an application scenario of a depth information determination method in the related art is exemplified. Figure 1b Another application scenario diagram of the depth information determination method in the related art is exemplarily shown. Figure 1a The scene shown may include: a bedroom 11 and a sweeping robot 12, Figure 1b The scene shown may include: a kitchen 13 and a cleaning robot 12 .

[0034] Specifically, when the sweeping robot 12 is in the bedroom 11 scene, it can emit outgoing light to the bedroom 11 and receive reflected light from each object in the bedroom 11 scene to determine the depth information of each object in the bedroom 11 scene, and determine the distance and position between the sweeping robot 12 and each object in the bedroom 11 scene based on the depth information, so as to automatically complete the cleaning, vacuuming, mopping and other tasks in the bedroom 11 scene. Similarly, when the sweeping robot 12 is in the kitchen 13 scene, it can emit outgoing light to the kitchen 13 and receive reflected light from each object in the kitchen 13 scene to determine the depth information of each object in the kitchen 13 scene, and determine the distance and position between the sweeping robot 12 and each object in the kitchen 13 scene based on the depth information, so as to automatically complete the cleaning, vacuuming, mopping and other tasks in the kitchen 13 scene.

[0035] It is understandable that different scenes such as the bedroom 11 and the kitchen 13 include different objects. For example, the bedroom mainly includes objects such as beds, cabinets, curtains, carpets, etc., which may absorb light, while the kitchen 13 mainly includes pots and pans, refrigerators, ovens, dishwashers, etc., which may be smooth objects with high reflectivity of glass. Therefore, at the same distance, the reflected light energy reflected by objects of different materials to the sweeping robot 12 may be different, and accordingly, the depth information and distance information calculated in the sweeping robot 12 may also be inaccurate.

[0036] Next, combine Figure 1a and Figure 1b The depth information determination method provided in the embodiment of the present application is introduced by referring to the application scenario diagram of the depth information determination method in the related technology.

[0037] The embodiment of the present application may provide a flow chart of a method for determining depth information. The method for determining depth information may be used for a device that needs to determine depth information, and the device may include at least one photographing device. Possibly, the device may be any device that needs to identify the distance between the location of the device and various objects in the room, for example, an autonomous mobile sweeping robot, a manually remote-controlled sweeping robot, a hand-push sweeping robot, and other cleaning devices.

[0038] In one embodiment, Figure 2a A flow chart of a method for determining depth information is shown in FIG. Figure 2a As shown, the depth information determination method may include the following steps:

[0039] S201, obtaining a scene image of an initial scene.

[0040] The initial scene in the embodiment of the present application refers to a scene that is pre-stored in the device, has a light intensity within a preset range, and does not contain objects of a preset type.

[0041] Specifically, the initial scene can be understood as the light of the indoor incident sunlight is not dazzling during the day, and the light generated by the indoor lights is not dazzling and can clearly illuminate the entire scene when the sunlight is insufficient or at night. In addition, the initial scene cannot include objects of preset types such as multipath type objects and light absorption type objects.

[0042] Possibly, assuming that the light intensity range can be in the value [0, 100], the preset range of light intensity in the embodiment of the present application can be set between (30, 70). Among them, [0, 30] represents the weak light range, that is, insufficient indoor light during the day or dim light at night; [70, 100] represents the strong light range, that is, indoor incident light during the day or indoor light glare at night.

[0043] S202: Use at least one shooting device to shoot a target scene to obtain a first scene picture of the target scene.

[0044] Among them, the target scene in the embodiment of the present application represents the scene of the space where the device is located. The shooting device in the embodiment of the present application is not limited to a conventional camera. Specifically, it may include a sensor for semantic recognition of each object in the target scene after acquiring the target scene image. Among them, the role of semantic recognition can be to identify the category of each object in the target scene image, for example, whether the object to be identified is a table, a chair, a pot, a bowl, clothing, or a plush toy.

[0045] Possibly, the device in the embodiment of the present application may include multiple shooting devices. Specifically, multiple shooting devices may be used to shoot the same target scene at the same time to obtain multiple first scene pictures. Among them, a passive depth sensor may be set in each shooting device in the embodiment of the present application, and these passive depth sensors receive energy (electromagnetic signals, etc.) reflected or emitted from the outside world to determine the depth information of each object in the target scene.

[0046] Possibly, the device in the embodiment of the present application may also include a shooting device and a depth detection sensor. Specifically, the shooting device may be used to shoot the target scene to obtain a first scene image of the target scene. Among them, the depth detection sensor may be an active depth sensor, which may detect the depth information of each object in the target scene by actively emitting energy (including but not limited to electromagnetic waves, microwaves, etc.).

[0047] S203: When there is a preset first category change between the scene image of the initial scene and the first scene image of the target scene, determine whether there is an object of a preset type in the target scene.

[0048] The first type of change in the embodiment of the present application indicates that the light intensity difference between the outgoing light emitted by at least one camera and the reflected light received by at least one camera is less than a first preset threshold or greater than a second preset threshold. Possibly, the first preset threshold is less than the second threshold.

[0049] It is understandable that preset types of objects such as multipath type objects and light-absorbing type objects may cause a sharp change in the energy of the reflected light. It is understandable that the difference in light intensity between the outgoing light emitted by the camera and the received reflected light is less than the first preset threshold value, which can be understood as the intensity of the light reflected by the outgoing light emitted by the camera after hitting a strong reflective object such as a mirror or a transparent object (for example, a glass product) is similar to the intensity of the outgoing light, that is, the light energy attenuation is small. The difference in light intensity between the outgoing light emitted by the camera and the received reflected light is greater than the second preset threshold value, which can be understood as the intensity of the reflected light reflected by the outgoing light emitted by the camera after hitting a light-absorbing object such as a carpet or black is almost zero, that is, the light energy attenuation is large.

[0050] S204: Determine a depth parameter corresponding to the target scene based on an object of a preset type in the target scene.

[0051] Among them, the depth parameter in the embodiment of the present application represents a parameter for adjusting the depth information.

[0052] It is understandable that depth parameters such as the exposure time of the shooting device and the filtering strength of the filter may affect the accuracy of the depth information of each object in the target scene. Therefore, it is necessary to determine the exposure time of the shooting device, the filtering strength of the filter and other depth parameters required for the target scene according to the preset type of objects in the target scene.

[0053] S205: Determine depth information of at least one object in the target scene based on the depth parameter corresponding to the target scene.

[0054] It can be understood that the embodiment of the present application can process the acquired first scene image of the target scene after determining depth parameters such as the exposure time of the shooting device and the filtering strength of the filter, so as to obtain accurate depth information of each object in the target scene, for example, the three-dimensional coordinate information of each object in the first scene image, the distance of each object relative to the shooting device or a numerical value proportional to the distance.

[0055] Possibly, when a plurality of shooting devices are used to shoot the target scene respectively in the embodiment of the present application, each shooting device outputs at least one first scene picture of the target scene. The at least one first scene picture of the target scene output by each shooting device is processed based on the depth parameter of the target scene to determine the depth information of at least one object in the target scene.

[0056] Possibly, the embodiment of the present application can calculate the depth information of the object by comparing the phase change of the outgoing light and the reflected light. The method of calculating the distance is called the continuous wave (CW) method. Specifically, Figure 2bAs shown, C1-C4 represent the window periods of the shutters of the four shooting devices, and each window period is offset by 0°, 180°, 90°, and 270° relative to the phase of the outgoing light signal. Q1-Q4 represent the number of photons accumulated in the C1-C4 window periods.

[0057] Furthermore, Q1-Q4 must satisfy the relationship Q1+Q4=Q3+Q4=K and |Q1-Q2|+|Q3-Q4|=k. K is the total number of photons reflected in one pulse period, and the values ​​of Q1-Q4 can be calculated by autocorrelation between the window function and the reflected light.

[0058] Furthermore, the relationship between the phase shift between Q1-Q4 and the return light is as follows:

[0059]

[0060] Where a represents the amplitude of the incident light, represents the phase shift of the reflected light, ωτ represents the phase shift of the window function, b represents the fixed offset, τ represents the phase shift of the outgoing light, so Q1=c(0), Q2=c(π), Q3==c(π / 2), Q4==c(3π / 2).

[0061] Among them, according to the calculation relationship of trigonometric functions shown below, we can get

[0062]

[0063] So the phase deviation The distance d between the device and the object in the target scene can be calculated. Furthermore, the amplitude of the incident light and the size of the fixed offset can also be calculated using the following formula:

[0064]

[0065] Furthermore, the amplitude a of the incident light can also be used to generate a grayscale image corresponding to the depth information.

[0066] Possibly, when the device in the embodiment of the present application includes a shooting device and a depth detection sensor, the depth parameter of the target scene and the first scene image of the target scene can be input into the depth detection sensor to obtain the depth information of at least one object in the target scene.

[0067] Specifically, the loss of detection performance can be improved to a certain extent by adjusting the depth parameters of the depth detection sensor. In the embodiment of the present application, when the depth parameters of the depth detection sensor can be adjusted in time according to the environment of the target scene, the distance information of objects in different directions can be used through the depth information algorithm pre-stored in the device to realize scene construction and map construction and other scene applications.

[0068] In a specific example, the sweeping robot can obtain a scene picture whose light intensity is within a first preset range and does not contain objects of a preset type, and use a camera to shoot a bedroom scene to obtain a first scene picture of the bedroom scene. When the light intensity difference between the outgoing light emitted by the camera and the received reflected light between the scene picture of the initial scene and the first scene picture of the bedroom scene is greater than a second preset threshold, it can be determined that there is a light-absorbing object in the bedroom scene. Furthermore, the first scene picture can be processed by the depth parameter corresponding to the light-absorbing object to determine the depth information of the light-absorbing object in the bedroom scene, that is, the distance and position between the sweeping robot and the light-absorbing object.

[0069] In summary, in the embodiments of the present application, a scene picture of an initial scene can be obtained; at least one shooting device is used to shoot the target scene to obtain a first scene picture of the target scene; when there is a preset first category change between the scene picture of the initial scene and the first scene picture of the target scene, it is determined that there are objects of a preset type in the target scene; based on the objects of the preset type in the target scene, a depth parameter corresponding to the target scene is determined; and depth information of at least one object in the target scene is determined based on the depth parameter corresponding to the target scene. Thus, in the embodiments of the present application, a shooting device can be used to shoot the target scene, and the depth parameter corresponding to the target scene can be determined based on the objects of the preset type in the first scene picture of the target scene. Thereby, the depth parameters that may exist in different scenes and affect the accuracy of the depth information of the detected object are adjusted to solve the problem of inaccurate depth information detected by the depth sensor in the related art when using a unified depth parameter in different scenes.

[0070] In some implementations, the preset type of object in the embodiments of the present application may include a multipath type object. Wherein, the depth parameter may include: a filtering threshold and / or the number of neighborhood points.

[0071] Specifically, Figure 3 The flowchart of another method for determining depth information provided by an embodiment of the present application is exemplarily shown. Figure 3 As shown, the depth information determination method may at least include the following steps:

[0072] S301, obtaining a scene image of an initial scene.

[0073] Specifically, S301 is consistent with S201 and will not be described in detail here.

[0074] S302: Use at least one shooting device to shoot a target scene to obtain a first scene picture of the target scene.

[0075] Specifically, S302 is consistent with S202 and will not be repeated here.

[0076] S303, when a first category change between the scene image of the initial scene and the first scene image of the target scene is that the light intensity difference between the outgoing light emitted by at least one shooting device and the received reflected light is less than a first preset threshold, it is determined that there is a multipath type object in the target scene.

[0077] Specifically, the multipath type object in the embodiment of the present application refers to an object whose reflected light intensity is significantly enhanced due to multipath (multiple reflecting surfaces, such as corners, etc.) or strong reflection (such as mirrors, transparent objects, etc.), that is, an object with small light energy attenuation.

[0078] Specifically, the light intensity difference between the outgoing light emitted by the shooting device and the received reflected light is less than the first preset threshold value, which can be understood as the reflected light intensity received by the shooting device is larger, so that the light intensity difference between the scene image of the initial scene and the first scene image of the target scene is smaller (less than the first preset threshold value), that is, there is a sudden change in light intensity from the initial scene to the target scene. The reason for this sudden change in the embodiment of the present application can be considered to be the presence of a multipath type object in the target scene.

[0079] S304: Obtain a first confidence level of a multipath type object in the target scene based on a first scene image of the target scene.

[0080] Possibly, the embodiment of the present application can perform semantic recognition of objects on the first scene image of the target scene, thereby determining the two-dimensional position information, approximate shape information, size, and material of each object in the first scene image. Furthermore, the relevant information of the identified multipath class object type can be compared with the relevant information of each category of objects of the multipath class object type stored in the database, thereby providing at least one similar multipath class object type and its confidence.

[0081] For example, based on the first scene image of the target scene, it can be obtained that there are multipath object types in the target scene. Specifically, the confidence level of the multipath object type being a glass locker is 0.8, the confidence level of the multipath object type being a glass baffle is 0.3, and the confidence level of the multipath object type being a window is 0.2.

[0082] S305, compare the first confidence of the multipath type object in the target scene with the first type confidence threshold; if the first confidence of the multipath type object in the target scene is greater than or equal to the first type confidence threshold, execute S306-S307; if the first confidence of the multipath type object in the target scene is less than the first type confidence threshold, execute S308-S311.

[0083] The first type confidence threshold of the multipath type object in the embodiment of the present application represents a value used to determine the specific category of the multipath type object in the target scene. Possibly, the first type confidence threshold in the embodiment of the present application can be set to 0.7.

[0084] S306: Determine that the depth parameter corresponding to the target scene is a depth parameter corresponding to a multipath type object in a first scene picture of the target scene.

[0085] It is understandable that the embodiment of the present application can determine the specific category of the multipath type object in the target scene by comparing the first confidence and the first type confidence threshold. For example, if the first type confidence threshold is 0.7, the confidence of the multipath type object in the target scene is 0.8 for a glass locker, the confidence of the multipath type object is a glass baffle is 0.3, and the confidence of the multipath type object is a window is 0.2, it can be determined that the confidence of the glass locker meets the requirements of the first type confidence threshold.

[0086] Furthermore, the embodiment of the present application can obtain the pre-stored depth parameter corresponding to the multipath type object through the database. For example, the pre-stored depth parameter corresponding to the glass locker can be obtained through the database.

[0087] S307 : Determine depth information of the multipath type object in the target scene based on the depth parameter corresponding to the multipath type object in the first scene picture of the target scene.

[0088] It can be understood that, in the embodiments of the present application, the depth parameter corresponding to the multipath type object can be determined according to the specific type of the multipath type object, that is, the filtering threshold and / or the number of neighborhood points.

[0089] Furthermore, the embodiments of the present application can filter the point cloud information of the multipath type objects in the first scene image of the target scene based on the filtering threshold and / or the number of neighborhood points corresponding to this type of objects, so as to obtain the depth information of the multipath type objects in the target scene.

[0090] S308, after adjusting the filter threshold and / or the number of neighborhood points in the depth parameter, photograph the target scene again to obtain a second confidence level of the multipath type object in a second scene picture of the target scene.

[0091] Possibly, the embodiment of the present application can reduce the sigma value of the filter in the target scene or increase the number of neighborhood points to adjust the effect of filtering out flying spots, thereby providing an adjusted second confidence level of the multipath type object. After the second confidence level reaches the first type confidence level threshold (≥0.7), the result can be considered credible.

[0092] Specifically, the sigma value in the embodiment of the present application represents the standard deviation from the average value, that is, the amplitude of each point in the point cloud deviating from the mean point. When the sigma value is reduced, the points in the point cloud can be closer to the mean point, thereby effectively eliminating the flying points that deviate from the mean. The number of neighborhood points represents the number of points in the point cloud within a preset range. A more accurate mean point can be calculated by increasing the number of points in the point cloud within a preset range, and then eliminating points outside the mean point range, thereby effectively eliminating the flying points that deviate from the mean.

[0093] Possibly, the embodiment of the present application can also be set that if after completing the process of repeated shooting for multiple times, the confidence of the multipath type object obtained in each shooting is <0.7 and the confidence fluctuation is <0.1, then the confidence of the multipath type object obtained for multiple times can be averaged, and the depth parameter can be further adjusted based on the average.

[0094] S309, compare the second confidence of the multipath type object in the target scene with the first type confidence threshold; if the second confidence of the multipath type object in the target scene is less than the first type confidence threshold, execute S310; if the second confidence of the multipath type object in the target scene is greater than or equal to the first type confidence threshold, execute S311.

[0095] Specifically, the second confidence level in the embodiment of the present application represents the confidence level after the depth parameter is adjusted.

[0096] S310, then return to S308 to adjust the filter threshold and / or the number of neighborhood points in the depth parameter, and then shoot the target scene again until the second confidence of the multipath type object in the second scene picture of the target scene is greater than or equal to the first type confidence threshold.

[0097] It can be understood that when the second confidence of the multipath type object in the second scene picture of the target scene does not meet the preset conditions, that is, the second confidence is less than the first type confidence threshold, it is necessary to adjust the depth parameters in the device, and after the adjustment is completed, the target scene is photographed again. If the preset conditions are still not met, it is still necessary to adjust the depth parameters in the device again. After the adjustment is completed, the target scene is photographed again until the second confidence of the multipath type object in the second scene picture of the target scene meets the preset conditions, that is, the second confidence is greater than or equal to the first type confidence threshold.

[0098] S311: Determine that a depth parameter corresponding to the target scene is a depth parameter corresponding to a second confidence level of a multipath type object in a second scene picture of the target scene.

[0099] It can be understood that after the embodiment of the present application completes the adjustment of the filtering threshold and / or the number of neighborhood points in the depth parameter, when the second confidence of the multipath type object in the second scene image of the target scene is greater than or equal to the first type confidence threshold, the current depth parameter can be considered to be the depth parameter corresponding to the target scene.

[0100] S312: Determine depth information of the multipath type object in the target scene based on a depth parameter corresponding to a second confidence level of the multipath type object in a second scene picture of the target scene.

[0101] Specifically, the embodiment of the present application can use the adjusted filtering threshold and / or the number of neighborhood points corresponding to the second confidence level of the multipath type objects in the target scene to filter the multipath type objects in the second scene picture of the target scene to obtain the depth information of the multipath type objects in the target scene.

[0102] In a specific example, the sweeping robot can obtain a scene picture whose light intensity is within a preset range and does not contain objects of a preset type, and use two shooting devices to shoot the bedroom scene respectively to obtain two first scene pictures of the bedroom scene. Further, the common feature points of the same objects in the two first scene pictures can be marked, and the point cloud position information corresponding to each object in the first scene picture can be calculated according to the installation position of the shooting device. Furthermore, if the light intensity difference between the outgoing light emitted by the shooting device and the received reflected light between the scene picture of the initial scene and the first scene picture of the bedroom scene is less than a first preset threshold, it can be determined that there are multipath type objects in the bedroom scene. Furthermore, the first confidence level 0.5 of the multipath type object can be determined based on the semantic recognition technology. Since the first confidence level 0.5 is less than the first type confidence level threshold 0.7, it is necessary to adjust the depth parameters in the device and then use two shooting devices to shoot the target scene to obtain the second confidence level 0.75 of the multipath type object. Since the second confidence level 0.75 is greater than the first type confidence level threshold 0.7, it can be considered that the adjusted depth parameters are the depth parameters corresponding to the bedroom scene. In other words, the second scene picture of the bedroom scene can be processed using the adjusted parameters to determine the depth information of the multipath type object in the bedroom scene, that is, information such as the distance and position between the sweeping robot and the multipath type object.

[0103] Therefore, when the light intensity difference between the outgoing light emitted by the shooting device and the received reflected light is less than a first preset threshold value, the embodiment of the present application determines the depth parameter of the target scene by determining whether the confidence of the multipath type object in the target scene meets the preset conditions, so as to reduce the factors that may cause inaccurate depth information detection due to the multipath type objects in the target scene.

[0104] In some implementations, the preset type of object in the embodiments of the present application may include a light absorbing type object. Wherein, the depth parameter may include: a filtering threshold and / or the number of neighborhood points.

[0105] Specifically, Figure 4 The flowchart of another method for determining depth information provided by an embodiment of the present application is exemplarily shown. Figure 4 As shown, the depth information determination method may at least include the following steps:

[0106] S401, obtaining a scene image of an initial scene.

[0107] Specifically, S401 is consistent with S201 and will not be described in detail here.

[0108] S402: Use at least one shooting device to shoot a target scene to obtain a first scene picture of the target scene.

[0109] Specifically, S402 is consistent with S202 and will not be repeated here.

[0110] S403, a first category change between the scene image of the initial scene and the first scene image of the target scene is that the light intensity difference between the outgoing light emitted by at least one shooting device and the received reflected light is greater than a second preset threshold, and it is determined that there is a light-absorbing type object in the target scene.

[0111] Specifically, the light-absorbing type object in the embodiment of the present application refers to an object whose reflected light intensity is significantly reduced due to color (eg, black, dark color, etc.) or material (eg, carpet, plush toy, etc.).

[0112] Specifically, the light intensity difference between the outgoing light emitted by the shooting device and the received reflected light is greater than the second preset threshold value, which can be understood as the reflected light intensity received by the shooting device is smaller, so that the light intensity difference between the scene image of the initial scene and the first scene image of the target scene is larger (greater than the second preset threshold value), that is, there is a sudden change in light intensity from the initial scene to the target scene. The reason for this sudden change in the embodiment of the present application can be considered to be the presence of a light-absorbing object in the target scene.

[0113] S404: Obtain a first confidence level of a light-absorbing object in the target scene based on a first scene image of the target scene.

[0114] Possibly, the embodiment of the present application can perform semantic recognition of the object on the first scene image of the target scene, thereby determining the two-dimensional position information, approximate shape information, size, and material, etc. of the light-absorbing type object in the first scene image. Furthermore, the relevant information of the identified light-absorbing type object can be compared with the relevant information of each category of light-absorbing type objects stored in the database, thereby providing at least one similar light-absorbing type object and its confidence.

[0115] For example, based on the first scene image of the target scene, it can be obtained that there are light-absorbing objects in the target scene. Specifically, the confidence that the light-absorbing object is a dark blue curtain is 0.2, the confidence that the light-absorbing object is a cashmere coat is 0.2, and the confidence that the light-absorbing object is a plush toy is 0.7.

[0116] S405, compare the first confidence of the light-absorbing type object in the target scene with the first type confidence threshold; when the first confidence of the light-absorbing type object in the target scene is greater than or equal to the first type confidence threshold, execute S406-S407; when the first confidence of the light-absorbing type object in the target scene is less than the first type confidence threshold, execute S408-S410.

[0117] The first type confidence threshold of the light-absorbing type object in the embodiment of the present application represents a value used to determine the specific category of the light-absorbing type object in the target scene. Possibly, the first type confidence threshold of the light-absorbing type object in the embodiment of the present application can be set to 0.7.

[0118] S406: Determine that the depth parameter corresponding to the target scene is the depth parameter corresponding to the light-absorbing type object in the first scene picture of the target scene.

[0119] It is understandable that the embodiment of the present application can determine the specific category of the light-absorbing type object in the target scene by comparing the first confidence with the first type confidence threshold. For example, when the first type confidence threshold is 0.7, the confidence that the light-absorbing type object in the target scene is a dark blue curtain is 0.2, the confidence that the light-absorbing type object is a cashmere coat is 0.2, and the confidence that the light-absorbing type object is a plush toy is 0.7, it can be determined that the confidence of the plush toy meets the requirements of the first type confidence threshold.

[0120] Furthermore, in the embodiment of the present application, the depth parameter corresponding to the light-absorbing type object stored in advance can be obtained through a database. For example, the depth parameter corresponding to the plush toy stored in advance can be obtained through a database.

[0121] S407 : Determine depth information of the light-absorbing type object in the target scene based on the depth parameter corresponding to the light-absorbing type object in the first scene picture of the target scene.

[0122] It can be understood that the embodiments of the present application can determine the depth parameter corresponding to the type of object according to the specific type of the light-absorbing object, that is, the filtering threshold and / or the number of neighborhood points.

[0123] Furthermore, the embodiments of the present application can filter the point cloud information of the light-absorbing type objects in the first scene image of the target scene based on the filtering threshold and / or the number of neighborhood points corresponding to this type of objects, so as to obtain the depth information of the light-absorbing type objects in the target scene.

[0124] S408: Lower the first type confidence threshold to a preset second type confidence threshold.

[0125] The second type confidence threshold is less than the first confidence of the light-absorbing type object in the target scene.

[0126] It is understandable that when the outgoing light emitted by the shooting device encounters a light-absorbing object with strong light-absorbing ability, this type of light-absorbing object may absorb most of the light energy. Therefore, the shooting device cannot obtain the reflected light information that can effectively identify the object. In other words, when it is impossible to determine the specific shape, size and other information of the light-absorbing object by semantically recognizing the first scene image of the target scene, the depth parameters corresponding to the target scene can be determined by directly lowering the first type confidence threshold to the preset second type confidence threshold.

[0127] Possibly, the embodiment of the present application can perform semantic recognition on the first scene image of the target scene, and then segment the objects in the first scene image, and only lower the confidence of the light-absorbing type objects, while the confidence of other non-light-absorbing objects remains unchanged. In this way, the specific categories of each object in the entire image can be effectively identified to obtain relatively accurate depth parameters.

[0128] S409: Determine the depth parameter corresponding to the second type confidence threshold as the depth parameter corresponding to the target scene.

[0129] It can be understood that the embodiment of the present application can pre-store the depth parameters (filter threshold and / or number of neighborhood points) corresponding to the second type of confidence threshold in the database of the device. When the first confidence of the light-absorbing type object in the target scene is less than the first type of confidence threshold, the filter threshold and / or number of neighborhood points corresponding to the pre-stored second type of confidence threshold is directly called to obtain the depth parameters corresponding to the target scene.

[0130] Specifically, due to the filtering threshold and / or the number of neighborhood points corresponding to the light-absorbing type object in the first scene image, the specific shape, size and other information of the light-absorbing type object cannot be obtained. Therefore, the embodiment of the present application needs to increase the sigma value of the filter and / or reduce the number of domain points to a predetermined value to increase the number of points in the point cloud. Specifically, increasing the sigma value of the filter can be understood as increasing the deviation of each point in the point cloud from the mean point. When the sigma value is increased, more feature points can be obtained in the point cloud to calculate the range of increasing the mean point. Reducing the number of neighborhood points can be understood as calculating the range of increasing the mean point by reducing the number of points in the point cloud within a preset range, thereby effectively increasing the identifiable point cloud range of the light-absorbing type object.

[0131] It is understandable that, in the embodiment of the present application, for the light-absorbing type object, it can be assumed that there is an obstacle by default, therefore, only the size of the obstacle and other information need to be determined without the specific category information of the obstacle. Therefore, the process of determining the depth parameter corresponding to the light-absorbing type object can be completed by lowering the confidence threshold.

[0132] S410: Determine depth information of a light-absorbing object in a target scene based on a depth parameter corresponding to a second type confidence threshold.

[0133] Specifically, the embodiment of the present application can use the adjusted depth parameters corresponding to the second type confidence threshold of the laser type objects in the target scene to filter the light-absorbing type objects in the second scene image of the target scene to obtain the depth information of the light-absorbing type objects in the target scene.

[0134] In a specific example, the sweeping robot can obtain a scene picture whose light intensity is within a preset range and does not contain objects of a preset type, and use two shooting devices to shoot the bedroom scene respectively to obtain two first scene pictures of the bedroom scene. Further, the common feature points of the same objects in the two first scene pictures can be marked, and the point cloud position information corresponding to each object in the first scene picture can be calculated according to the installation position of the shooting device. Furthermore, if the light intensity difference between the outgoing light emitted by the shooting device and the received reflected light between the scene picture of the initial scene and the first scene picture of the bedroom scene is greater than a second preset threshold, it can be determined that there is a light-absorbing type object in the bedroom scene. Furthermore, the first confidence level 0.5 of the multipath type object can be determined based on semantic recognition technology. Since the first confidence level 0.5 is less than the first type confidence threshold 0.7, the first type confidence threshold is lowered to the preset second type confidence threshold, and the depth parameter corresponding to the second type confidence threshold is obtained. Then, two shooting devices are used to shoot the target scene to determine the depth information of the light-absorbing type object in the bedroom scene, that is, the distance and position between the sweeping robot and the light-absorbing type object.

[0135] Therefore, when the light intensity difference between the outgoing light emitted by the shooting device and the received reflected light is greater than a second preset threshold, the embodiment of the present application determines the depth parameter of the target scene by determining whether the confidence of the light-absorbing type objects in the target scene meets the preset conditions, so as to reduce the factors that may cause inaccurate depth information detection due to the light-absorbing type objects in the target scene.

[0136] In some embodiments, the depth parameter may include any one or more of a shooting frequency of a shooting device, a filtering threshold, and a number of neighborhood points.

[0137] Figure 5 The flowchart of another method for determining depth information provided by an embodiment of the present application is exemplarily shown. Figure 5 As shown, the depth information determination method may at least include the following steps:

[0138] S501, obtaining a scene image of an initial scene.

[0139] Specifically, S501 is consistent with S201 and will not be described in detail here.

[0140] S502: Use at least one shooting device to shoot a target scene to obtain a first scene picture of the target scene.

[0141] Specifically, S502 is consistent with S202 and will not be described in detail here.

[0142] S503, when there is no first category change but there is a preset second category change between the scene picture of the initial scene and the first scene picture of the target scene, determine the type of each object in the scene picture of the target scene based on the first scene picture of the target scene.

[0143] Among them, the second category change in the embodiment of the present application indicates that the light intensity difference between the outgoing light emitted by at least one camera device and the reflected light received by at least one camera device is greater than or equal to the first preset threshold and less than or equal to the second preset threshold, the object position, and the object type. One or more changes have occurred.

[0144] Specifically, the second category of changes in the embodiment of the present application can be understood as the absence of multipath type objects and light-absorbing objects in the target scene. Therefore, the light intensity difference between the outgoing light emitted by the shooting device and the received reflected light can be between the first preset threshold and the second preset threshold.

[0145] Possibly, the presence of a preset second category of changes between the scene picture of the initial scene and the first scene picture of the target scene in the embodiment of the present application can be understood as a change in the scene location of the device, for example, moving from a bedroom scene to a kitchen scene; it can also be understood as a change in the light in the scene where the device is located, for example, in the evening, the ambient light intensity in the bedroom scene decreases; it can also be understood as a change in the position of the object in the scene due to human factors, for example, the user moves the cabinet out of the bedroom scene. It is understandable that the target scene in the embodiment of the present application may also have the above-mentioned multiple changes at the same time.

[0146] S504, performing object semantic segmentation on the first scene picture of the target scene to obtain the type of each object in the first scene picture of the target scene and a third confidence level of the type of each object in the first scene picture of the target scene.

[0147] Specifically, the embodiment of the present application can perform object semantic segmentation on the first scene picture of the target scene through semantic recognition technology to obtain the types of each object in the first scene picture of the target scene. For example, by using semantic recognition technology to perform semantic segmentation on the target scene, objects such as dining tables, chairs, and refrigerators may be obtained.

[0148] Furthermore, the embodiment of the present application can also compare the object information obtained by semantic segmentation with the object information stored in the database to obtain a third confidence level of the type of each object in the first scene image of the target scene. For example, the confidence level of the dining table is 0.8, the confidence level of the chair is 0.7, and the confidence level of the refrigerator is 0.75.

[0149] S505, compare the third confidence of the type of each object in the first scene picture of the target scene with the object type confidence threshold; when the third confidence of the type of each object in the first scene picture of the target scene is greater than or equal to the object type confidence threshold, execute S506-S511; when the third confidence of the type of each object in the first scene picture of the target scene is less than the object type confidence threshold, execute S512-S518.

[0150] The object type confidence threshold in the embodiment of the present application represents a numerical value for determining the probability of the object type.

[0151] S506: Determine a preset scene type of the target scene and a confidence level of the preset scene type of the target scene based on the type of each object in the first scene picture of the target scene.

[0152] It is understandable that the embodiment of the present application can determine the confidence of the preset scene type of the target scene by determining the preset scene type that each object in the first scene picture of the target scene may belong to. The preset scene type represents a scene type pre-stored in the device, such as a living room, a dining room, a bedroom, a bathroom, a kitchen, etc.

[0153] Possibly, assuming that the object type confidence threshold is 0.65, since the confidence of the dining table in the target scene is 0.8, the confidence of the chair is 0.7, and the confidence of the refrigerator is 0.75, all of which are greater than the object type confidence threshold, the preset scene type of the target scene can be further determined based on the type of each object in the target scene. For example, based on the objects such as the dining table, chair, and refrigerator that appear in the target scene, it can be determined that the confidence that the target scene belongs to the restaurant scene type is 0.9, the confidence that it belongs to the kitchen scene type is 0.8, and the confidence that it belongs to the living room scene type is 0.5.

[0154] S507, compare the confidence of the preset scene type of the target scene with the scene type confidence threshold. If the confidence of the preset scene type of the target scene is greater than or equal to the scene type confidence threshold, execute S508-S509; and execute S510-S511.

[0155] The scene type confidence threshold in the embodiment of the present application represents a numerical value for determining the scene type probability.

[0156] S508: Determine a depth parameter corresponding to the target scene based on a preset scene type of the target scene.

[0157] It can be understood that in an embodiment of the present application, if there are multiple preset scene types of the target scene whose confidence levels are greater than or equal to the scene type confidence threshold, then the preset scene type of the target scene is the preset scene type corresponding to the maximum confidence; and the depth parameter corresponding to the target scene is determined based on the preset scene type corresponding to the maximum confidence.

[0158] For example, assuming that the scene type confidence threshold is 0.75, when the confidence that the target scene belongs to the restaurant scene type is 0.9, the confidence that it belongs to the kitchen scene type is 0.8, and the confidence that it belongs to the living room scene type is 0.5, the preset scene type of the target scene is the preset scene type corresponding to the maximum confidence - the restaurant scene type. Further, the embodiment of the present application can obtain the depth parameters corresponding to the preset scene type - the restaurant scene type pre-stored in the database of the device.

[0159] S509: Determine depth information of each object in the target scene based on the depth parameter corresponding to the target scene.

[0160] It is understandable that the embodiment of the present application can use the pre-stored depth parameters corresponding to the preset scene type to filter the point cloud information of the first scene image of the target scene to obtain the depth information of each object in the target scene.

[0161] It is understandable that if the conditions are met, the depth parameter corresponding to the target scene is determined according to the depth parameter corresponding to the preset scene type. In this way, the depth parameter can be further adjusted according to different types of objects that may exist in the preset scene type (for example, there may be a large number of smooth utensils in the kitchen and a large number of wooden furniture in the living room) to obtain more accurate depth information of each object in the target scene.

[0162] S510: When the confidence of the preset scene type of the target scene is less than the scene type confidence threshold, the depth parameter corresponding to the third confidence of the type of each object in the first scene picture of the target scene is used as the depth parameter corresponding to the target scene.

[0163] It can be understood that in the embodiment of the present application, if the confidence levels of multiple preset scene types of the target scene are all less than the scene type confidence threshold, no depth parameter will be adjusted, and the depth parameter corresponding to the third confidence level of the type of each object in the first scene image of the target scene will be used as the depth parameter corresponding to the target scene. In other words, if the confidence levels of each object belonging to the preset scene do not meet the scene confidence threshold, that is, there is no suitable preset scene, the target scene parameter can be determined based on the current depth parameter (the depth parameter corresponding to the third confidence level).

[0164] For example, assuming that the scene type confidence threshold is 0.75, when the third confidence of each object in the target scene meets the object type confidence threshold, the confidence that the target scene belongs to the restaurant scene type is 0.2, the confidence that it belongs to the kitchen scene type is 0.2, and the confidence that it belongs to the living room scene type is 0.5. Since the confidences of the preset scene types of the target scenes are all less than the scene type confidence threshold, the depth parameters in the current device can be directly used as the depth parameters corresponding to the target scene.

[0165] S511 : Determine depth information of each object in the target scene based on a depth parameter corresponding to a third confidence level of a type of each object in a first scene picture of the target scene.

[0166] It can be understood that the embodiment of the present application can use the current depth parameters in the device (depth parameters corresponding to the third confidence level) to determine the target scene parameters to filter the point cloud information of the first scene image of the target scene to obtain the depth information of each object in the target scene.

[0167] S512: After adjusting any one or more of the depth parameters, photograph the target scene again to obtain a fourth confidence level of the type of the object in the second scene picture of the target scene.

[0168] It is understandable that, when the third confidence level of the type of each object in the first scene picture of the target scene is less than the object type confidence level threshold, the depth parameter needs to be adjusted.

[0169] Possibly, the embodiment of the present application can determine what changes exist between the target scene and the initial scene by comparing the picture of the initial scene with the first scene picture of the target scene, for example, the device is always in the same target scene, and only the ambient light in the target scene has changed. For example, at noon when the ambient light intensity in the room is high on a sunny day, the modulation frequency can be appropriately increased to prevent overexposure, and then the target scene is photographed again to obtain a fourth confidence level of the type of object in the second scene picture of the target scene; at dusk when the ambient light intensity in the room decreases, the modulation frequency can be appropriately reduced to improve the stability of the device measurement, and then the target scene is photographed again to obtain a fourth confidence level of the type of object in the second scene picture of the target scene. Among them, the modulation frequency represents the phase difference between the emitted light and the received light, which can be understood as the switching frequency of the shutter of the shooting device.

[0170] For another example, the device is always in the same target scene, but the ambient light and the positions of multiple objects in the target scene have changed. For example, at noon on a sunny day, the indoor ambient light intensity is high and the user moves the table in the target scene and adds a bookshelf in front of the background wall. Accordingly, the modulation frequency of the shooting device can be appropriately increased and the sigma value of the filter can be reduced or the number of domain neighborhood points can be increased to more effectively filter out flying points, and then the target scene can be shot again to obtain the fourth confidence of the type of the object in the second scene picture of the target scene.

[0171] It is understandable that, since scattered points are likely to appear at the edge between the foreground (the object in the target scene) and the background wall, the depth parameter needs to be adjusted to effectively distinguish the target scene containing the foreground and the background.

[0172] S513, compare the fourth confidence of the type of the object in the second scene picture of the target scene with the object type confidence threshold. If the fourth confidence of the type of the object in the second scene picture of the target scene is less than the object type confidence threshold, execute S514-S515; if the fourth confidence of the type of the object in the second scene picture of the target scene is greater than or equal to the object type confidence threshold, execute S516 and then return to S507 and execute S508-S511.

[0173] It is understandable that, since the target scene needs to be photographed again after adjusting the depth parameters, the embodiment of the present application determines the fourth confidence level of the type of each object in the second scene picture of the target scene obtained by photographing again.

[0174] S514, after adjusting any one or more of the depth parameters, photograph the target scene again to obtain a fifth confidence level of the type of the object in the second scene picture of the target scene.

[0175] Among them, the second scene picture in the embodiment of the present application represents a picture regenerated by the shooting device shooting the target scene again.

[0176] It is understandable that, since the shooting device re-shoots the target scene, it is necessary to re-determine the probability of the type of the object in the second scene picture of the target scene to obtain the fifth confidence level of the type of the object in the second scene picture of the target scene.

[0177] S515, compare the fifth confidence of the type of the object in the second scene picture of the target scene with the object type confidence threshold; if the fifth confidence of the type of the object in the second scene picture of the target scene is greater than or equal to the object type confidence threshold, execute S516-S518; if the fifth confidence of the type of the object in the second scene picture of the target scene is less than the object type confidence threshold, return to S514.

[0178] It is understandable that the embodiment of the present application needs to determine whether the depth parameter adjustment is completed, and therefore needs to re-judge the size between the fifth confidence of the type of the object in the second scene picture of the target scene and the object type confidence threshold.

[0179] S516: Determine a preset scene type of the target scene and a confidence level of the preset scene type of the target scene based on the type of the object in the second scene picture of the target scene.

[0180] It can be understood that in the embodiment of the present application, when the fifth confidence of the type of an object in the second scene picture of the target scene is greater than or equal to the object type confidence threshold, the preset scene type of the target scene and the confidence of the preset scene type of the target scene can be re-determined based on the type of the object in the second scene picture, thereby completing the adjustment process of the depth parameters corresponding to the target scene, returning to S507 and executing S508-S511 to determine the depth information of each object in the target scene based on the depth parameters corresponding to the target scene.

[0181] In a specific example, the difference in light intensity between the outgoing light emitted by the shooting device and the received reflected light between the scene picture of the initial scene and the first scene picture of the target scene that can be acquired by the sweeping robot is between the first preset threshold and the second preset threshold, and the types of objects in the scene picture of the target scene can be determined based on the first scene picture of the target scene. Specifically, the first scene picture of the target scene can be semantically segmented to obtain the confidences of the objects in the first scene picture of the target scene, which are the third confidence of 0.8 for the bed, 0.7 for the wardrobe, 0.5 for the cabinet, and 0.6 for the plant. Since the confidence of 0.5 for the cabinet and 0.6 for the plant in the first scene picture of the target scene are less than the object type confidence threshold of 0.65, the depth parameters of the device need to be adjusted. Specifically, if the ambient light intensity in the target scene has not changed, only the sigma value of the filter and / or the number of domain neighborhood points can be adjusted, and the target scene can be photographed again to obtain the confidence of the objects in the second scene picture of the target scene, which are the third confidence of 0.85 for the bed, 0.75 for the wardrobe, 0.7 for the bookshelf, and 0.65 for the green plants. Since the confidence of each object in the second scene picture of the target scene is greater than or equal to the object type confidence threshold of 0.65, it is no longer necessary to adjust the depth parameters of the device. Based on the type of each object in the second scene picture of the target scene, it can be determined that the confidence of the target scene belonging to the bedroom scene type is 0.85, the confidence of the living room scene type is 0.7, and the confidence of the study scene type is 0.3. Since the confidence of the bedroom scene type of 0.85 is significantly greater than the preset scene type confidence of 0.75, the depth parameters can be further adjusted according to the bedroom scene type to adapt to the impact of the materials of each object in the bedroom scene on the detection performance. After obtaining the depth parameters corresponding to the bedroom scene in the database of the device, the second scene image can be filtered based on the depth parameters to obtain information such as the distance and position between the sweeping robot and each object in the bedroom scene.

[0182] Therefore, in the embodiment of the present application, when the light intensity difference between the outgoing light emitted by the shooting device and the received reflected light is between the first preset threshold and the second preset threshold, the depth parameter of the target scene can be determined by determining whether the confidence of the preset scene type corresponding to the target scene meets the preset conditions, so as to adaptively adjust the depth information of each object in different target scenes to make the detected depth information more accurate.

[0183] Figure 6 1 is a schematic diagram of a method and device for determining depth information provided by an exemplary embodiment of the present application. The method for determining depth information is used for a device that needs to determine depth information, and the device includes at least one shooting device. Figure 6 As shown, the depth information determination method and device includes:

[0184] An acquisition module 61 is used to acquire a scene image of an initial scene; wherein the initial scene represents a scene whose light intensity is within a preset range and does not contain objects of a preset type;

[0185] An obtaining module 62 is used to use the at least one shooting device to shoot the target scene to obtain a first scene image of the target scene;

[0186] A first determination module 63 is configured to determine that an object of the preset type exists in the target scene when there is a preset first category change between the scene image of the initial scene and the first scene image of the target scene; wherein the first category change indicates that a light intensity difference between the outgoing light emitted by the at least one camera and the reflected light received by the at least one camera is greater than a first preset threshold or less than a second preset threshold;

[0187] A second determination module 64 is used to determine a depth parameter corresponding to the target scene based on the preset type of object in the target scene; wherein the depth parameter represents a parameter for adjusting the depth information;

[0188] The third determining module 65 is used to determine the depth information of at least one object in the target scene based on the depth parameter corresponding to the target scene.

[0189] In summary, in the embodiments of the present application, a scene picture of an initial scene can be acquired; at least one shooting device is used to shoot the target scene to obtain a first scene picture of the target scene; when there is a preset first category change between the scene picture of the initial scene and the first scene picture of the target scene, it is determined that there is a preset type of object in the target scene; based on the preset type of object in the target scene, a depth parameter corresponding to the target scene is determined; and depth information of at least one object in the target scene is determined based on the depth parameter corresponding to the target scene. Therefore, in the embodiments of the present application, a shooting device can be used to shoot the target scene, and depth parameters corresponding to the target scene can be determined based on the preset type of object in the first scene picture of the target scene. Thereby, the depth parameters that may exist in different scenes and affect the accuracy of the depth information of the detected object are adjusted to solve the problem of inaccurate depth information detected by the depth sensor in different scenes using the same depth parameter in the related art. In some possible embodiments,

[0190] In some embodiments, the preset type of object includes a multipath type object; the depth parameter includes: a filtering threshold and / or a number of neighborhood points;

[0191] The first determination module 63 is specifically configured to: determine that the multipath type object exists in the target scene when the first category change between the scene picture of the initial scene and the first scene picture of the target scene is that the light intensity difference between the outgoing light emitted by the at least one shooting device and the received reflected light is less than a first preset threshold;

[0192] Before the second determining module 64, the device further includes:

[0193] A first confidence obtaining module, used to obtain a first confidence of a multipath type object in the target scene based on a first scene image of the target scene;

[0194] The second determination module 64 is specifically configured to: when the first confidence of the multipath type object in the target scene is greater than or equal to the first type confidence threshold, determine that the depth parameter corresponding to the target scene is the depth parameter corresponding to the multipath type object in the first scene picture of the target scene;

[0195] The third determination module 65 is specifically configured to determine the depth information of the multipath type object in the target scene based on the depth parameter corresponding to the multipath type object in the first scene picture of the target scene.

[0196] In some implementations, the second determining module 64 includes:

[0197] A second confidence obtaining unit is used to, when the first confidence of the multipath type object in the target scene is less than the first type confidence threshold, adjust the filter threshold and / or the number of neighborhood points in the depth parameter, and then photograph the target scene again to obtain a second confidence of the multipath type object in a second scene picture of the target scene;

[0198] A first depth parameter determining unit, configured to determine, if a second confidence of the multipath type object in the second scene picture of the target scene is greater than or equal to the first type confidence threshold, that the depth parameter corresponding to the target scene is a depth parameter corresponding to the second confidence of the multipath type object in the second scene picture of the target scene;

[0199] A first adjustment unit is used for, if the second confidence of the multipath type object in the second scene picture of the target scene is still less than the first type confidence threshold, returning to the adjustment of the filter threshold and / or the number of neighborhood points in the depth parameter, and photographing the target scene again until the second confidence of the multipath type object in the second scene picture of the target scene is greater than or equal to the first type confidence threshold;

[0200] The third determination module 65 is specifically configured to determine the depth information of the multipath type object in the target scene based on a depth parameter corresponding to a second confidence level of the multipath type object in the second scene image of the target scene.

[0201] In some embodiments, the preset type of object includes a light-absorbing type object; the depth parameter includes: a filtering threshold and / or a number of neighborhood points;

[0202] The first determining module 63 is specifically configured to:

[0203] The first category change between the scene picture of the initial scene and the first scene picture of the target scene is that the light intensity difference between the outgoing light emitted by the at least one shooting device and the received reflected light is greater than a second preset threshold, and it is determined that the light absorbing type object exists in the target scene;

[0204] Before the second determining module 64, the device further includes:

[0205] A first confidence determination module, configured to obtain a first confidence of a light-absorbing object in the target scene based on a first scene image of the target scene;

[0206] The second determination module 64 is specifically configured to: when the first confidence of the light-absorbing type object in the target scene is greater than or equal to the first type confidence threshold, determine that the depth parameter corresponding to the target scene is the depth parameter corresponding to the light-absorbing type object in the first scene picture of the target scene;

[0207] The third determination module 65 is specifically configured to determine the depth information of the light-absorbing type object in the target scene based on the depth parameter corresponding to the light-absorbing type object in the first scene image of the target scene.

[0208] In some implementations, the second determining module 64 includes:

[0209] A threshold adjustment unit, configured to reduce the first type confidence threshold to a preset second type confidence threshold when the first confidence of the light absorption type object in the target scene is less than the first type confidence threshold; wherein the second type confidence threshold is less than the first confidence of the light absorption type object in the target scene;

[0210] A first determining unit, configured to determine a depth parameter corresponding to the second type confidence threshold as a depth parameter corresponding to the target scene;

[0211] The third determination module 65 is specifically configured to determine the depth information of the light-absorbing type object in the target scene based on the depth parameter corresponding to the second type confidence threshold.

[0212] In some implementations, before the third determining module 65, the apparatus further includes:

[0213] An object type determination module, configured to determine the type of each object in the scene picture of the target scene based on the first scene picture of the target scene, when there is no first category change between the scene picture of the initial scene and the first scene picture of the target scene but there is a preset second category change; wherein the second category change indicates that one or more of the following changes have occurred: a difference in light intensity between the outgoing light emitted by the at least one camera and the reflected light received by the at least one camera is greater than or equal to the first preset threshold and less than or equal to the second preset threshold, an object position, and an object type;

[0214] A scene type confidence module, configured to determine a preset scene type of the target scene and a confidence of the preset scene type of the target scene based on the type of each object in the scene picture of the target scene, when a third confidence of the type of each object in the first scene picture of the target scene is greater than or equal to an object type confidence threshold;

[0215] A first depth parameter determination module, configured to determine a depth parameter corresponding to the target scene based on the preset scene type of the target scene when the confidence of the preset scene type of the target scene is greater than or equal to a scene type confidence threshold;

[0216] The third determination module 65 is specifically configured to determine the depth information of each object in the target scene based on the depth parameter corresponding to the target scene.

[0217] In some implementations, before the scene type confidence module, the device further includes:

[0218] The third confidence obtaining module is used to perform object semantic segmentation on the first scene picture of the target scene to obtain the type of each object in the first scene picture of the target scene and the third confidence of the type of each object in the first scene picture of the target scene.

[0219] In some implementations, the first depth parameter determination module includes:

[0220] a maximum confidence determination unit, configured to determine, if there are multiple preset scene types of the target scene whose confidences are all greater than or equal to a scene type confidence threshold, the preset scene type of the target scene is the preset scene type corresponding to the maximum confidence;

[0221] The second depth parameter determining unit is configured to determine the depth parameter corresponding to the target scene based on the preset scene type corresponding to the maximum confidence level.

[0222] In some implementations, after the scene type confidence module and before the third determination module 65, the device further includes:

[0223] A second depth parameter determination module is used to use the depth parameter corresponding to the third confidence of the type of each object in the first scene picture of the target scene as the depth parameter corresponding to the target scene when the confidence of the preset scene type of the target scene is less than the scene type confidence threshold;

[0224] The third determination module 65 is specifically configured to determine the depth information of each object in the target scene based on a depth parameter corresponding to a third confidence level of the type of each object in the first scene image of the target scene.

[0225] In some embodiments, the depth parameter includes any one or more of a shooting frequency of a shooting device, a filtering threshold, and a number of neighborhood points;

[0226] Before the third determining module 65, the device further includes:

[0227] A fourth confidence obtaining module is used to, when the third confidence of the type of each object in the first scene picture of the target scene is less than the object type confidence threshold, adjust any one or more of the depth parameters, and then photograph the target scene again to obtain a fourth confidence of the type of the object in the second scene picture of the target scene;

[0228] The first preset scene type determination module is used to determine the preset scene type of the target scene based on the type of the object in the second scene picture of the target scene if the fourth confidence of the type of the object in the second scene picture of the target scene is greater than or equal to the object type confidence threshold.

[0229] In some implementations, before the third determining module 65, the apparatus further includes:

[0230] a fifth confidence obtaining module, configured to, when a fourth confidence of the type of each object in the scene picture of the target scene is less than the object type confidence threshold, adjust any one or more of the depth parameters, photograph the target scene again, and obtain a fifth confidence of the type of the object in the second scene picture of the target scene;

[0231] A second preset scene type determination module, configured to determine the preset scene type of the target scene based on the type of the object in the second scene picture of the target scene if the fifth confidence of the type of the object in the second scene picture of the target scene is greater than or equal to the object type confidence threshold;

[0232] A second adjustment unit is used to return to the step of adjusting any one or more of the depth parameters and photographing the target scene again if the fifth confidence of the type of the object in the second scene picture of the target scene is still less than the object type confidence threshold, until the fifth confidence of the type of the object in the second scene picture of the target scene is greater than or equal to the object type confidence threshold.

[0233] In some embodiments, the apparatus includes a plurality of camera devices;

[0234] The obtaining module 62 is specifically used to: use the multiple shooting devices to shoot the target scene respectively, and each shooting device outputs at least one first scene picture of the target scene;

[0235] The third determination module 65 is specifically used to process at least one first scene picture of the target scene output by each shooting device based on the depth parameter corresponding to the target scene to determine the depth information of at least one object in the target scene.

[0236] In some embodiments, the device includes a camera and a depth detection sensor;

[0237] The third determination module 65 is specifically configured to input the depth parameter of the target scene and the first scene image of the target scene into the depth detection sensor to obtain depth information of at least one object in the target scene.

[0238] It should be noted that the depth information determination method device provided in the above embodiment only uses the division of the above functional modules as an example when executing the depth information determination method. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device for improving the laser ranging capability of the radar system provided in the above embodiment and the method embodiment for improving the laser ranging capability of the radar system belong to the same concept. The implementation process is detailed in the method embodiment and will not be repeated here.

[0239] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0240] See also Figure 7, is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 7 As shown, the electronic device 70 may include: at least one processor 71 , at least one network interface 74 , a user interface 73 , a memory 75 , and at least one communication bus 72 .

[0241] The communication bus 72 is used to realize the connection and communication between these components.

[0242] The user interface 73 may include a display screen (Display) and a camera (Camera), and the optional user interface 73 may also include a standard wired interface and a wireless interface.

[0243] The network interface 74 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0244] Among them, the processor 71 may include one or more processing cores. The processor 71 uses various interfaces and lines to connect various parts of the entire electronic device 70, and executes various functions and processes data of the electronic device 70 by running or executing instructions, programs, code sets or instruction sets stored in the memory 75, and calling data stored in the memory 75. Optionally, the processor 71 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 71 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 71, and can be implemented separately through a chip.

[0245] Among them, the memory 75 may include a random access memory (Random Access Memory, RAM) and may also include a read-only memory (Read-Only Memory). Optionally, the memory 75 includes a non-transitory computer-readable storage medium. The memory 75 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 75 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 75 may also be optionally at least one storage device located away from the aforementioned processor 71. As Figure 7 As shown, the memory 75 as a computer storage medium may include an operating system, a network communication module, a user interface module, and an application program for improving the laser ranging capability of the radar system.

[0246] exist Figure 7 In the electronic device 70 shown, the user interface 73 is mainly used to provide an input interface for the user and obtain the data input by the user; and the processor 71 can be used to call the application program for improving the laser ranging capability of the radar system stored in the memory 75, and specifically perform the following operations:

[0247] Acquire a scene image of an initial scene; wherein the initial scene represents a scene whose light intensity is within a preset range and does not contain objects of a preset type;

[0248] Using the at least one shooting device to shoot a target scene to obtain a first scene picture of the target scene;

[0249] In a case where there is a preset first category change between the scene picture of the initial scene and the first scene picture of the target scene, determining that there is an object of the preset type in the target scene; wherein the first category change indicates that a light intensity difference between the outgoing light emitted by the at least one shooting device and the reflected light received by the at least one shooting device is less than a first preset threshold or greater than a second preset threshold;

[0250] Determining a depth parameter corresponding to the target scene based on the preset type of object in the target scene; wherein the depth parameter represents a parameter for adjusting the depth information;

[0251] Determine depth information of at least one object in the target scene based on the depth parameter corresponding to the target scene.

[0252] In a possible embodiment, the preset type of object includes a multipath type object; the depth parameter includes: a filtering threshold and / or a number of neighborhood points;

[0253] When the processor 71 determines that there is an object of the preset type in the target scene when there is a preset first category change between the scene picture of the initial scene and the first scene picture of the target scene, the processor 71 specifically executes:

[0254] When the first category change between the scene picture of the initial scene and the first scene picture of the target scene is that the light intensity difference between the outgoing light emitted by the at least one shooting device and the received reflected light is less than a first preset threshold, determining that the multipath type object exists in the target scene;

[0255] Before executing the step of determining the depth parameter corresponding to the target scene based on the object of the preset type in the target scene, the processor 71 further executes:

[0256] Obtaining a first confidence level of a multipath type object in the target scene based on a first scene picture of the target scene;

[0257] When the processor 71 determines the depth parameter corresponding to the target scene based on the object of the preset type in the target scene, the processor 71 specifically performs:

[0258] When the first confidence of the multipath type object in the target scene is greater than or equal to the first type confidence threshold, determining the depth parameter corresponding to the target scene as the depth parameter corresponding to the multipath type object in the first scene picture of the target scene;

[0259] When the processor 71 determines the depth information of at least one object in the target scene based on the depth parameter corresponding to the target scene, the processor 71 specifically performs:

[0260] Determine depth information of the multipath type object in the target scene based on a depth parameter corresponding to the multipath type object in the first scene picture of the target scene.

[0261] In a possible embodiment, when the processor 71 determines the depth parameter corresponding to the target scene based on the object of the preset type in the target scene, the processor 71 specifically performs:

[0262] When the first confidence of the multipath type object in the target scene is less than the first type confidence threshold, adjusting the filter threshold and / or the number of neighborhood points in the depth parameter, photographing the target scene again to obtain a second confidence of the multipath type object in a second scene picture of the target scene;

[0263] If the second confidence of the multipath type object in the second scene picture of the target scene is greater than or equal to the first type confidence threshold, determining the depth parameter corresponding to the target scene as the depth parameter corresponding to the second confidence of the multipath type object in the second scene picture of the target scene;

[0264] If the second confidence of the multipath type object in the second scene picture of the target scene is still less than the first type confidence threshold, returning to the step of adjusting the filter threshold and / or the number of neighborhood points in the depth parameter, and photographing the target scene again until the second confidence of the multipath type object in the second scene picture of the target scene is greater than or equal to the first type confidence threshold;

[0265] When the processor 71 determines the depth information of at least one object in the target scene based on the depth parameter corresponding to the target scene, the processor 71 specifically performs:

[0266] Determine the depth information of the multipath type object in the target scene based on a depth parameter corresponding to a second confidence level of the multipath type object in a second scene picture of the target scene.

[0267] In a possible embodiment, the preset type of object includes a light-absorbing type object; the depth parameter includes: a filtering threshold and / or a number of neighborhood points;

[0268] When the processor 71 determines that there is an object of the preset type in the target scene when there is a preset first category change between the scene picture of the initial scene and the first scene picture of the target scene, the processor 71 specifically executes:

[0269] The first category change between the scene picture of the initial scene and the first scene picture of the target scene is that the light intensity difference between the outgoing light emitted by the at least one shooting device and the received reflected light is greater than a second preset threshold, and it is determined that the light absorbing type object exists in the target scene;

[0270] Before executing the step of determining the depth parameter corresponding to the target scene based on the object of the preset type in the target scene, the processor 71 further executes:

[0271] Obtaining a first confidence level of a light-absorbing object in the target scene based on a first scene image of the target scene;

[0272] When the processor 71 determines the depth parameter corresponding to the target scene based on the object of the preset type in the target scene, the processor 71 specifically performs:

[0273] When the first confidence of the light-absorbing type object in the target scene is greater than or equal to the first type confidence threshold, determining the depth parameter corresponding to the target scene to be the depth parameter corresponding to the light-absorbing type object in the first scene picture of the target scene;

[0274] When the processor 71 determines the depth information of at least one object in the target scene based on the depth parameter corresponding to the target scene, the processor 71 specifically performs:

[0275] Determine depth information of the light-absorbing type object in the target scene based on a depth parameter corresponding to the light-absorbing type object in the first scene picture of the target scene.

[0276] In a possible embodiment, when the processor 71 determines the depth parameter corresponding to the target scene based on the object of the preset type in the target scene, the processor 71 specifically performs:

[0277] When the first confidence level of the light-absorbing type object in the target scene is less than the first type confidence level threshold, lowering the first type confidence level threshold to a preset second type confidence level threshold; wherein the second type confidence level threshold is less than the first confidence level of the light-absorbing type object in the target scene;

[0278] Determining the depth parameter corresponding to the second type confidence threshold as the depth parameter corresponding to the target scene;

[0279] When the processor 71 determines the depth information of at least one object in the target scene based on the depth parameter corresponding to the target scene, the processor 71 specifically performs:

[0280] Determine the depth information of the light-absorbing type object in the target scene based on the depth parameter corresponding to the second type confidence threshold.

[0281] In a possible embodiment, before executing the step of determining the depth information of at least one object in the target scene based on the depth parameter corresponding to the target scene, the processor 71 further executes:

[0282] In a case where there is no first-category change but there is a preset second-category change between the scene picture of the initial scene and the first scene picture of the target scene, determining the type of each object in the scene picture of the target scene based on the first scene picture of the target scene; wherein the second-category change indicates that one or more of the difference in light intensity between the outgoing light emitted by the at least one shooting device and the reflected light received by the at least one shooting device is greater than or equal to the first preset threshold and less than or equal to the second preset threshold, the position of the object, and the type of the object changes;

[0283] When a third confidence level of the type of each object in the first scene picture of the target scene is greater than or equal to the object type confidence threshold, determining a preset scene type of the target scene and a confidence level of the preset scene type of the target scene based on the type of each object in the scene picture of the target scene;

[0284] When the confidence of the preset scene type of the target scene is greater than or equal to the scene type confidence threshold, determining the depth parameter corresponding to the target scene based on the preset scene type of the target scene;

[0285] When the processor 71 determines the depth information of at least one object in the target scene based on the depth parameter corresponding to the target scene, the processor 71 specifically performs:

[0286] Depth information of each object in the target scene is determined based on the depth parameter corresponding to the target scene.

[0287] In a possible embodiment, before executing the condition that the third confidence of the type of each object in the first scene picture of the target scene is greater than or equal to the object type confidence threshold, the processor 71 further executes:

[0288] Object semantic segmentation is performed on the first scene picture of the target scene to obtain the type of each object in the first scene picture of the target scene and a third confidence level of the type of each object in the first scene picture of the target scene.

[0289] In a possible embodiment, when the processor 71 determines the depth parameter corresponding to the target scene based on the preset scene type of the target scene when the confidence of the preset scene type of the target scene is greater than or equal to the scene type confidence threshold, the processor 71 specifically performs:

[0290] If there are multiple preset scene types of the target scene whose confidences are all greater than or equal to the scene type confidence threshold, the preset scene type of the target scene is the preset scene type corresponding to the maximum confidence;

[0291] A depth parameter corresponding to the target scene is determined based on the preset scene type corresponding to the maximum confidence.

[0292] In a possible embodiment, the processor 71, after executing the process in which the third confidence of the type of each object in the first scene picture of the target scene is greater than or equal to the object type confidence threshold, and before determining the depth information of each object in the target scene based on the depth parameter of the target scene, further executes:

[0293] When the confidence of the preset scene type of the target scene is less than the scene type confidence threshold, using the depth parameter corresponding to the third confidence of the type of each object in the first scene picture of the target scene as the depth parameter corresponding to the target scene;

[0294] When the processor 71 determines the depth information of each object in the target scene based on the depth parameter corresponding to the target scene, the processor 71 specifically performs:

[0295] Determine depth information of each object in the target scene based on a depth parameter corresponding to a third confidence level of a type of each object in the first scene picture of the target scene.

[0296] In a possible embodiment, the depth parameter includes any one or more of a shooting frequency of a shooting device, a filtering threshold, and a number of neighborhood points;

[0297] Before executing the step of determining the depth information of each object in the target scene based on the depth parameter corresponding to the target scene, the processor 71 further executes:

[0298] When the third confidence of the type of each object in the first scene picture of the target scene is less than the object type confidence threshold, after adjusting any one or more of the depth parameters, the target scene is photographed again to obtain a fourth confidence of the type of the object in the second scene picture of the target scene;

[0299] If the fourth confidence of the type of the object in the second scene picture of the target scene is greater than or equal to the object type confidence threshold, the preset scene type of the target scene is determined based on the type of the object in the second scene picture of the target scene.

[0300] In a possible embodiment, before the processor 71 determines the depth information of each object in the target scene based on the depth parameter corresponding to the target scene, it further executes:

[0301] When a fourth confidence level of the type of each object in the scene picture of the target scene is less than the object type confidence level threshold, after adjusting any one or more of the depth parameters, the target scene is photographed again to obtain a fifth confidence level of the type of the object in a second scene picture of the target scene;

[0302] If the fifth confidence level of the type of the object in the second scene picture of the target scene is greater than or equal to the object type confidence threshold, the type of the object in the second scene picture of the target scene determines the preset scene type of the target scene;

[0303] If the fifth confidence of the type of object in the second scene picture of the target scene is still less than the object type confidence threshold, return to the step of adjusting any one or more of the depth parameters and photograph the target scene again until the fifth confidence of the type of object in the second scene picture of the target scene is greater than or equal to the object type confidence threshold.

[0304] In a possible embodiment, the device includes a plurality of photographing devices;

[0305] When the processor 71 executes the step of photographing the target scene using the at least one photographing device to obtain a scene image of the target scene, the processor 71 specifically executes:

[0306] Using the multiple shooting devices to shoot the target scene respectively, each shooting device outputs at least one first scene picture of the target scene;

[0307] When the processor 71 determines the depth information of at least one object in the target scene based on the depth parameter corresponding to the target scene, the processor 71 specifically performs:

[0308] At least one first scene picture of the target scene output by each shooting device is processed based on the depth parameter corresponding to the target scene to determine the depth information of at least one object in the target scene.

[0309] In a possible embodiment, the device includes a photographing device and a depth detection sensor;

[0310] When the processor 71 determines the depth information of at least one object in the target scene based on the depth parameter corresponding to the target scene, the processor 71 specifically performs:

[0311] The depth parameter corresponding to the target scene and the first scene picture of the target scene are input into the depth detection sensor to obtain the depth information of at least one object in the target scene.

[0312] The embodiment of the present application also provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed on a computer or a processor, the computer or the processor executes the above-mentioned Figure 2a , Figure 3-Figure 5 One or more steps in the illustrated embodiment. If the components of the above-mentioned device for improving the laser ranging capability of the radar system are implemented in the form of software functional units and sold or used as independent products, they can be stored in the computer-readable storage medium.

[0313] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted by the computer-readable storage medium. The computer instructions can be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (Digital Subscriber Line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server, a data center, etc. that contains one or more available media integrated. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a digital versatile disc (DVD)), or a semiconductor medium (eg, a solid state disk (SSD)).

[0314] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The aforementioned storage medium includes: various media that can store program codes, such as system memory (Read Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk. In the absence of conflict, the technical features in this embodiment and the implementation scheme can be combined arbitrarily.

[0315] The embodiments described above are merely preferred embodiments of the present application and are not intended to limit the scope of the present application. Without departing from the design spirit of the present application, various modifications and improvements made to the technical solutions of the present application by ordinary technicians in this field should fall within the protection scope determined by the claims of the present application.

Claims

1. A method for determining depth information, characterized in that: The depth information determination method is used for a device that needs to determine depth information, and the device includes at least one shooting device; Acquire a scene image of an initial scene; wherein the initial scene represents a scene whose light intensity is within a preset range and does not contain objects of a preset type; Using the at least one shooting device to shoot a target scene to obtain a first scene picture of the target scene; In a case where there is a preset first category change between the scene picture of the initial scene and the first scene picture of the target scene, determining that there is an object of the preset type in the target scene; wherein the first category change indicates that a light intensity difference between the outgoing light emitted by the at least one shooting device and the reflected light received by the at least one shooting device is less than a first preset threshold or greater than a second preset threshold; Determining a depth parameter corresponding to the target scene based on the preset type of object in the target scene; wherein the depth parameter represents a parameter for adjusting the depth information; Determine depth information of at least one object in the target scene based on the depth parameter corresponding to the target scene.

2. The method according to claim 1, characterized in that The preset type of object includes a multipath type object; the depth parameter includes: a filtering threshold and / or the number of neighborhood points; In the case where there is a preset first category change between the scene picture of the initial scene and the first scene picture of the target scene, determining that there is an object of the preset type in the target scene includes: When the first category change between the scene picture of the initial scene and the first scene picture of the target scene is that the light intensity difference between the outgoing light emitted by the at least one shooting device and the received reflected light is less than a first preset threshold, determining that the multipath type object exists in the target scene; Before determining the depth parameter corresponding to the target scene based on the object of the preset type in the target scene, the method further includes: Obtaining a first confidence level of a multipath type object in the target scene based on a first scene picture of the target scene; The determining the depth parameter corresponding to the target scene based on the object of the preset type in the target scene includes: When the first confidence of the multipath type object in the target scene is greater than or equal to the first type confidence threshold, determining the depth parameter corresponding to the target scene as the depth parameter corresponding to the multipath type object in the first scene picture of the target scene; The determining the depth information of at least one object in the target scene based on the depth parameter corresponding to the target scene includes: Determine depth information of the multipath type object in the target scene based on a depth parameter corresponding to the multipath type object in the first scene picture of the target scene.

3. The method according to claim 2, characterized in that The determining the depth parameter corresponding to the target scene based on the object of the preset type in the target scene includes: When the first confidence of the multipath type object in the target scene is less than the first type confidence threshold, adjusting the filter threshold and / or the number of neighborhood points in the depth parameter, photographing the target scene again to obtain a second confidence of the multipath type object in a second scene picture of the target scene; If the second confidence of the multipath type object in the second scene picture of the target scene is greater than or equal to the first type confidence threshold, determining the depth parameter corresponding to the target scene as the depth parameter corresponding to the second confidence of the multipath type object in the second scene picture of the target scene; If the second confidence of the multipath type object in the second scene picture of the target scene is still less than the first type confidence threshold, returning to the step of adjusting the filter threshold and / or the number of neighborhood points in the depth parameter, and photographing the target scene again until the second confidence of the multipath type object in the second scene picture of the target scene is greater than or equal to the first type confidence threshold; The determining the depth information of at least one object in the target scene based on the depth parameter corresponding to the target scene includes: Determine the depth information of the multipath type object in the target scene based on a depth parameter corresponding to a second confidence level of the multipath type object in a second scene picture of the target scene.

4. The method according to claim 1, characterized in that The preset type of object includes a light-absorbing type object; the depth parameter includes: a filtering threshold and / or the number of neighborhood points; In the case where there is a preset first category change between the scene picture of the initial scene and the first scene picture of the target scene, determining that there is an object of the preset type in the target scene includes: The first category change between the scene picture of the initial scene and the first scene picture of the target scene is that the light intensity difference between the outgoing light emitted by the at least one shooting device and the received reflected light is greater than a second preset threshold, and it is determined that the light absorbing type object exists in the target scene; Before determining the depth parameter corresponding to the target scene based on the object of the preset type in the target scene, the method further includes: Obtaining a first confidence level of a light-absorbing object in the target scene based on a first scene image of the target scene; The determining the depth parameter corresponding to the target scene based on the object of the preset type in the target scene includes: When the first confidence of the light-absorbing type object in the target scene is greater than or equal to the first type confidence threshold, determining the depth parameter corresponding to the target scene to be the depth parameter corresponding to the light-absorbing type object in the first scene picture of the target scene; The determining the depth information of at least one object in the target scene based on the depth parameter corresponding to the target scene includes: Determine depth information of the light-absorbing type object in the target scene based on a depth parameter corresponding to the light-absorbing type object in the first scene picture of the target scene.

5. The method according to claim 4, characterized in that The determining the depth parameter corresponding to the target scene based on the object of the preset type in the target scene includes: When the first confidence level of the light-absorbing type object in the target scene is less than the first type confidence level threshold, lowering the first type confidence level threshold to a preset second type confidence level threshold; wherein the second type confidence level threshold is less than the first confidence level of the light-absorbing type object in the target scene; Determining the depth parameter corresponding to the second type confidence threshold as the depth parameter corresponding to the target scene; The determining the depth information of at least one object in the target scene based on the depth parameter corresponding to the target scene includes: Determine the depth information of the light-absorbing type object in the target scene based on the depth parameter corresponding to the second type confidence threshold.

6. The method according to claim 1, characterized in that Before determining the depth information of at least one object in the target scene based on the depth parameter corresponding to the target scene, the method further includes: In a case where there is no first-category change but there is a preset second-category change between the scene picture of the initial scene and the first scene picture of the target scene, determining the type of each object in the scene picture of the target scene based on the first scene picture of the target scene; wherein the second-category change indicates that one or more of the difference in light intensity between the outgoing light emitted by the at least one shooting device and the reflected light received by the at least one shooting device is greater than or equal to the first preset threshold and less than or equal to the second preset threshold, the position of the object, and the type of the object changes; When a third confidence level of the type of each object in the first scene picture of the target scene is greater than or equal to the object type confidence threshold, determining a preset scene type of the target scene and a confidence level of the preset scene type of the target scene based on the type of each object in the scene picture of the target scene; When the confidence of the preset scene type of the target scene is greater than or equal to the scene type confidence threshold, determining the depth parameter corresponding to the target scene based on the preset scene type of the target scene; The determining the depth information of at least one object in the target scene based on the depth parameter corresponding to the target scene includes: Depth information of each object in the target scene is determined based on the depth parameter corresponding to the target scene.

7. The method according to claim 6, characterized in that Before the third confidence of the type of each object in the first scene picture of the target scene is greater than or equal to the object type confidence threshold, the method further includes: Object semantic segmentation is performed on the first scene picture of the target scene to obtain the type of each object in the first scene picture of the target scene and a third confidence level of the type of each object in the first scene picture of the target scene.

8. The method according to claim 6, characterized in that The step of determining the depth parameter corresponding to the target scene based on the preset scene type of the target scene when the confidence of the preset scene type of the target scene is greater than or equal to the scene type confidence threshold comprises: If there are multiple preset scene types of the target scene whose confidences are all greater than or equal to the scene type confidence threshold, the preset scene type of the target scene is the preset scene type corresponding to the maximum confidence; A depth parameter corresponding to the target scene is determined based on the preset scene type corresponding to the maximum confidence.

9. The method according to claim 6, characterized in that After the third confidence level of the type of each object in the first scene picture of the target scene is greater than or equal to the object type confidence level threshold, and before determining the depth information of each object in the target scene based on the depth parameter of the target scene, the method further includes: When the confidence of the preset scene type of the target scene is less than the scene type confidence threshold, using the depth parameter corresponding to the third confidence of the type of each object in the first scene picture of the target scene as the depth parameter corresponding to the target scene; The determining the depth information of each object in the target scene based on the depth parameter corresponding to the target scene includes: Determine depth information of each object in the target scene based on a depth parameter corresponding to a third confidence level of a type of each object in the first scene picture of the target scene.

10. The method according to claim 6, characterized in that The depth parameters include any one or more of the shooting frequency of the shooting device, the filtering threshold, and the number of neighborhood points; Before determining the depth information of each object in the target scene based on the depth parameter corresponding to the target scene, the method further includes: When the third confidence of the type of each object in the first scene picture of the target scene is less than the object type confidence threshold, after adjusting any one or more of the depth parameters, the target scene is photographed again to obtain a fourth confidence of the type of the object in the second scene picture of the target scene; If the fourth confidence of the type of the object in the second scene picture of the target scene is greater than or equal to the object type confidence threshold, the preset scene type of the target scene is determined based on the type of the object in the second scene picture of the target scene.

11. The method according to claim 10, characterized in that Before determining the depth information of each object in the target scene based on the depth parameter corresponding to the target scene, the method further includes: When the fourth confidence of the type of each object in the scene picture of the target scene is less than the object type confidence threshold, after adjusting any one or more of the depth parameters, the target scene is photographed again to obtain a fifth confidence of the type of the object in the second scene picture of the target scene; If the fifth confidence level of the type of the object in the second scene picture of the target scene is greater than or equal to the object type confidence threshold, the type of the object in the second scene picture of the target scene determines the preset scene type of the target scene; If the fifth confidence of the type of object in the second scene picture of the target scene is still less than the object type confidence threshold, return to the step of adjusting any one or more of the depth parameters and photograph the target scene again until the fifth confidence of the type of object in the second scene picture of the target scene is greater than or equal to the object type confidence threshold.

12. The method according to claim 1, characterized in that The device includes a plurality of photographing devices; The step of photographing the target scene using the at least one photographing device to obtain a scene picture of the target scene includes: Using the multiple shooting devices to shoot the target scene respectively, each shooting device outputs at least one first scene picture of the target scene; The determining the depth information of at least one object in the target scene based on the depth parameter corresponding to the target scene includes: At least one first scene picture of the target scene output by each shooting device is processed based on the depth parameter corresponding to the target scene to determine the depth information of at least one object in the target scene.

13. The method according to claim 1, characterized in that The device includes a camera and a depth detection sensor; The determining the depth information of at least one object in the target scene based on the depth parameter corresponding to the target scene includes: The depth parameter corresponding to the target scene and the first scene picture of the target scene are input into the depth detection sensor to obtain the depth information of at least one object in the target scene.

14. A depth information determination device, characterized in that: The depth information determination method is used for a device that needs to determine depth information, and the device includes at least one shooting device; An acquisition module, used to acquire a scene image of an initial scene; wherein the initial scene represents a scene whose light intensity is within a preset range and does not contain objects of a preset type; An obtaining module, configured to use the at least one photographing device to photograph a target scene to obtain a first scene image of the target scene; A first determination module is configured to determine that an object of the preset type exists in the target scene when there is a preset first category change between the scene image of the initial scene and the first scene image of the target scene; wherein the first category change indicates that a light intensity difference between the outgoing light emitted by the at least one camera and the reflected light received by the at least one camera is greater than a first preset threshold or less than a second preset threshold; A second determination module is used to determine a depth parameter corresponding to the target scene based on the preset type of object in the target scene; wherein the depth parameter represents a parameter for adjusting the depth information; The third determination module is used to determine the depth information of at least one object in the target scene based on the depth parameter corresponding to the target scene.

15. A computer storage medium, characterized in that The computer storage medium stores a plurality of instructions, which are suitable for being loaded by a processor and executing the method steps according to any one of claims 1 to 13.

16. An electronic device, characterized in that: include: A processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the method steps according to any one of claims 1 to 13.

Images