Light supplementing working parameter adjusting method and device, electronic equipment and storage medium

Abstract

The invention provides a light supplementing working parameter adjusting method and device, electronic equipment and a storage medium. The method comprises the steps that a first texture feature point of a first scene image of a target scene is acquired; dividing the first scene image into a plurality of first areas according to a preset rule, and determining first sub-texture feature points of each first area based on the first texture feature points; and based on the depth map of the first scene image and the first sub-texture feature point of each first area, light supplementing working parameters of the light supplementing equipment are adjusted, and the light supplementing working parameters comprise working power and / or light supplementing parameters. According to the technical scheme provided by the embodiment of the invention, the light supplementing equipment does not need to be continuously started, speckle lighting is carried out according to the fixed power, the light supplementing working parameters of the light supplementing equipment are adjusted through the texture feature points on the first scene image of the target scene, and the power consumption of the light supplementing equipment is greatly reduced.

Description

Technical Field

[0001] This disclosure belongs to the field of information processing technology, and in particular relates to a method, apparatus, electronic device and storage medium for adjusting supplementary lighting working parameters. Background Technology

[0002] In constructing a 3D image of the current scene, it is necessary to acquire the texture feature points of the image corresponding to the current scene. When the current scene is a white wall, a solid-color desktop, etc., the acquired texture feature points are weak texture feature points. When the current scene is a carpet, wood-grain furniture, etc., the acquired texture feature points are repetitive texture feature points. These acquired texture feature points for the current scene are few in number and have low accuracy. In order to improve the number and accuracy of texture feature points in the image corresponding to the scene, related technologies mainly use supplementary lighting devices to actively apply speckled lighting to the current scene. By applying speckled lighting, the number of texture feature points in the image corresponding to the current scene can be increased, thereby improving the accuracy of texture feature point determination. However, actively and continuously applying speckled lighting will bring huge power consumption to the imaging device and cause serious heat generation problems. Summary of the Invention

[0003] This disclosure provides a solution to address the issue in related technologies where supplementary lighting devices actively apply speckled lighting to the current scene. While speckled lighting can increase the number of texture feature points in the image corresponding to the current scene to improve the accuracy of texture feature point determination, actively and continuously applying speckled lighting can lead to huge power consumption for the shooting device and cause severe overheating.

[0004] In a first aspect, this disclosure provides a method for adjusting supplementary lighting operating parameters, the method comprising:

[0005] Obtain the first texture feature points of the first scene image of the target scene;

[0006] The first scene image is divided into multiple first regions according to a preset rule, and the first sub-texture feature point of each first region is determined based on the first texture feature point.

[0007] Based on the depth map of the first scene image and the first sub-texture feature points of each first region, the supplementary lighting parameters of the supplementary lighting device are adjusted, including the working power and / or supplementary lighting parameters.

[0008] Secondly, this disclosure provides a device for adjusting supplementary lighting operating parameters, the device comprising:

[0009] The acquisition unit is used to acquire the first texture feature points of the first scene image of the target scene;

[0010] The determining unit is used to divide the first scene image into multiple first regions according to a preset rule, and to determine the first sub-texture feature point of each first region based on the first texture feature point;

[0011] The supplementary lighting unit is used to adjust the supplementary lighting operating parameters of the supplementary lighting device based on the depth map of the first scene image and the first sub-texture feature points of each first region. The supplementary lighting operating parameters include operating power and / or supplementary lighting parameters.

[0012] Thirdly, this disclosure provides an electronic device, including:

[0013] Processor; and

[0014] Memory for storing the executable instructions of the processor;

[0015] The processor is configured to execute the first aspect or any method in a possible implementation of the first aspect by executing the executable instructions.

[0016] Fourthly, embodiments of this disclosure provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the methods in the first aspect or any possible implementations of the first aspect.

[0017] Fifthly, embodiments of this disclosure provide a computer program product including computer instructions that, when executed by a processor, implement any method in the first aspect or any possible implementation of the first aspect.

[0018] The technical solution provided in this disclosure acquires first texture feature points of a first scene image of a target scene; divides the first scene image into multiple first regions according to preset rules, and determines first sub-texture feature points of each first region based on the first texture feature points; and adjusts the supplementary lighting parameters of the supplementary lighting device based on the depth map of the first scene image and the first sub-texture feature points of each first region. The technical solutions provided in the embodiments of this disclosure, by acquiring a first scene image of a target scene, partitioning the first scene image, and adjusting the supplementary lighting parameters of the supplementary lighting device based on the texture characteristics and depth map of different regions, reduce the power consumption of the supplementary lighting device and avoid the serious overheating problem caused by continuous speckled lighting from the supplementary lighting device. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments or related technologies of this disclosure, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:

[0020] Figure 1 A flowchart illustrating a method for adjusting supplementary lighting parameters according to an embodiment of this disclosure;

[0021] Figure 2 This is a schematic flowchart illustrating a process for determining a target depth value corresponding to the first region, as provided in an embodiment of this disclosure.

[0022] Figure 3 A schematic flowchart illustrating the process of determining the target power of the supplementary lighting device according to an embodiment of this disclosure;

[0023] Figure 4 A schematic flowchart illustrating another method for determining the target power of the supplementary lighting device according to an embodiment of this disclosure;

[0024] Figure 5 This is a schematic flowchart illustrating a supplementary lighting device control method according to an embodiment of the present disclosure.

[0025] Figure 6 This is a schematic diagram of the structure of a shooting device provided in one embodiment of the present disclosure;

[0026] Figure 7 This is a schematic diagram of a supplementary lighting working parameter adjustment device provided in an embodiment of the present disclosure;

[0027] Figure 8 This is a schematic diagram of the structure of an electronic device provided in one embodiment of the present disclosure. Detailed Implementation

[0028] Embodiments of this disclosure are described in detail below, with examples of embodiments illustrated in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this disclosure, and should not be construed as limiting this disclosure.

[0029] The terms "first" and "second," etc., used in the specification, claims, and drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the present disclosure described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0030] The supplementary lighting parameter adjustment method provided in this disclosure can run on a terminal device or a server. The terminal device can be a local terminal device, including wearable devices such as VR (Virtual Reality), AR (Augmented Reality), and MR (Mixed Reality). The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0031] Texture feature points are points in an image that possess unique texture properties. These points exhibit good robustness to image changes (such as rotation, scaling, and illumination variations). In binocular depth calculation, disparity matching is performed on the texture features of the left and right eye images after epipolar alignment, thereby completing binocular depth calculation based on feature point disparity. Binocular depth calculation provides depth information of the scene, and 3D reconstruction utilizes this depth information to construct a 3D model of the scene. Therefore, texture feature points play a crucial role, providing a reliable basis for matching and depth estimation.

[0032] Binocular depth calculation relies on the parallax between images taken from two different viewpoints. Parallax refers to the difference in pixel position of the same object in two images. By comparing two images and finding matching feature points, the parallax can be calculated, and thus the depth information of the scene can be inferred. Texture feature points, due to their uniqueness, are easier to match in different images, thereby improving the accuracy of depth calculation and 3D reconstruction.

[0033] In binocular vision systems, feature point matching is a crucial step. The richness of texture features in both the left and right eye images significantly impacts the completeness and confidence of binocular depth calculation. Insufficient or unevenly distributed feature points in the image will lead to matching difficulties, thus affecting the accuracy of depth calculation.

[0034] For example, in natural environments, areas with little or no texture, such as white walls, uniformly colored planes, and floors, cannot be effectively matched using binocular parallax depth calculation. This results in incorrect depth values, manifesting as depth holes or invalid values ​​in these areas. Similarly, for areas with highly similar, repetitive textures, such as wooden tabletops or furniture, or plush carpets, binocular parallax depth calculation is prone to matching errors during feature point matching, leading to incorrect depth calculations. This manifests as messy depth values ​​or low confidence levels in these areas. Therefore, the number and distribution of texture feature points play a crucial role in the final depth estimation. They not only improve matching accuracy but also enhance the algorithm's adaptability to complex scenes.

[0035] In summary, texture feature points play a crucial role in binocular depth calculation and 3D scene reconstruction, directly impacting the accuracy of feature point matching, depth information computation, and the results of 3D scene reconstruction. Supplementing with effective texture feature points can significantly improve the performance of binocular vision systems.

[0036] To increase the number and accuracy of texture feature points in the image corresponding to the scene, and thus improve the accuracy of binocular depth calculation, related technologies mainly use supplementary lighting devices to actively apply speckled lighting to the current scene. By applying speckled lighting, the number of texture feature points in the image corresponding to the current scene can be increased, thereby improving the accuracy of texture feature point determination. However, actively and continuously applying speckled lighting will bring huge power consumption to the shooting device and cause serious overheating problems to the shooting device.

[0037] The technical solutions of this disclosure and how they solve the aforementioned technical problems will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of this disclosure will now be described with reference to the accompanying drawings.

[0038] Figure 1 This is a flowchart illustrating a method for adjusting supplementary lighting parameters, provided as an exemplary embodiment of the present disclosure. This method is applicable to electronic devices equipped with supplementary lighting devices for supplementing lighting of a target scene. Taking the application of this method to a shooting device as an example, the solution includes at least the following steps S101-S103:

[0039] S101, Obtain the first texture feature points of the first scene image of the target scene.

[0040] In some embodiments, the first scene image refers to the current frame image during the shooting process of the shooting device.

[0041] Since SLAM (Simultaneous Localization and Mapping) does not allow infrared speckle functionality to be enabled during operation (i.e., without supplemental lighting), the feature detection results of the SLAM algorithm module can be read out by the FW (Firmware) as environmental feature representation information for binocular depth calculation. The supplemental lighting control decision logic is completed by the software algorithm, and the control adjustment is implemented by the control module of the sensor master configured by the FW.

[0042] Preferably, a SLAM camera is used to capture the target scene to obtain a first scene image, and the detection results of real-world texture feature points in the captured image are calculated according to existing SLAM algorithms, which are used as the first texture feature points of the first scene image. SLAM (Simultaneous Localization and Mapping) technology is a technology that allows devices to autonomously locate themselves in unknown environments and build environmental maps.

[0043] In other embodiments, under supplemental lighting conditions, the first texture feature points can be obtained from the binocular images captured by the binocular camera, and then the disparity matching feature points calculated according to SGM (Semi-Global Matching) or a similar binocular depth scheme.

[0044] In some embodiments, the shooting device may be a wearable device.

[0045] In some embodiments, the imaging device is a binocular imaging device.

[0046] Specifically, the head-mounted device is equipped with a SLAM camera that can capture images of the natural world and extract texture feature points of the current frame.

[0047] S102, the first scene image is divided into multiple first regions according to preset rules, and the first sub-texture feature point of each first region is determined based on the first texture feature point.

[0048] Specifically, preset rules refer to preset division methods. For example, dividing the first scene image into m*n regions.

[0049] Furthermore, after dividing the first scene image into m*n regions, i.e. m*n first regions, the first texture feature point falling into the first region is the first sub-texture feature point in the first region, based on the determined first texture feature point.

[0050] S103, based on the depth map of the first scene image and the first sub-texture feature points of each first region, adjust the supplementary lighting parameters of the supplementary lighting device.

[0051] In some embodiments, the supplementary lighting operating parameters include operating power and / or supplementary lighting parameters.

[0052] In some implementations, adjusting the supplementary lighting parameters of the supplementary lighting device based on the depth map of the first scene image and the first sub-texture feature points of each first region includes steps S11-S12:

[0053] S11, for any one of the plurality of first regions, based on the depth map of the first scene image and the first sub-texture feature point of the first region, determine the target depth value corresponding to the first region, and then obtain the plurality of target depth values ​​corresponding to the plurality of first regions.

[0054] In some embodiments, the target depth value is the average depth value, the median depth value, or the mode depth value.

[0055] In some implementations, to ensure the accuracy of the target depth value, it is preferable to use the average depth value, combined with... Figure 2 As shown, when the target depth value is the average depth value, the step of determining the target depth value corresponding to the first region based on the depth map of the first scene image and the first sub-texture feature points of the first region includes steps S111-S113:

[0056] S111, determine a first number of feature points contained in the first sub-texture feature points of the first region. The first number refers to the number of feature points contained in the first sub-texture feature points of the first region.

[0057] S112, based on the depth map of the first scene image, determine the depth value corresponding to each feature point contained in the first sub-texture feature point of the first region.

[0058] S113, based on the first quantity and the depth value corresponding to each feature point contained in the first sub-texture feature point of the first region, determine the average depth value corresponding to the first region.

[0059] The average depth value refers to the average of the first number of depth values.

[0060] Specifically, for example, if the first sub-texture feature point in the first region contains 10 feature points, and the depth values ​​corresponding to each feature point in the first sub-texture feature point in the first region are 2, 2, 3, 3, 4, 4, 4, 4, 5, and 6 respectively, then the average depth value is determined to be 3.7. By using the average value method, the distribution of the first sub-texture feature points in the first region can be better reflected, thus providing data support for adjusting the supplementary lighting parameters of the supplementary lighting device.

[0061] In other embodiments, in order to reduce the amount of computation, when faced with a large number of texture feature points, we can use the median of the depth values ​​of each feature point contained in the first sub-texture feature points of the first region as the target depth value. Therefore, if the target depth value is the median depth value, the step of determining the target depth value of the first region based on the depth map of the first scene image and the first sub-texture feature points of the first region includes: arranging the depth values ​​of each feature point contained in the first sub-texture feature points of the first region in descending or ascending order, and using the median as the median depth value, i.e., the target depth value.

[0062] Specifically, the depth values ​​corresponding to each feature point in the first sub-texture feature point of the first region are 2, 2, 3, 3, 4, 4, 4, 4, 5 respectively. The median is determined to be 4, and the target depth value is determined to be 4.

[0063] In other embodiments, in order to improve work efficiency and speed up the determination of the target depth value, we can take the mode of the depth values ​​corresponding to each feature point contained in the first sub-texture feature points of the first region as the target depth value. Therefore, if the target depth value is the mode depth value, the step of determining the target depth value corresponding to the first region based on the depth map of the first scene image and the first sub-texture feature points of the first region includes: taking the most frequent value among the depth values ​​corresponding to each feature point contained in the first sub-texture feature points of the first region as the target depth value.

[0064] Specifically, the depth values ​​corresponding to each feature point in the first sub-texture feature point of the first region are 2, 2, 3, 3, 4, 4, 4, 4, 5 respectively. The value that appears most frequently is determined to be 4, and the target depth value is determined to be 4.

[0065] S12, based on the multiple target depth values, adjust the supplementary lighting operating parameters of the supplementary lighting device.

[0066] The supplementary lighting parameters refer to the density of the speckle light from the supplementary lighting device.

[0067] In some embodiments, adjusting the supplementary lighting parameters of the supplementary lighting device based on the plurality of target depth values ​​includes:

[0068] If the minimum value among the plurality of target depth values ​​is greater than a first preset value, the supplementary lighting parameters of the supplementary lighting device are adjusted to increase the density of the speckle light from the supplementary lighting device. Specifically, when the minimum value among the plurality of target depth values ​​is greater than the first preset value, it indicates that the target scene is too far from the shooting device, resulting in a distant image in the captured first scene image. Consequently, the feature points at the farthest distance in the target scene are too sparse. Therefore, in order to increase the number of feature points, it is necessary to increase the density of the speckle light from the supplementary lighting device.

[0069] If the minimum value among the plurality of target depth values ​​is not greater than the first preset value, the supplementary lighting parameters of the supplementary lighting device are adjusted to reduce the density of the speckle light from the supplementary lighting device. Specifically, when the minimum value among the plurality of target depth values ​​is not greater than the first preset value, it indicates that the target scene is too close to the shooting device, resulting in a close-up image in the captured first scene image. This leads to an overly dense concentration of feature points at the closest distance in the target scene. Therefore, to reduce the number of feature points, it is necessary to reduce the density of the speckle light from the supplementary lighting device.

[0070] In practice, the density of speckle light generally depends on the characteristics of the light source of the supplementary lighting equipment, the aberrations and defects of the optical system, and the characteristics of the medium through which the light passes. Therefore, adjusting the density of speckle light requires controlling it by adjusting the speckle parameters. These speckle parameters typically include the diameter, roundness, and position of the speckle.

[0071] In some embodiments, the density of speckle light increases linearly with distance, while the operating power of the supplementary lighting device increases quadratically with distance. Therefore, in practical applications, power consumption requirements are given priority, followed by consideration of increasing the density of speckle light at long distances.

[0072] In some embodiments, during the adjustment of the supplementary lighting device's operating parameters, the operating power of the supplementary lighting device is adjusted first, followed by the density of the speckle light from the supplementary lighting device. To ensure the supplementary lighting device achieves optimal operating performance, the operating power is adjusted first. Therefore, in other embodiments, adjusting the supplementary lighting device's operating parameters based on the depth map of the first scene image and the first sub-texture feature points of each first region includes:

[0073] Based on the depth map of the first scene image and the first sub-texture feature points of each first region, the farthest distance of the scene in the target scene is determined; if the farthest distance of the scene is greater than a second preset value, the working power of the supplementary lighting device is increased; if the farthest distance of the scene is not greater than the second preset value, the working power of the supplementary lighting device is decreased.

[0074] Specifically, if the farthest distance of the scene is greater than the second preset value, it means that the target scene is too far away from the shooting device, so that the image in the first scene image is too far away. Therefore, in order to avoid the image being unclear, it is necessary to increase the working power of the supplementary lighting device. If the farthest distance of the scene is not greater than the second preset value, it means that the target scene is too close to the shooting device, so that the image in the first scene image is too close. Therefore, in order to avoid the image being overexposed and wasting the power consumption of the shooting device, it is necessary to reduce the working power of the supplementary lighting device.

[0075] In this embodiment, when adjusting the operating power of the supplementary lighting device, in order to improve the accuracy of the adjustment, it is necessary to first determine the target power of the supplementary lighting device, that is, the optimal power for adjustment, and combine it with... Figure 3 If the first scene image was acquired while the supplementary lighting device was on, the method further includes steps S21-S23:

[0076] S21, with the supplementary lighting device off, acquire a second scene image of the target scene, and acquire the second texture feature points of the second scene image.

[0077] In this embodiment, when the supplementary lighting device is turned off while the first scene image is being acquired, a second scene image of the target scene is acquired, and the first scene image and the second scene image are acquired simultaneously.

[0078] In some embodiments, after acquiring the second scene image, the second texture feature points of the second scene image are extracted using the SGM algorithm.

[0079] SGM (Semi-Global Matching) is a calculation algorithm for semi-global matching and is also a dense matching algorithm used in binocular vision. It is mainly used to calculate the disparity between corresponding points in an image to achieve stereo vision and depth perception.

[0080] In other embodiments, the depth value of the scene image is calculated using the SGM algorithm.

[0081] S22, the second scene image is divided into multiple second regions according to preset rules, and the second sub-texture feature points of each second region are determined based on the second texture feature points.

[0082] In this embodiment, the preset rule is the same as the preset rule in step S102, that is, step S22 and step S102 are the same in dividing the scene image. Further details will not be provided here.

[0083] S23, based on the second sub-texture feature points of each second region, determine the target power of the supplementary lighting device.

[0084] Specifically, in the actual operation of supplementary lighting equipment, the number of texture feature points in the acquired scene image varies depending on the power. Therefore, to obtain the most accurate target power, the power of the supplementary lighting equipment can only be considered the target power if the number of texture feature points in the acquired scene image remains unchanged despite changes in the power of the supplementary lighting equipment. Figure 4 As shown, the specific steps are as follows:

[0085] The step of determining the target power of the supplementary lighting device based on the second sub-texture feature points of each second region includes steps S231-S234:

[0086] S231, for any second region among the plurality of second regions, based on the second sub-texture feature points of the second region and the depth map of the second scene image, determine the target depth value corresponding to the second region, and then obtain the plurality of target depth values ​​corresponding to the plurality of second regions.

[0087] S232, the second region corresponding to the maximum value among the multiple target depth values ​​corresponding to the multiple second regions is taken as the target region.

[0088] S233, control the supplementary lighting device to supplement the target scene according to the rule of increasing supplementary lighting power, and obtain the second number of texture feature points located in the target area of ​​the third scene image of the target scene under different supplementary lighting power, and obtain multiple second numbers corresponding to different supplementary lighting power.

[0089] Among them, the supplementary light power corresponds one-to-one with the second quantity.

[0090] S234, when it is detected that the second quantity remains unchanged, the lowest supplementary light power among the multiple supplementary light powers corresponding to the second quantity when the second quantity remains unchanged is taken as the target power.

[0091] In some embodiments, adjusting the lighting parameters of the supplementary lighting device based on the depth map of the first scene image and the first sub-texture feature points of each first region includes: adjusting the lighting parameters of the supplementary lighting device based on the depth map of the first scene image, the first sub-texture feature points of each first region, and the target power.

[0092] Specifically, after determining the target power, when adjusting the supplementary lighting operating parameters of the supplementary lighting device, the current power of the supplementary lighting device is adjusted to the target power.

[0093] In actual operation, if the user is outdoors, no supplemental lighting is needed when the shooting device is taking pictures. Therefore, to reduce the power consumption of the shooting device, it is necessary to determine whether the user is outdoors in a timely manner, thereby reducing the power consumption of the shooting device. Figure 5 As shown, the specific steps include S31-S34:

[0094] S31, determine the third number of third regions whose target depth values ​​are greater than a third preset value among the multiple target depth values ​​corresponding to the multiple first regions.

[0095] The third region is the first region.

[0096] Specifically, if the plurality of first regions are first region 1, first region 2, second region 3, and first region 4 respectively, and the target depth value corresponding to first region 1 is 2, the target depth value corresponding to first region 2 is 4, the target depth value corresponding to second region 3 is 5, the target depth value corresponding to first region 4 is 6, and the third preset value is 3, then the third region refers to first region 2, second region 3, and first region 4, and the third quantity is 3.

[0097] S32, determine the fourth number of the fourth region in the plurality of first regions.

[0098] The fourth region is the first region.

[0099] Specifically, if the plurality of first regions are first region 1, first region 2, second region 3, and first region 4 respectively, then the fourth region refers to the first region, and the number of fourth regions is 4.

[0100] S33, if the ratio of the third quantity to the fourth quantity is greater than the fourth preset value, then control the supplementary lighting device to turn off.

[0101] In this embodiment, if the ratio of the third quantity to the fourth quantity is greater than the fourth preset value, it means that in the target scene captured by the user, many texture features are more than a preset distance away from the shooting device, that is, the current environment is an outdoor environment.

[0102] S34, if the ratio of the third quantity to the fourth quantity is not greater than the fourth preset value, then control the supplementary lighting device to turn on.

[0103] In this embodiment, adjusting the supplementary lighting parameters of the supplementary lighting device based on the depth map of the first scene image and the first sub-texture feature points of each first region further includes: adjusting the supplementary lighting parameters of the supplementary lighting device based on the depth map of the first scene image and the first sub-texture feature points of each first region when the supplementary lighting device is turned on.

[0104] In some implementations, the shooting equipment has a chip that can realize SGM or similar binocular depth calculation functions, and can control the lighting equipment to perform speckle lighting.

[0105] Why is it better to understand the embodiments of this solution, in conjunction with Figure 6 As shown, the process of adjusting the supplementary lighting parameters in this scheme is further explained as follows: During the shooting process, the shooting device captures a first scene image and a second scene image through a binocular camera. The first scene image is processed by SLAM to extract the first texture feature points and form a distribution map of the first texture feature points. The second scene image is processed by SGM to extract the second texture feature points and form a distribution map of the second texture feature points. The depth maps of the first scene image and the second scene image are also obtained. Finally, the control unit calculates and controls the supplementary lighting device to adjust the supplementary lighting parameters.

[0106] The technical solution provided in this disclosure acquires first texture feature points of a first scene image of a target scene; divides the first scene image into multiple first regions according to preset rules, and determines first sub-texture feature points of each first region based on the first texture feature points; and adjusts the supplementary lighting parameters of the supplementary lighting device based on the depth map of the first scene image and the first sub-texture feature points of each first region. The technical solutions provided in the embodiments of this disclosure, by acquiring a first scene image of a target scene, partitioning the first scene image, and adjusting the supplementary lighting parameters of the supplementary lighting device based on the texture characteristics and depth map of different regions, reduce the power consumption of the supplementary lighting device and avoid the serious overheating problem caused by continuous speckled lighting from the supplementary lighting device.

[0107] Figure 7 A schematic diagram of a supplementary lighting working parameter adjustment device provided for an exemplary embodiment of this disclosure;

[0108] The device includes: an acquisition unit 201, a determination unit 202, and a supplementary lighting unit 203;

[0109] The acquisition unit 201 is used to acquire the first texture feature points of the first scene image of the target scene;

[0110] The determining unit 202 is used to divide the first scene image into multiple first regions according to a preset rule, and determine the first sub-texture feature point of each first region based on the first texture feature point;

[0111] The supplementary lighting unit 203 is used to adjust the supplementary lighting operating parameters of the supplementary lighting device based on the depth map of the first scene image and the first sub-texture feature points of each first region. The supplementary lighting operating parameters include operating power and / or supplementary lighting parameters.

[0112] In some embodiments, the method is applicable to a shooting device equipped with a lighting device for providing supplemental lighting to a target scene.

[0113] In some embodiments, the device is used to adjust the supplementary lighting parameters of the supplementary lighting device based on the depth map of the first scene image and the first sub-texture feature points of each first region, specifically for:

[0114] For any one of the plurality of first regions, based on the depth map of the first scene image and the first sub-texture feature points of the first region, the target depth value corresponding to the first region is determined, thereby obtaining the plurality of target depth values ​​corresponding to the plurality of first regions;

[0115] Based on the multiple target depth values, the supplementary lighting parameters of the supplementary lighting device are adjusted.

[0116] In some embodiments, the device is used to adjust the supplementary lighting operating parameters of the supplementary lighting device based on the plurality of target depth values, specifically for:

[0117] If the minimum value among the plurality of target depth values ​​is greater than the first preset value, the supplementary lighting parameters of the supplementary lighting device are adjusted to increase the density of the speckle light of the supplementary lighting device;

[0118] If the minimum value among the plurality of target depth values ​​is not greater than the first preset value, the supplementary lighting parameters of the supplementary lighting device are adjusted to reduce the density of speckle light from the supplementary lighting device.

[0119] In some embodiments, the target depth value is the average depth value, the median depth value, or the mode depth value.

[0120] In some embodiments, the apparatus is configured to determine the target depth value corresponding to the first region based on the depth map of the first scene image and the first sub-texture feature points of the first region when the target depth value is an average depth value, specifically for:

[0121] Determine the first number of feature points contained in the first sub-texture feature points of the first region;

[0122] Based on the depth map of the first scene image, determine the depth value corresponding to each feature point contained in the first sub-texture feature point of the first region.

[0123] Based on the first quantity and the depth value corresponding to each feature point contained in the first sub-texture feature point of the first region, the average depth value corresponding to the first region is determined.

[0124] In some embodiments, the device is used to adjust the supplementary lighting parameters of the supplementary lighting device based on the depth map of the first scene image and the first sub-texture feature points of each first region, specifically for:

[0125] Based on the depth map of the first scene image and the first sub-texture feature points of each first region, the farthest distance of the scene in the target scene is determined;

[0126] If the farthest distance in the scene is greater than the second preset value, then the working power of the supplementary lighting device is increased;

[0127] If the farthest distance in the scene is not greater than the second preset value, then reduce the operating power of the supplementary lighting device.

[0128] In some embodiments, the first scene image is acquired when the supplementary lighting device is turned on, and the apparatus is further configured to:

[0129] With the supplementary lighting device off, a second scene image of the target scene is acquired, and the second texture feature points of the second scene image are acquired;

[0130] The second scene image is divided into multiple second regions according to a preset rule, and the second sub-texture feature points of each second region are determined based on the second texture feature points;

[0131] The target power of the supplementary lighting device is determined based on the second sub-texture feature points of each second region;

[0132] In some embodiments, the device is used to adjust the lighting parameters of the supplementary lighting device based on the depth map of the first scene image and the first sub-texture feature points of each first region. Specifically, it is used to adjust the lighting parameters of the supplementary lighting device based on the depth map of the first scene image, the first sub-texture feature points of each first region, and the target power.

[0133] In some embodiments, the device is configured to determine the target power of the supplementary lighting device based on the second sub-texture feature points of each second region, specifically for:

[0134] For any second region among the plurality of second regions, based on the second sub-texture feature points of the second region and the depth map of the second scene image, the target depth value corresponding to the second region is determined, thereby obtaining the plurality of target depth values ​​corresponding to the plurality of second regions;

[0135] The second region corresponding to the maximum value among the multiple target depth values ​​corresponding to the multiple second regions is taken as the target region;

[0136] The supplementary lighting device is controlled to supplement the target scene according to the rule of increasing supplementary lighting power, and the second number of texture feature points located in the target area of ​​the third scene image of the target scene under different supplementary lighting power is obtained, so as to obtain multiple second numbers corresponding to different supplementary lighting power, wherein the supplementary lighting power and the second number correspond one-to-one;

[0137] When the second quantity is detected to remain unchanged, the lowest supplementary light power among the multiple supplementary light powers corresponding to the second quantity when the second quantity remains unchanged is taken as the target power.

[0138] In some embodiments, the device is further configured to:

[0139] Determine a third number of third regions among the multiple target depth values ​​corresponding to the multiple first regions, wherein the target depth value of the third region is greater than a third preset value, and the third region is a first region;

[0140] Determine the fourth number of fourth regions among the plurality of first regions, wherein the fourth region is a first region;

[0141] If the ratio of the third quantity to the fourth quantity is greater than a fourth preset value, then the supplementary lighting device is controlled to turn off;

[0142] If the ratio of the third quantity to the fourth quantity is not greater than the fourth preset value, then the supplementary lighting device is controlled to turn on;

[0143] In some embodiments, the device is used to adjust the supplementary lighting parameters of the supplementary lighting device based on the depth map of the first scene image and the first sub-texture feature points of each first region, specifically for:

[0144] When the supplementary lighting device is turned on, the supplementary lighting parameters of the supplementary lighting device are adjusted based on the depth map of the first scene image and the first sub-texture feature points of each first region.

[0145] The technical solution provided in this disclosure acquires first texture feature points of a first scene image of a target scene; divides the first scene image into multiple first regions according to preset rules, and determines first sub-texture feature points of each first region based on the first texture feature points; and adjusts the supplementary lighting parameters of the supplementary lighting device based on the depth map of the first scene image and the first sub-texture feature points of each first region. The technical solutions provided in the embodiments of this disclosure, by acquiring a first scene image of a target scene, partitioning the first scene image, and adjusting the supplementary lighting parameters of the supplementary lighting device based on the texture characteristics and depth map of different regions, reduce the power consumption of the supplementary lighting device and avoid the serious overheating problem caused by continuous speckled lighting from the supplementary lighting device.

[0146] It should be understood that the device embodiments and method embodiments can correspond to each other, and similar descriptions can be referred to the method embodiments. To avoid repetition, they will not be repeated here. Specifically, the device can execute the above method embodiments, and the foregoing and other operations and / or functions of each module in the device correspond to the corresponding processes in the various methods in the above method embodiments, which will not be repeated here for the sake of brevity.

[0147] The apparatus of this disclosure embodiment has been described above from the perspective of functional modules in conjunction with the accompanying drawings. It should be understood that this functional module can be implemented in hardware, in software instructions, or in a combination of hardware and software modules. Specifically, the steps of the method embodiments in this disclosure can be completed by integrated logic circuits in the processor's hardware and / or by software instructions. The steps of the method disclosed in this disclosure embodiment can be directly embodied as being executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. Optionally, the software module can reside in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps in the above method embodiments.

[0148] Figure 8 This is a schematic block diagram of an electronic device provided in an embodiment of this disclosure. The electronic device may include:

[0149] The system includes a memory 301 for storing computer programs and a processor 302 for transferring program code to the processor 302. In other words, the processor 302 can retrieve and run the computer program from the memory 301 to implement the methods described in this embodiment.

[0150] For example, the processor 302 can be used to execute the above-described method embodiments according to instructions in the computer program.

[0151] In some embodiments of this disclosure, the processor 302 may include, but is not limited to:

[0152] General-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0153] In some embodiments of this disclosure, the memory 301 includes, but is not limited to:

[0154] Volatile memory and / or non-volatile memory. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

[0155] In some embodiments of this disclosure, the computer program may be divided into one or more modules, which are stored in the memory 301 and executed by the processor 302 to perform the method provided in this disclosure. The one or more modules may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the electronic device.

[0156] like Figure 8 As shown, the electronic device may further include:

[0157] Transceiver 303, which can be connected to processor 302 or memory 301.

[0158] The processor 302 can control the transceiver 303 to communicate with other devices; specifically, it can send information or data to other devices or receive information or data sent by other devices. The transceiver 303 may include a transmitter and a receiver. The transceiver 303 may further include antennas, and the number of antennas may be one or more.

[0159] It should be understood that the various components in the electronic device are connected through a bus system, which includes a data bus, a power bus, a control bus, and a status signal bus.

[0160] This disclosure also provides a computer storage medium storing a computer program thereon, which, when executed by a computer, enables the computer to perform the methods of the above-described method embodiments. Alternatively, embodiments of this disclosure also provide a computer program product containing instructions that, when executed by a computer, cause the computer to perform the methods of the above-described method embodiments.

[0161] When implemented using software, it can be implemented wholly or partially as a computer program product. This computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this disclosure are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital video disc (DVD)), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0162] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0163] In the embodiments provided in this disclosure, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0164] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. For example, the functional modules in the various embodiments of this disclosure may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0165] The above are merely specific embodiments of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. A method for adjusting supplementary lighting parameters, characterized in that, The method includes: Obtain the first texture feature points of the first scene image of the target scene; The first scene image is divided into multiple first regions according to a preset rule, and the first sub-texture feature point of each first region is determined based on the first texture feature point. Based on the depth map of the first scene image and the first sub-texture feature points of each first region, the supplementary lighting parameters of the supplementary lighting device are adjusted, including the working power and / or supplementary lighting parameters.

2. The method according to claim 1, characterized in that, The adjustment of the supplementary lighting parameters of the supplementary lighting device based on the depth map of the first scene image and the first sub-texture feature points of each first region includes: For any one of the plurality of first regions, based on the depth map of the first scene image and the first sub-texture feature points of the first region, the target depth value corresponding to the first region is determined, thereby obtaining the plurality of target depth values ​​corresponding to the plurality of first regions; Based on the multiple target depth values, the supplementary lighting parameters of the supplementary lighting device are adjusted.

3. The method according to claim 2, characterized in that, The adjustment of the supplementary lighting operating parameters of the supplementary lighting device based on the multiple target depth values ​​includes: If the minimum value among the plurality of target depth values ​​is greater than the first preset value, the supplementary lighting parameters of the supplementary lighting device are adjusted to increase the density of the speckle light of the supplementary lighting device; If the minimum value among the plurality of target depth values ​​is not greater than the first preset value, the supplementary lighting parameters of the supplementary lighting device are adjusted to reduce the density of speckle light from the supplementary lighting device.

4. The method according to any one of claims 2-3, characterized in that, The target depth value is the average depth value, the median depth value, or the mode depth value; When the target depth value is the average depth value, determining the target depth value corresponding to the first region based on the depth map of the first scene image and the first sub-texture feature points of the first region includes: Determine the first number of feature points contained in the first sub-texture feature points of the first region; Based on the depth map of the first scene image, determine the depth value corresponding to each feature point contained in the first sub-texture feature point of the first region. Based on the first quantity and the depth value corresponding to each feature point contained in the first sub-texture feature point of the first region, the average depth value corresponding to the first region is determined.

5. The method according to claim 1, characterized in that, The adjustment of the supplementary lighting parameters of the supplementary lighting device based on the depth map of the first scene image and the first sub-texture feature points of each first region includes: Based on the depth map of the first scene image and the first sub-texture feature points of each first region, the farthest distance of the scene in the target scene is determined; If the farthest distance in the scene is greater than the second preset value, then the working power of the supplementary lighting device is increased; If the farthest distance in the scene is not greater than the second preset value, then reduce the operating power of the supplementary lighting device.

6. The method according to claim 5, characterized in that, The first scene image was acquired with the supplementary lighting device turned on, and the method further includes: With the supplementary lighting device off, a second scene image of the target scene is acquired, and the second texture feature points of the second scene image are acquired; The second scene image is divided into multiple second regions according to a preset rule, and the second sub-texture feature points of each second region are determined based on the second texture feature points; The target power of the supplementary lighting device is determined based on the second sub-texture feature points of each second region; The step of adjusting the supplementary lighting parameters of the supplementary lighting device based on the depth map of the first scene image and the first sub-texture feature points of each first region includes: adjusting the supplementary lighting parameters of the supplementary lighting device based on the depth map of the first scene image, the first sub-texture feature points of each first region, and the target power.

7. The method according to claim 6, characterized in that, Determining the target power of the supplementary lighting device based on the second sub-texture feature points of each second region includes: For any second region among the plurality of second regions, based on the second sub-texture feature points of the second region and the depth map of the second scene image, the target depth value corresponding to the second region is determined, thereby obtaining the plurality of target depth values ​​corresponding to the plurality of second regions; The second region corresponding to the maximum value among the multiple target depth values ​​corresponding to the multiple second regions is taken as the target region; The supplementary lighting device is controlled to supplement the target scene according to the rule of increasing supplementary lighting power, and the second number of texture feature points located in the target area of ​​the third scene image of the target scene under different supplementary lighting power is obtained, so as to obtain multiple second numbers corresponding to different supplementary lighting power, wherein the supplementary lighting power and the second number correspond one-to-one; When the second quantity is detected to remain unchanged, the lowest supplementary light power among the multiple supplementary light powers corresponding to the second quantity when the second quantity remains unchanged is taken as the target power.

8. The method according to claim 2, characterized in that, The method further includes: Determine a third number of third regions among the multiple target depth values ​​corresponding to the multiple first regions, wherein the target depth value of the third region is greater than a third preset value, and the third region is a first region; Determine the fourth number of fourth regions among the plurality of first regions, wherein the fourth region is a first region; If the ratio of the third quantity to the fourth quantity is greater than a fourth preset value, then the supplementary lighting device is controlled to turn off; If the ratio of the third quantity to the fourth quantity is not greater than the fourth preset value, then the supplementary lighting device is controlled to turn on; The adjustment of the supplementary lighting parameters of the supplementary lighting device based on the depth map of the first scene image and the first sub-texture feature points of each first region further includes: When the supplementary lighting device is turned on, the supplementary lighting parameters of the supplementary lighting device are adjusted based on the depth map of the first scene image and the first sub-texture feature points of each first region.

9. A device for adjusting supplementary lighting parameters, characterized in that, The device includes: The acquisition unit is used to acquire the first texture feature points of the first scene image of the target scene; The determining unit is used to divide the first scene image into multiple first regions according to a preset rule, and to determine the first sub-texture feature point of each first region based on the first texture feature point; The supplementary lighting unit is used to adjust the supplementary lighting operating parameters of the supplementary lighting device based on the depth map of the first scene image and the first sub-texture feature points of each first region. The supplementary lighting operating parameters include operating power and / or supplementary lighting parameters.

10. An electronic device, characterized in that, include: processor; as well as Memory for storing the executable instructions of the processor; The processor is configured to execute the method of any one of claims 1-8 by executing the executable instructions.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1-8.

12. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the method described in any one of claims 1-8.

Images