Illumination system, illumination control method and electronic device

Abstract

The present application belongs to the technical field of illumination control. Disclosed are an illumination system, an illumination control method and an electronic device. The illumination system comprises a processor, and an illumination lamp, a target detection module and a jitter detection module, which are connected to the processor, wherein the illumination lamp comprises a plurality of light sources; the target detection module is used for acquiring and sending to the processor a first image of a first environment where the illumination system is located at a first moment; the jitter detection module is used for acquiring and sending to the processor the attitude angle variation of the jitter detection module; and the processor is used for predicting, on the basis of the received first image and attitude angle variation, information of a target occluded area in a second image of a second environment where the illumination system is located at a second moment, so as to determine the brightnesses of the plurality of light sources, and to control the plurality of light sources to form areas having different brightnesses in the second environment according to the corresponding brightnesses, the second moment being after the first moment. The present application can realize illumination control for the plurality of light sources in a timely and accurate manner.

Description

Lighting system, lighting control method and electronic equipment

[0001] This application claims priority to Chinese patent application No. 202311693314.X filed on December 8, 2023, entitled “Lighting system, lighting control method and electronic device”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] The present application relates to the field of lighting control technology, and in particular to a lighting system, a lighting control method and an electronic device. Background Art

[0003] With the development of lighting control technology, lighting control methods have become increasingly flexible, allowing the brightness of lighting sources to be controlled according to different needs and environments. For example, the brightness of the light sources on a vehicle can be determined and adjusted according to the vehicle's driving environment, achieving lighting control that flexibly adapts to the different brightness requirements of various driving situations such as meeting other vehicles and turning. Summary of the Invention

[0004] This application provides a lighting system, a lighting control method, and an electronic device to achieve timely and accurate lighting control. The technical solution is as follows:

[0005] In a first aspect, a lighting system is provided, which includes a lighting lamp, a target detection module, a jitter detection module and a processor, wherein the lighting lamp, the target detection module and the jitter detection module are all connected to the processor, and the lighting lamp includes multiple light sources; the target detection module is used to obtain and send a first image to the processor, where the first image is an image of a first environment in which the lighting system is located at a first moment; the jitter detection module is used to obtain and send a posture angle change of the jitter detection module to the processor; the processor is used to receive the first image and the posture angle change, predict information of a target shading area in a second image based on the first image and the posture angle change, determine the brightness of multiple light sources based on the information of the target shading area, and control the multiple light sources to form areas with different brightness in the second environment according to the corresponding brightness, where the second image is an image of a second environment in which the lighting system is located at a second moment, and the second moment is after the first moment.

[0006] The present application perceives the environment in which the lighting system is located at a first moment through a first image, perceives whether the lighting system has an attitude angle offset through an attitude angle change, and accurately predicts information about a target shielding area that needs to be light-shielded by the lighting system in a second environment based on the first environment indicated by the first image and the offset of the lighting system indicated by the attitude angle change, and determines the brightness of multiple light sources, so as to timely control multiple light sources according to the brightness of the multiple light sources to form areas with different brightness in the second environment, and timely and accurately perform light shielding on the target shielding area to achieve precise lighting control.

[0007] In one possible implementation, the lighting system further includes a distance sensing module connected to the processor; the distance sensing module is configured to obtain and transmit to the processor distance information between an obstructing object and the lighting fixture, where the obstructing object is an obstructing object in a first environment; the processor is configured to receive the distance information, obtain the movement speed of the lighting fixture, and predict information about a target obstructed area based on the movement speed, distance information, the first image, and the change in attitude angle. By sensing the distance information between the lighting fixture and the obstructing object through the distance sensing module, and comprehensively determining information about the target obstructed area based on the movement speed, distance information, the first image, and the change in attitude angle, the determined target obstructed area information is more accurate.

[0008] In one possible implementation, the lighting system further includes a speed sensing module connected to the processor; the speed sensing module is configured to acquire and transmit the speed of the lighting fixture to the processor; and the processor is configured to receive the speed. The speed sensing module efficiently senses the speed, eliminating the need for calculation. This improves the efficiency of speed acquisition and enables more accurate prediction of target obscured areas using the sensed speed.

[0009] In one possible implementation, a first image includes information about a first obscured area, where the first obscured area indicates an obscured area formed by multiple light sources in a first environment. A processor is configured to predict information about a second obscured area in a second image based on the information about the first obscured area and a change in attitude angle. The processor detects the second image to obtain information about a second detection area in the second image, where the second detection area is the area of ​​an obscured object in the second image. Based on the information about the second obscured area and the information about the second detection area, the processor determines information about a target obscured area, where the target obscured area indicates an obscured area formed by multiple light sources in the second environment. Because the second detection area is the area of ​​the obscured object in the second image, the information about the second detection area can accurately reflect the position of the obscured object in the second environment. Thus, more accurate information about the target obscured area can be obtained by combining the position of the obscured object in the second environment with the information about the second obscured area.

[0010] In one possible implementation, the processor is configured to determine a coordinate system change between a first coordinate system and a second coordinate system based on a posture angle change, wherein the first coordinate system is a coordinate system determined based on the posture angle of the target detection module at the first moment, and the second coordinate system is a coordinate system determined based on the posture angle of the target detection module at the second moment; calculate a mapping position of the first shielded area in the second coordinate system, wherein the change between the position of the first shielded area in the first coordinate system and the mapping position is the same as the coordinate system change; and predict information about the second shielded area based on the mapping position. Based on the coordinate system transformation amount, the offset between the position of the shielded object in the first image and the position in the second image can be predicted, and the mapping position of the shielded object in the second coordinate system can be accurately predicted, thereby obtaining the mapping position of the shielded object in the second image under the second coordinate system. The information about the target shielded area to which the shielded object belongs, determined based on the mapping position, is more accurate.

[0011] In one possible implementation, the processor is configured to calculate the attitude angle change of the target detection module based on the external parameter calibration result between the jitter detection module and the target detection module and the attitude angle change of the jitter detection module; and calculate the coordinate system change based on the attitude angle change of the target detection module. The external parameter calibration result in the present application can indicate the mapping relationship between the attitude angle change of the jitter detection module and the attitude angle change of the target detection module. Therefore, the attitude angle change of the target detection module can be calculated based on the external parameter calibration result and the attitude angle change of the jitter detection module, so that the coordinate system change can be determined based on the attitude angle change of the target detection module, thereby achieving accurate prediction of the information of the target occlusion area.

[0012] In one possible implementation, the processor is used to solve the attitude angle change of the jitter detection module to obtain the rotation matrix of the jitter detection module; based on the external parameter calibration result and the rotation matrix of the jitter detection module, the rotation matrix of the target detection module is calculated; according to the rotation matrix of the target detection module, the attitude angle change of the target detection module is calculated. Since the external parameter calibration result in the present application can indicate the mapping relationship between the attitude angle change of the jitter detection module and the attitude angle change of the target detection module, and the conversion between the attitude angle change of the target detection module and the attitude angle change of the jitter detection module can be realized based on the conversion between the rotation matrix of the target detection module and the rotation matrix of the jitter detection module, the rotation matrix of the target detection module can be calculated according to the external parameter calibration result and the rotation matrix of the jitter detection module, and then the attitude angle change of the target detection module can be obtained according to the rotation matrix of the target detection module, so that the coordinate system change can be determined according to the attitude angle change of the target detection module, thereby realizing accurate prediction of the information of the target occlusion area.

[0013] In one possible implementation, the processor is configured to determine a first degree of overlap between a second obscured area and a second detection area corresponding to the second obscured area; determine the second obscured area as a target obscured area based on the first degree of overlap being greater than a first degree of overlap threshold; and determine information about the target obscured area based on information about the second obscured area. If the first degree of overlap between the second detection area and the second obscured area is greater than the first degree of overlap threshold, it can be considered that the second obscured area and the second detection area match each other, i.e., the overlap between the second obscured area and the second detection area is large, the similarity is high, and the predicted information about the second obscured area is relatively accurate. Therefore, the information about the second obscured area can be determined as information about the target obscured area, thereby achieving accurate shielding of the obscured object.

[0014] In one possible implementation, the processor is further configured to determine, based on the first degree of overlap being less than or equal to a first degree of overlap threshold, whether the first obstructing object in the second detection area includes an obstructing object that does not belong to the first image, thereby obtaining a first determination result; and if the first determination result is that the first obstructing object in the second detection area includes an obstructing object that does not belong to the first image, update information about the second obstructing area based on information about the second detection area to obtain information about the target obstructing area. If the first degree of overlap between the second detection area and the second obstructing area is less than or equal to the first degree of overlap threshold, it can be determined that the second obstructing area does not match the second detection area, and further determination can be made as to whether the mismatch is due to the presence of an obstructing object in the second detection area that does not appear in the first image. If the occluded objects in the second detection area include occluded objects that do not belong to the first image, it is considered that the reason why the second detection area does not match the second occluded area is that the occluded object in the second detection area is a newly appeared occluded object in the second environment. Since the newly appeared occluded object was not predicted in the process of predicting the information of the second occluded area, the predicted information of the second occluded area is inaccurate. The information of the second occluded area can be updated according to the information of the second detection area to obtain more accurate information of the target occluded area.

[0015] In one possible implementation, the processor is further configured to, if the first determination result indicates that the first obstructing object in the second detection area is an obstructing object in the first image, obtain multiple second overlaps corresponding to the first obstructing object, wherein any second overlap in the multiple second overlaps is an overlap between any first detection area including the first obstructing object and the corresponding first obstructing area; determine a first number of consecutive overlaps less than a second overlap threshold in the multiple second overlaps; delete the second obstructing area based on the fact that the first number is greater than the first number threshold, thereby obtaining a third obstructing area in the first image; and determine information about the target obstructing area based on information about the third obstructing area. If the first obstructing object in the second detection area is an obstructing object in the first image, multiple second overlaps corresponding to the first obstructing object may be obtained. If the first number of consecutive overlaps less than the second overlap threshold in the multiple second overlaps is greater than the first number threshold, it is determined that a high number of prediction errors are made for the obstructing area of ​​the first obstructing object, and the second obstructing area of ​​the first obstructing object is deleted, and the prediction result is no longer referenced in the next prediction, thereby reducing the possibility of prediction errors.

[0016] In one possible implementation, the processor is further configured to determine, based on the first number being less than or equal to a first number threshold, a second number of the plurality of second overlaps that are less than a third overlap threshold; based on the second number being greater than the second number threshold, delete the second occluded region to obtain a fourth occluded region of the first image; and determine information about the target occluded region based on information about the fourth occluded region. If the number of consecutive first occluded regions less than the second overlap threshold in the plurality of second overlaps is greater than the first number threshold, it is determined that the prediction error for the occluded region of the first occluded object is relatively low. However, if the number of second occluded regions less than the third overlap threshold in the plurality of second overlaps is greater than the second number threshold, it is determined that the accuracy of the prediction for the occluded region of the first occluded object is unstable. In other words, although the prediction for the occluded region of the first occluded object does not have multiple consecutive errors, there are a high number of intermittent errors, and therefore the accuracy of the prediction for the occluded region of the first occluded object is unstable. The second occluded region of the first occluded object may be deleted, and the prediction result of the current prediction will no longer be referenced in the next prediction, thereby reducing the possibility of prediction error.

[0017] In one possible implementation, the processor is further configured to determine the second obscured area as a target obscured area based on the second number being less than or equal to a second number threshold, and determine information about the target obscured area based on information about the second obscured area. If the second number of the plurality of second overlaps that is less than the third overlap threshold is less than or equal to the second number threshold, that is, if there are no multiple consecutive errors in the prediction of the obscured area for the first obscured object and the number of intermittent errors is also relatively small, it can be considered that the accuracy of the prediction of the obscured area for the first obscured object is relatively stable, and the second obscured area can continue to be determined as the target obscured area, thereby achieving accurate obscuration of the obscured object.

[0018] In a second aspect, a lighting control method is provided, which is applied to a lighting system. The lighting system includes a lighting lamp, a target detection module, a jitter detection module and a processor. The lighting lamp, the target detection module and the jitter detection module are all connected to the processor. The lighting lamp includes multiple light sources. The method includes: acquiring and sending a first image through the target detection module to the processor, the first image is an image of a first environment in which the lighting system is located at a first moment; acquiring and sending a posture angle change of the jitter detection module to the processor through the jitter detection module; receiving the first image and the posture angle change through the processor; predicting information of a target shading area in a second image based on the first image and the posture angle change; determining the brightness of multiple light sources based on the information of the target shading area, and controlling the multiple light sources to form areas with different brightness in the second environment according to the corresponding brightness, the second image is an image of the second environment in which the lighting system is located at a second moment, and the second moment is after the first moment.

[0019] In one possible implementation, the lighting system further includes a distance sensing module connected to the processor; and before predicting information about the target obscured area in the second image based on the first image and the attitude angle change, the lighting system further includes:

[0020] The distance sensing module obtains and sends to the processor the distance information between the shielding object and the lighting lamp, where the shielding object is the shielding object in the first environment; the processor receives the distance information; obtains the movement speed of the lighting lamp; and predicts the information of the target shielding area in the second image based on the first image and the change in attitude angle, including: predicting the information of the target shielding area based on the movement speed, distance information, the first image and the change in attitude angle by the processor.

[0021] In one possible implementation, the lighting system also includes a speed sensing module, which is connected to the processor; before obtaining the movement speed of the lighting lamp, it also includes: obtaining the movement speed through the speed sensing module and sending the movement speed to the processor; obtaining the movement speed of the lighting lamp includes: receiving the movement speed through the processor.

[0022] In one possible implementation, the first image includes information about a first shielding area, where the first shielding area indicates a shielding area formed by multiple light sources in the first environment; based on the first image and the attitude angle change, information about a target shielding area in the second image is predicted, including: predicting, by a processor, information about a second shielding area in the second image based on the information about the first shielding area and the attitude angle change; detecting the second image to obtain information about a second detection area in the second image, where the second detection area is the area of ​​the shielding object in the second image; and determining information about the target shielding area based on the information about the second shielding area and the information about the second detection area, where the target shielding area indicates a shielding area formed by multiple light sources in the second environment.

[0023] In one possible implementation, information of the second shielded area in the second image is predicted based on information of the first shielded area and a change in attitude angle, including: determining, by a processor, a coordinate system change between a first coordinate system and a second coordinate system based on a change in attitude angle, the first coordinate system being a coordinate system determined based on the attitude angle of the target detection module at the first moment, and the second coordinate system being a coordinate system determined based on the attitude angle of the target detection module at the second moment; calculating a mapping position of the first shielded area in the second coordinate system, the change between the position of the first shielded area in the first coordinate system and the mapping position being the same as the coordinate system change; and predicting information of the second shielded area based on the mapping position.

[0024] In one possible implementation, determining a coordinate system change between the first coordinate system and the second coordinate system based on a posture angle change of the jitter detection module includes: calculating, by a processor, a posture angle change of the target detection module based on an external parameter calibration result between the jitter detection module and the target detection module and the posture angle change of the jitter detection module; and calculating a coordinate system change based on the posture angle change of the target detection module.

[0025] In one possible implementation, the attitude angle change of the target detection module is calculated based on the external parameter calibration result between the jitter detection module and the target detection module and the attitude angle change of the jitter detection module, including: solving the attitude angle change of the jitter detection module by a processor to obtain a rotation matrix of the jitter detection module; calculating the rotation matrix of the target detection module based on the external parameter calibration result and the rotation matrix of the jitter detection module; and calculating the attitude angle change of the target detection module according to the rotation matrix of the target detection module.

[0026] In one possible implementation, information of the target shielding area is determined based on information of the second shielding area and information of the second detection area, including: determining, by a processor, a first degree of overlap between the second shielding area and a second detection area corresponding to the second shielding area; determining the second shielding area as the target shielding area based on the first degree of overlap being greater than a first degree of overlap threshold; and determining information of the target shielding area based on the information of the second shielding area.

[0027] In one possible implementation, after determining the first degree of overlap between the second masked area and the second detection area corresponding to the second masked area, it also includes: determining, by a processor, whether the first masked object in the second detection area includes a masked object that does not belong to the first image based on the first degree of overlap being less than or equal to a first degree of overlap threshold, to obtain a first determination result; when the first determination result is that the first masked object in the second detection area includes a masked object that does not belong to the first image, updating the information of the second masked area according to the information of the second detection area to obtain the information of the target masked area.

[0028] In one possible implementation, after obtaining the first judgment result, it also includes: obtaining, by the processor, multiple second degrees of overlap corresponding to the first obscured object when the first judgment result is that the first obscured object in the second detection area belongs to the obscured object in the first image, any second degree of overlap among the multiple second degrees of overlap is the degree of overlap between any first detection area including the first obscured object and the corresponding first obscured area; determining the continuous first number of the multiple second degrees of overlap that is less than the second degree of overlap threshold; deleting the second obscured area based on the first number being greater than the first number threshold to obtain the third obscured area of ​​the first image; and determining the information of the target obscured area based on the information of the third obscured area.

[0029] In one possible implementation, after determining the continuous first number of multiple second overlaps that is less than the second overlap threshold, it also includes: determining, by a processor, a second number of multiple second overlaps that is less than a third overlap threshold based on the first number being less than or equal to the first number threshold; deleting the second masked area based on the second number being greater than the second number threshold to obtain a fourth masked area of ​​the first image; and determining information of the target masked area based on the information of the fourth masked area.

[0030] In one possible implementation, after determining the second number of multiple second overlaps that is less than the third overlap threshold, it also includes: determining, by a processor, the second masked area as the target masked area based on the second number being less than or equal to the second number threshold; and determining information of the target masked area based on the information of the second masked area.

[0031] According to a third aspect, an electronic device is provided, on which the lighting system according to the first aspect and possible implementations thereof is installed.

[0032] In one possible implementation, the electronic device includes a vehicle, a drone, or a robot.

[0033] In a fourth aspect, a computer-readable storage medium is provided, in which at least one instruction is stored. The instruction is loaded and executed by a processor to implement the lighting control method in the second aspect or any possible implementation of the second aspect.

[0034] In a fifth aspect, a computer program (product) is provided, which includes a computer program / instructions, and the computer program / instructions are executed by a processor to enable an electronic device to implement the lighting control method in the second aspect or any possible implementation of the second aspect.

[0035] In a sixth aspect, a communication device is provided, comprising: a transceiver, a memory, and a processor. The transceiver, the memory, and the processor communicate with each other via an internal connection path; the memory is configured to store instructions; and the processor is configured to execute the instructions stored in the memory to control the transceiver to receive signals and to control the transceiver to transmit signals. When the processor executes the instructions stored in the memory, the processor performs the method of the second aspect or any possible implementation of the second aspect.

[0036] Optionally, there are one or more processors and one or more memories.

[0037] Optionally, the memory may be integrated with the processor, or the memory may be provided separately from the processor.

[0038] In the specific implementation process, the memory can be a non-transitory memory, such as a read-only memory (ROM), which can be integrated on the same chip as the processor or be set on different chips. This application does not limit the type of memory and the setting method of the memory and the processor.

[0039] In the seventh aspect, a chip is provided, including a processor, for calling and executing program instructions or codes stored in the memory from the memory, so that a communication device equipped with the chip executes the method in the above-mentioned second aspect or any possible implementation of the second aspect.

[0040] In the eighth aspect, another chip is provided, comprising: an input interface, an output interface, a processor and a memory, wherein the input interface, the output interface, the processor and the memory are connected through an internal connection path, and the processor is used to execute the code in the memory. When the code is executed, the processor is used to execute the method in the above-mentioned second aspect or any possible implementation of the second aspect.

[0041] It should be understood that the beneficial effects achieved by the technical solutions of the second to eighth aspects of this application and the corresponding possible implementation methods can be referred to the technical effects of the first aspect and its corresponding possible implementation methods mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] FIG1 is a schematic structural diagram of a lighting system provided in an embodiment of the present application;

[0043] FIG2 is a schematic structural diagram of another lighting system provided in an embodiment of the present application;

[0044] FIG3 is a schematic diagram of a process for estimating the position of an obscured object in a second image according to an embodiment of the present application;

[0045] FIG4 is a schematic diagram of a lighting control process of a lighting system provided by an embodiment of the present application;

[0046] FIG5 is a flowchart of lighting control performed by a lighting system provided in an embodiment of the present application;

[0047] FIG6 is a schematic diagram of an effect of lighting control provided by an embodiment of the present application;

[0048] FIG7 is a flowchart of a lighting control method provided in an embodiment of the present application;

[0049] FIG8 is a process diagram of a lighting control method provided by an embodiment of the present application;

[0050] FIG9 is a schematic diagram of another lighting control method according to an embodiment of the present application;

[0051] FIG10 is a schematic diagram of the composition of an electronic device provided in an embodiment of the present application;

[0052] FIG11 is a schematic diagram of the composition of a vehicle provided in an embodiment of the present application. DETAILED DESCRIPTION

[0053] The terms used in the implementation section of this application are only used to explain the specific embodiments of this application and are not intended to limit this application.

[0054] With the development of lighting control, lighting control methods have become more flexible and diverse. The brightness of the light source used for lighting can be controlled according to different needs and different scenarios, so that the light emitted by the light source can present different effects. For example, when a vehicle is driving in a dark environment such as at night, in a garage or tunnel, it is necessary to turn on the headlights to illuminate the driving environment. The brightness of the light source and the illuminated area can be adjusted according to the brightness of the driving environment and the presence of pedestrians and vehicles to avoid safety hazards.

[0055] For vehicles, lighting control and adjustment of the lighting area can be achieved by switching between high and low beams. For example, when meeting other vehicles at night, the vehicle should switch from high beam to low beam 150 meters away from the oncoming vehicle. Or, when meeting non-motor vehicles on narrow roads or bridges, the vehicle should switch from high beam to low beam when meeting non-motor vehicles. However, if the vehicle driver is inexperienced and has difficulty switching lights in a timely manner when necessary, other motor vehicle or non-motor vehicle drivers and pedestrians in the meeting scene may be dazzled by the high beam of the vehicle that has not switched lights, causing traffic accidents.

[0056] In the field of lighting control, an adaptive driving beam (ADB) system can be used to adaptively switch between high and low beams according to road conditions, avoiding the problem of untimely switching caused by the driver manually switching between high and low beams. In addition, the headlights used for lighting can also have a million-level pixel module, which includes multiple light arrays. By adjusting the brightness of the multiple light arrays to different brightnesses, areas with different brightness can be formed in the lighting area, thereby shielding vehicles or pedestrians in the lighting area. This adaptive light shielding solution can not only prevent vehicles or pedestrians in the lighting area from being dazzled, but also maintain a high lighting brightness for areas in the environment that do not need to be shielded, thereby improving the driver's field of vision and further avoiding safety hazards. With the development of adaptive light shielding technology, the requirements for light shielding technology are getting higher and higher. How to shield the target objects that need to be shielded in a timely and accurate manner has become an urgent problem to be solved.

[0057] An embodiment of the present application provides a lighting system that can flexibly and timely adjust the brightness of a light source used for lighting, form areas of different brightness in the environment where the lighting system is located, realize lighting control, and accurately and timely shield target objects that need to be shielded, that is, shield the shielded objects with light.

[0058] For example, referring to Figure 1, a schematic diagram of the structure of a lighting system is shown. The lighting system includes an object detection module 11, a vibration detection module 12, a lighting fixture 13, and a processor 14. The object detection module 11, the vibration detection module 12, and the lighting fixture 13 can all communicate with the processor 14 via wired or wireless communication.

[0059] The embodiments of the present application do not limit the structure and type of each module in the lighting system. For example, the target detection module 11 may include, for example, a camera, a camera module, a lidar, or an image sensor. The target detection module 11 is used to acquire and send a first image to the processor 14. The first image is an image of the first environment in which the lighting system is located at a first moment. The shake detection module 12 may include, for example, a gyroscope, a suspension system, or an angular velocity sensor. The shake detection module 12 is used to acquire and send the attitude angle change of the shake detection module 12 to the processor 14. If the shake detection module 12 includes a gyroscope, the shake detection module 12 may also be referred to as a gyroscope module or a gyroscope compensation module. The lighting lamp 13 may also be referred to as a lighting control module or a lighting shielding module. The lighting lamp 13 may include multiple light sources, the brightness of which can be adjusted to form areas of different brightness in the environment in which the lighting system is located. The multiple light sources may be, for example, light-emitting diodes (LEDs) or other devices with light emitting functions. The processor 14 may also be referred to as a multi-target tracking module or computing unit. For example, the processor 14 may be a module having data processing capabilities, such as a microcontroller unit (MCU) or a system on chip (SOC). The processor 14 is configured to receive a first image and a change in attitude angle, predict information about a target obscured area in a second image based on the first image and the change in attitude angle, determine the brightness of multiple light sources based on the information about the target obscured area, and control the multiple light sources to form areas of different brightness in a second environment according to the corresponding brightness. The second image is an image of the second environment in which the lighting system is located at a second moment, the second moment being after the first moment.

[0060] The following will further illustrate the role, function and interaction process of each module in the lighting system through examples.

[0061] After the lighting system is activated, the target detection module 11 can acquire images at a first frequency to obtain images of the environment in which the lighting system is located. The first frequency can be an image acquisition frequency set based on experience or user needs, for example, 50 Hertz (Hz), meaning that the target detection module 11 can continuously and averagely acquire 50 frames of images per second. The images acquired by the target detection module 11 can be color images, black and white images, or images synthesized based on a captured point cloud.

[0062] The first image may be an image acquired by the target detection module 11 at the first moment. After acquiring the first image, the target detection module 11 may send the first image to the processor 14 .

[0063] In one possible implementation, before the lighting system begins lighting control, for example, in the preparatory stage before the lighting lamp 13 is turned on, the jitter detection module 12 may first obtain multiple angular velocities and determine the initial error value of the jitter detection module 12 reflected by the multiple angular velocities through an error estimation algorithm, so as to remove interference from the initial error value when subsequently determining the attitude angle change of the jitter detection module 12. The error estimation algorithm may be, for example, an error state Kalman filter algorithm, and the initial error value may include the zero bias and noise matrix of the jitter detection module 12. In some cases, the zero bias and noise matrix of the jitter detection module 12 may also be referred to as the internal parameters of the jitter detection module 12, and the process of determining the internal parameters of the jitter detection module 12 may be referred to as internal parameter calibration of the jitter detection module 12.

[0064] For example, after the lighting system begins lighting control, the shake detection module 12 may obtain its angular velocity at a second frequency. Two angular velocities may be obtained at two moments in time, and the change in the posture angle of the shake detection module 12 between the two moments in time may be obtained by integrating the change in the two angular velocities. Optionally, the second frequency may be greater than or equal to the first frequency.

[0065] Hereinafter, examples are given for the case where the second frequency is equal to the first frequency and the case where the second frequency is greater than the first frequency.

[0066] In the case where the second frequency is equal to the first frequency, the first frequency at which the target detection module 11 acquires images and the second frequency at which the jitter detection module 12 acquires angular velocity can both be 50 Hz. In this case, the time interval between two adjacent frames of images acquired by the target detection module 11 and the time interval between two adjacent angular velocities acquired by the jitter detection module 12 are both 20 milliseconds (ms). The target detection module 11 can acquire image 0 at time t0 and image 1 at time t1, 20 ms after time t0. The jitter detection module 12 can acquire angular velocity 0 at time t0 and angular velocity 1 at time t1. In this case, the attitude angle change of the jitter detection module 12 calculated by the jitter detection module 12 based on angular velocity 0 and angular velocity 1 is the attitude angle change between the acquisition times of the two frames of image 0 and image 1.

[0067] When the second frequency is greater than the first frequency, the first frequency at which the target detection module 11 acquires images can be 50 Hz, and the second frequency at which the jitter detection module 12 acquires angular velocities can be 100 Hz. The time interval between two adjacent image frames acquired by the target detection module 11 is 20 ms, and the time interval between two adjacent angular velocities acquired by the jitter detection module 12 is 10 ms. The target detection module 11 can acquire image 2 at time t2 and image 3 at time t3, 20 ms after time t2. The jitter detection module 12 can acquire angular velocity 2 at time t2 and acquire angular velocity 4 at time t4, 10 ms after time t2. In this case, the attitude angle change calculated by the jitter detection module 12 based on angular velocity 2 and angular velocity 4 is not the attitude angle change between the acquisition times of image 2 and image 3, but rather the attitude angle change from the acquisition time of image 2 to the time before the acquisition time of image 3.

[0068] Regardless of whether the second frequency is greater than the first frequency, it is possible to determine whether the lighting system has experienced an angular shift based on the attitude angle change obtained by the jitter detection module 12. If the attitude angle change is 0 within a period of time, it means that the jitter detection module 12 has not experienced any change in angular velocity during this period of time, that is, the jitter detection module 12 and the lighting system to which it belongs have not experienced an angular shift during this period of time. If the attitude angle change is not 0 within a period of time, it means that the angular velocity of the jitter detection module 12 has changed during this period of time, that is, the jitter detection module 12 and the lighting system to which it belongs have experienced an angular shift during this period of time. After the jitter detection module 12 obtains the attitude angle change, it can send the attitude angle change to the processor 14, so that the processor 14 can determine whether the lighting system has experienced an angular shift.

[0069] Taking the lighting system configured on a vehicle as an example, if the attitude angle change between t0 and t1 is 0, it means that the vehicle has no angular deviation between t0 and t1, and the vehicle may be at a standstill or in a stable driving state; if the attitude angle change between t0 and t1 is not 0, it means that the vehicle has an angular deviation between t0 and t1, and the vehicle may be shaking, or the vehicle may be driving on an unstructured road including speed bumps, potholes, etc.

[0070] After receiving the first image and the attitude angle change, the processor 14 can predict the information of the target obscured area in the second image in various ways. For example, the processor 14 can predict the information of the target obscured area using a prediction model. For example, the prediction model for predicting the information of the target obscured area can be determined based on the information included in the first image or the numerical value of the attitude angle change. The following describes different ways in which the processor 14 determines the prediction model, using Cases A1 and A2 as examples.

[0071] Case A1: determining a prediction model based on information included in the first image.

[0072] In one possible implementation, the first image includes information about the first detection area and information about the first shielding area. The first shielding area may also be referred to as a first shielding frame or a historical track frame (track, trk), indicating the shielding area formed by multiple light sources in the first environment. The first detection area may also be referred to as a first detection frame or a first target frame, indicating the area to which the shielded object in the first image belongs. The information about the first shielding area may include but is not limited to the shape, size, position of the first shielding area, and indication information indicating the shielding object in the first shielding area. The information about the first detection area may include but is not limited to the shape, size, position of the first detection area, and indication information indicating the shielding object in the first detection area.

[0073] Optionally, the information of the first detection area and the information of the first shielded area can be obtained by analyzing the first image by the target detection module 11 and sent to the processor 14. Taking the case where the target detection module 11 autonomously detects and obtains the information of the first detection area as an example, the target detection module 11 can detect the target to be shielded in the first image through a deep learning method, obtain a 2D first detection area, and output the information of the first detection area.

[0074] Alternatively, the information of the first detection area and the information of the first shielding area can also be obtained by the processor 14 analyzing the first image. The embodiment of the present application does not limit the method for determining the information of the first detection area and the information of the first shielding area in the first image.

[0075] Regardless of how the processor 14 obtains the information of the first detection area and the information of the first shielding area, the processor 14 can determine the prediction model based on the overlap between the first detection area and the first shielding area indicated by the information of the first detection area and the information of the first shielding area. Optionally, the number of first detection areas and first shielding areas included in the first image may be multiple, so the prediction model can be determined based on the comprehensive overlap between the multiple first detection areas and the multiple first shielding areas. For example, the comprehensive overlap between the multiple first detection areas and the multiple first shielding areas can be determined by the Hungarian matching algorithm, and the size of the comprehensive overlap and the overlap threshold can be compared, and the prediction model can be determined based on the comparison result. Among them, the Hungarian matching algorithm is an algorithm for solving the maximum weight matching problem of a bipartite graph. In a bipartite graph, each node has a weight, and the goal is to find a match so that the sum of the matching weights is maximized. The input of the Hungarian matching algorithm is an m*n weight matrix, and the output is the path of the maximum match.

[0076] Exemplarily, in an embodiment of the present application, the 2D first detection area in the first image and the 2D first shielded area in the first image are bipartite graphs to be matched, and each first detection area and each first shielded area correspond to a weight. Through the Hungarian matching algorithm, based on the matching cost function, in the case of maximum matching weight, that is, when the number of first shielded areas and first detection areas that can match each other is the largest, the matching relationship between each first detection area and each first shielded area, the matching relationship is the maximum matching path. Wherein, the matching cost function can be at least one of the intersection of union loss (IOU Loss) function, the generalized intersection of union loss (GIOU Loss) function, the distance of union loss (DIOU Loss) function, or the complete intersection of union loss (CIOU Loss) function.

[0077] For example, the first image includes the first detection area 1, the first detection area 2, the first detection area 3, the first shielded area 1, the first shielded area 2 and the first shielded area 3. The weights of each first shielded area and each first detection area are the same. The first detection area 1 can be matched with the first shielded area 1, the first shielded area 2 and the first shielded area 3. The first detection area 1 is pre-matched with the first shielded area 1, and the matching degree 1 and weight 1 are 80%. The first detection area 1 is pre-matched with the first shielded area 2, and the matching degree 2 and weight 2 are 60%. The first detection area 1 is pre-matched with the first shielded area 3, and the matching degree 3 and weight 3 are 40%. It can be determined that the weight and matching degree of the first detection area 1 are the highest when it matches the first shielded area 1, and it is considered that the first detection area 1 matches the first shielded area 1. Based on the same process, the first detection area 2 is pre-matched with the first obscured area 1, the first obscured area 2, and the first obscured area 3. If the weight and degree of match between the first detection area 2 and the first obscured area 2 are the highest, at 90%, then the first detection area 2 and the first obscured area 2 are considered to match. Similarly, based on the same process, the first detection area 3 is pre-matched with the first obscured area 1, the first obscured area 2, and the first obscured area 3. If the weight and degree of match between the first detection area 3 and the first obscured area 3 are the highest, at 85%, then the first detection area 3 and the first obscured area 3 are considered to match. Using the Hungarian matching algorithm, it is determined that, under the maximum weight, the first detection area 1 and the first obscured area 1 match, the first detection area 2 and the first obscured area 2 match, and the first detection area 3 and the first obscured area 3 match. The comprehensive degree of overlap between multiple first obscured areas and multiple first detection areas can be determined based on the average of the multiple maximum weights determined, for example, 85%.

[0078] After determining the comprehensive overlap between the multiple first detection areas and the multiple first shielding areas, the comprehensive overlap and the overlap threshold can be compared, and a prediction model can be determined based on the comparison result. For example, if the comprehensive overlap is greater than the overlap threshold, it means that the information of the multiple first shielding areas predicted last time is relatively accurate, and the prediction model used last time can be continued to be used. Alternatively, if the comprehensive overlap is greater than the overlap threshold, it can be considered that the environment in which the lighting system is currently located is relatively simple, the motion trajectory of the shielding objects in the environment is relatively regular, and the motion law of the lighting system conforms to the law of linear motion, which is easy to achieve accurate prediction. Therefore, the model used last time can be ignored, and a simpler prediction model suitable for the current environment and the motion law of the lighting system can be selected, such as a linear motion model or a linear motion model based on Kalman filtering, to improve prediction efficiency and reduce resource consumption while achieving accurate prediction.

[0079] If the comprehensive overlap is less than or equal to the overlap threshold, it can be assumed that the lighting system's current environment is complex, the motion trajectory of the obstructing objects in the environment and the motion pattern of the lighting system are unstable, and the motion pattern of the lighting system does not conform to the law of linear motion. Therefore, a more sophisticated prediction model suitable for the current environment can be selected to ensure prediction accuracy. Examples of such a sophisticated prediction model include a motion compensation model, a camera motion compensation model based on a Kalman filter, or a jitter compensation algorithm model.

[0080] According to the method in case A1, the prediction model used for each use is determined. The prediction accuracy of the information of the first shielding area and the prediction state reflected by the prediction accuracy can be determined through the record of the matching between the first detection area and the first shielding area, and then it is determined whether to switch the prediction model. That is, whether the prediction model needs to be switched is determined by the accuracy of the last prediction, thereby ensuring high adaptability of the prediction model to the environment in which the lighting system is located, thereby improving the accuracy of the prediction.

[0081] In case A2, the prediction model is determined based on the numerical value of the attitude angle change.

[0082] In an embodiment of the present application, the moment of starting to predict the information of the target detection area can be the third moment between the first moment and the second moment, so the processor 14 can determine the prediction model for predicting the information of the target obscured area based on the change in posture angle between the first moment and the third moment.

[0083] For example, if the attitude angle change between the first moment and the third moment is zero, it indicates that the lighting system has not experienced any angular offset between the first moment and the third moment, and the lighting system is moving according to the law of linear motion, or the lighting system is stationary. In this case, the linear motion model can be selected as the prediction model. If the attitude angle change between the first moment and the third moment is not zero, it indicates that the lighting system has experienced any angular offset between the first moment and the third moment, and the lighting system is not moving according to the law of linear motion, or the lighting system is shaking. In this case, the motion compensation model can be selected as the prediction model.

[0084] Determining the prediction model used each time according to the method in case A2 can reduce the impact of the jitter of the lighting system itself and improve the accuracy of the prediction.

[0085] In one possible implementation, regardless of whether the comprehensive overlap corresponding to the first image is greater than a threshold, and regardless of whether the change in attitude angle between the first and third moments is zero, a user-specified prediction model can be selected to predict information about the target obscured area. This embodiment of the present application can switch prediction models based on the motion state of the lighting system and historical prediction states, using a prediction model adapted to the lighting system's environment for each prediction, thereby improving prediction flexibility and accuracy.

[0086] The target detection module 11 may have a low first frequency of acquiring images due to limitations of its own structure. Therefore, before acquiring the second image, the processor 14 performs an advance prediction of the target occlusion area, effectively compensating for the occlusion offset caused by the target detection module 11 acquiring images at the lower first frequency. That is, the processor 14 performs a prediction at a third moment between the first moment and the second moment, and is able to predict the information of the target occlusion area before the target detection module 11 acquires the second image, and implement lighting control based on the information of the target occlusion area, thereby achieving light occlusion in a timely manner, rather than relying on delayed lighting control based on the images acquired by the target detection module 11. The prediction frequency is higher than the frequency of image acquisition by the target detection module 11, and the lighting control frequency is also higher than the frequency of image acquisition by the target detection module 11, thereby avoiding occlusion offset caused by untimely lighting control, and further preventing the occlusion area formed by the lighting lamp 13 from being unable to cover the occluded object.

[0087] After determining the prediction model for predicting the information of the target obscured area, the prediction model can be used to predict the information of the target obscured area based on the first image and the change in attitude angle. In one possible implementation, the embodiment of the present application can also predict the information of the target obscured area without relying on the prediction model, that is, the embodiment of the present application can also predict the information of the target obscured area based on the first image and the change in attitude angle without relying on the prediction model. Below, the process of predicting the information of the target obscured area based on the first image and the change in attitude angle according to the embodiment of the present application is exemplarily described, and the following process is applicable to both the prediction method of predicting the information of the target obscured area through the prediction model and the prediction method of predicting the information of the target obscured area without relying on the model.

[0088] Optionally, when processor 14 predicts information about the target obstructed area, in addition to the first image and the attitude angle change, it may also be based on information about the distance between the obstructing object in the first environment and the illuminator 13, as well as the movement speed of the illuminator 13, to achieve more accurate prediction results. Therefore, before predicting information about the target obstructed area in the second image based on the first image and the attitude angle change, processor 14 may also obtain the distance information and movement speed.

[0089] For example, see Figure 2, which shows a schematic diagram of the structure of another lighting system. Compared to the lighting system shown in Figure 1, the lighting system shown in Figure 2 adds a distance sensing module 15 and a speed sensing module 16. Both the distance sensing module 15 and the speed sensing module 16 are connected to the processor 14 via a wired or wireless connection. Optionally, the speed sensing module 16 can be located within the distance sensing module 15, or the distance sensing module 15 can be located within the speed sensing module 16. In addition, the lighting system can include the distance sensing module 15 but not the speed sensing module 16, or the lighting system can also include the speed sensing module 16 but not the distance sensing module 15.

[0090] The embodiment of the present application does not limit the manner in which the processor 14 obtains distance information. For example, the processor 14 can calculate the distance between the obstructing object and the target detection module 11 by analyzing the position of the obstructing object in the first image, and then determine the distance information between the obstructing object and the illuminating lamp 13 based on the relative position of the target detection module 11 and the illuminating lamp 13. The second image is an image of the second environment in which the lighting system is located at the second moment. For example, by analyzing the position of the obstructing object A in the first image, the processor 14 determines that the distance between the obstructing object A and the target detection module 11 in the first direction is S1, the illuminating lamp 13 is in the first direction of the target detection module 11, and the distance between the target detection module 11 and the illuminating lamp 13 in the first direction is S2. Then, the distance information between the obstructing object and the illuminating lamp 13 can be determined to be S3, where S3 is the sum of S1 and S2.

[0091] For another example, as mentioned above, the lighting system may further include a distance sensing module 15, which may include, for example, a laser radar or a distance sensor. Distance sensing module 15 may obtain and send to processor 14 the distance information between the obstructing object and lighting fixture 13; processor 14 receives the distance information, thereby enabling processor 14 to obtain the distance information.

[0092] Accordingly, the embodiment of the present application does not limit the manner in which the processor 14 obtains the motion speed. For example, the processor 14 may obtain the image preceding the first image, and determine the motion speed of the lighting lamp 13 by comparing the position change of the same object in the first image and the previous image.

[0093] For another example, the lighting system may further include a speed sensing module 16, which may include a speed sensor. The speed sensing module 16 may acquire and send the motion speed to the processor 14; the processor 14 receives the motion speed, thereby enabling the processor 14 to acquire the motion speed.

[0094] Regardless of the method used by the processor 14 to obtain the motion speed and distance information, the processor 14 can predict the information of the target shielding area based on the motion speed, distance information, the first image and the attitude angle change.

[0095] Exemplarily, the processor 14 can predict the distance information between the lighting lamp 13 and the shielding object based on the distance information between the lighting lamp 13 and the shielding object and the movement speed of the lighting lamp 13, and comprehensively determine the information of the target shielding area based on the distance information between the lighting lamp 13 and the shielding object, the first image and the change in attitude angle, so that the determined information of the target shielding area is more accurate.

[0096] In the embodiment of the present application, the speed sensing module 16 can efficiently sense the movement speed of the lighting lamp 13, and the distance sensing module 15 can efficiently sense the distance information between the shielding object and the lighting lamp 13. Therefore, the position of the shielding object in the second environment can be predicted more efficiently and accurately through the sensed movement speed and distance information, and more accurate information of the target shielding area can be predicted through the prediction model.

[0097] In one possible implementation, the lighting system provided in the embodiment of the present application can not only directly predict the information of the target obscured area based on the first image and the attitude angle change, as well as the optional motion speed and distance information, but can also correct and filter the predicted obscured area information in combination with the second image to obtain more accurate information of the target obscured area.

[0098] Exemplarily, the processor 14 can predict information of a second shielding area in the second image based on information of the first shielding area and the amount of change in attitude angle; detect the second image to obtain information of a second detection area in the second image, where the second detection area is the area of ​​the shielding object in the second image; determine information of a target shielding area based on information of the second shielding area and information of the second detection area, where the target shielding area indicates a shielding area formed by multiple light sources in the second environment.

[0099] Based on the previous description, it can be seen that the first image includes information of the first occluded area. The processor 14 can determine the position of the occluded object in the first image based on the information of the first occluded area, and then predict the information of the second occluded area in the second image based on the information of the first occluded area and the change in posture angle.

[0100] The embodiment of the present application does not limit the method by which the processor 14 predicts the information of the second obscured area based on the information of the first obscured area and the change in attitude angle. Since the jitter detection module 12 and the target detection module 11 belong to the same lighting system, if the attitude angle of the jitter detection module 12 changes, the attitude angle of the target detection module 11 will also change. In the embodiment of the present application, a coordinate system can be established for each frame of image. For example, for the first image acquired at the first moment, a first coordinate system can be established based on the attitude angle of the target detection module 11 at the first moment. For the second image acquired at the second moment, a second coordinate system can be established based on the attitude angle of the target detection module 11 at the second moment. If the attitude angle of the target detection module 11 changes between the first moment and the second moment, there is an offset between the first coordinate system and the second coordinate system. The magnitude of the offset can be reflected by the coordinate system change between the first coordinate system and the second coordinate system. The coordinate system change can also be called a pixel coordinate system change or a pixel offset.

[0101] For example, the processor 14 can determine the coordinate system change between the first coordinate system and the second coordinate system based on the attitude angle change, calculate the mapping position of the first shielded area in the second coordinate system, and predict information about the second shielded area based on the mapping position. The first coordinate system is a coordinate system determined based on the attitude angle of the target detection module 11 at the first moment, and the second coordinate system is a coordinate system determined based on the attitude angle of the target detection module 11 at the second moment. The change between the position of the first shielded area in the first coordinate system and the mapping position is the same as the coordinate system change.

[0102] In one possible implementation, a mapping relationship exists between the attitude angle change of the shake detection module 12 and the coordinate system change. The processor 14 can determine the coordinate system change based on the mapping relationship and the attitude angle change. Optionally, the mapping relationship between the attitude angle change of the shake detection module 12 and the coordinate system change can include a first mapping relationship between the attitude angle change of the shake detection module 12 and the attitude angle change of the target detection module 11, and a second mapping relationship between the attitude angle change of the target detection module 11 and the coordinate system change.

[0103] Since there is a first mapping relationship between the attitude angle change of the target detection module 11 and the attitude angle change of the jitter detection module 12, the attitude angle change of the target detection module 11 can be calculated based on the attitude angle change of the jitter detection module 12 and the first mapping relationship, so that the coordinate system change can be determined based on the attitude angle change of the target detection module 11 and the second mapping relationship, thereby realizing accurate prediction of the information of the target obscured area.

[0104] The embodiments of the present application do not limit the first mapping relationship and the second mapping relationship. Below, the process of transforming the posture angle change amount of the jitter detection module 12 to the posture angle change amount of the target detection module 11 according to the first mapping relationship is explained by taking an example.

[0105] Taking the first mapping relationship as the external parameter calibration result between the jitter detection module 12 and the target detection module 11 as an example, the processor 14 can calculate the attitude angle change of the target detection module 11 based on the external parameter calibration result between the jitter detection module 12 and the target detection module 11 and the attitude angle change of the jitter detection module 12; and calculate the coordinate system change based on the attitude angle change of the target detection module 11.

[0106] Among them, the extrinsic parameter calibration result between the jitter detection module 12 and the target detection module 11 can be obtained by offline calibration of the extrinsic parameter matrix of the jitter detection module 12 and the extrinsic parameter matrix of the target detection module 11. The extrinsic parameter matrix of the jitter detection module 12 is used to describe the position and posture of the jitter detection module 12 in the world coordinate system, including a rotation matrix and a translation vector. The rotation matrix indicates the orientation of the jitter detection module 12, that is, the posture of the jitter detection module 12, and the translation vector indicates the position of the jitter detection module 12. Correspondingly, the extrinsic parameter matrix of the target detection module 11 is used to describe the position and posture of the target detection module 11 in the world coordinate system. The composition of the extrinsic parameter matrix of the target detection module 11 can refer to the above description of the extrinsic parameter matrix of the jitter detection module 12, and will not be repeated here.

[0107] In one possible implementation, the mapping relationship between the attitude angle change of the target detection module 11 and the attitude angle change of the shake detection module 12 is equivalent to the mapping relationship between the rotation matrix of the target detection module 11 and the rotation matrix of the shake detection module 12. Therefore, based on the rotation matrix as an intermediate variable, the conversion from the attitude angle change of the shake detection module 12 to the attitude angle change of the target detection module 11 can be achieved. Exemplarily, the processor 14 can solve the attitude angle change of the shake detection module 12 to obtain the rotation matrix of the shake detection module 12, and based on the external parameter calibration result and the rotation matrix of the shake detection module 12, calculate the rotation matrix of the target detection module 11, and then calculate the attitude angle change of the target detection module 11 according to the rotation matrix of the target detection module 11.

[0108] After determining the change in the attitude angle of the target detection module 11, the change in the coordinate system can be calculated based on a second mapping relationship including a mathematical relationship or a geometric relationship. Based on the coordinate system transformation, the offset between the position of the shielded object in the first image and the position in the second image can be predicted, and then the mapping position of the first shielded area including the shielded object in the second image can be accurately predicted. Since the first image is acquired before the prediction, the first image can be called a historical image, and the second image is acquired after the prediction or during the prediction process, the second image can also be called a current image. Accordingly, the mapping position of the first shielded area including the shielded object in the second image is predicted, that is, the mapping position of the shielded object in the historical image in the current image is estimated, and the information of the second shielded area can be accurately determined based on the mapping position.

[0109] Referring to Figure 3, a schematic diagram of a process for estimating the position of an obscured object in a second image is shown. The jitter detection module 12 obtains the angular velocity between the first moment and the third moment, and integrates the angular velocity at the first moment with the angular velocity at the third moment to obtain the attitude angle change, which includes the change in pitch angle and the change in yaw angle. The coordinate system change between the first coordinate system and the second coordinate system is then calculated based on the geometric relationship. The mapped position of the first obscured area in the second image is calculated based on the coordinate system change and a mathematical model, or the offset between the position of the first obscured area in the first coordinate system and the position of the first obscured area in the second coordinate system is calculated.

[0110] Optionally, in the embodiment of the present application, the mapped position can be directly determined as the second occluded area, and the information of the second occluded area can be determined. Alternatively, the position offset of the second occluded area based on the mapped position can be determined based on the motion speed and distance information, and the offset area can be determined as the second occluded area, and the information of the second occluded area can be determined to achieve the position prediction of the second occluded area.

[0111] In one possible implementation, the processor 14 can acquire a second image. For example, the object detection module 11 can acquire the second image at the second moment and send the second image to the processor 14, which can then receive the second image. After acquiring the second image, the processor 14 can detect the second image and obtain information about the second detection area in the second image.

[0112] The information of the second detection area can accurately reflect the position of the shielding object in the second environment, so the processor 14 can filter and correct the information of the second shielding area according to the accurate position of the shielding object in the second environment, that is, it can associate the same shielding object of the historical image (first image) with the current image (second image) to obtain more accurate information of the target shielding area. The embodiment of the present application does not limit the process of determining the information of the target shielding area based on the information of the second detection area and the information of the second shielding area. For example, the processor 14 can determine the first overlap between the second shielding area and the second detection area; based on the first overlap being greater than the first overlap threshold, determine the second shielding area as the target shielding area; and determine the information of the target shielding area based on the information of the second shielding area. Among them, the first overlap threshold can be set when the lighting system is tested at the factory, or it can be specified by user input.

[0113] The embodiment of the present application does not limit the method for determining the first degree of overlap between the second shielded area and the second detection area. For example, the second detection area and the second shielded area can be matched based on a matching cost function using a Hungarian matching algorithm to obtain the first degree of overlap between the second detection area and the second shielded area.

[0114] An embodiment of the present application predicts the current position of the shielding object based on the historical trajectory indicated by the first shielding area of ​​the shielding object, that is, predicts the second shielding area to which the shielding object belongs based on the first shielding area to which the shielding object belongs, and determines whether the second shielding area is successfully matched or successfully associated with the second detection area by judging the degree of overlap between the predicted second shielding area and the detected second detection area.

[0115] If the first degree of overlap between the second detection area and the second shielding area is higher than the first degree of overlap threshold, it can be considered that the degree of match between the second shielding area and the second detection area is high, that is, the range of overlap between the second shielding area and the second detection area is large, the similarity is high, and the predicted information of the second shielding area is more accurate. Therefore, the information of the second shielding area can be determined as the information of the target shielding area to achieve accurate shielding of the shielded object.

[0116] After determining the first degree of overlap, the processor 14 can also determine whether the first obscured object in the second detection area includes an obscured object that does not belong to the first image based on the first degree of overlap being less than or equal to the first degree of overlap threshold, and obtain a first determination result; when the first determination result is that the first obscured object in the second detection area includes an obscured object that does not belong to the first image, the information of the second obscured area is updated according to the information of the second detection area to obtain the information of the target obscured area.

[0117] If the first overlap between the second detection area and the second occluded area is less than or equal to the first overlap threshold, the second occluded area is considered to be mismatched with the second detection area. The determination can then be made as to whether the mismatch is due to the presence of an occluded object in the second detection area that is not present in the first image. If the occluded object in the second detection area includes an occluded object that is not present in the first image, the second detection area is considered to be an unmatched detection. The mismatch between the second detection area and the second occluded area is due to the occluded object in the second detection area being a newly appearing occluded object in the second environment. Since the newly appearing occluded object was not predicted during the process of predicting the information of the second occluded area, the predicted information of the second occluded area is inaccurate. Therefore, the mismatch between the second occluded area and the second detection area is normal. The information of the second occluded area can be updated based on the information of the second detection area to obtain more accurate information about the target occluded area. For example, the position of the second occluded area can be adjusted based on the position of the second detection area so that the second occluded area covers the second detection area, thereby updating the position of the second occluded area and updating the information of the second occluded area. Afterwards, the second masking frame to which the newly appearing masking object belongs may be initialized as a new masking frame track and added to the track queue so that the track can be referenced in subsequent predictions.

[0118] In another possible implementation, if the first determination result indicates that the first obstructing object in the second detection area is an obstructing object in the first image, multiple second overlaps corresponding to the first obstructing object are obtained, where any second overlap in the multiple second overlaps is the overlap between any first detection area including the first obstructing object and the corresponding first obstructing area; a first number of consecutive overlaps less than a second overlap threshold is determined; based on the first number being greater than the first number threshold, the second obstructing area is deleted to obtain a third obstructing area in the first image; and information about the target obstructing area is determined based on the information about the third obstructing area. The second overlap threshold and the first number threshold can be set when the lighting system is tested at the factory or specified by user input.

[0119] If the first obstructing object in the second detection area is an obstructing object in the first image, multiple second coincidence degrees in the historical record corresponding to the first obstructing object can be obtained to determine whether the obstruction area trajectory corresponding to the first obstructing object is in an active state, thereby determining whether the prediction of the obstruction area of ​​the first obstructing object in the historical record is accurate. The multiple historical records corresponding to the first obstructing object include multiple matching results between the obstruction area to which the first obstructing object belongs and the detection area before a first moment. The obstruction area trajectory corresponding to the first obstructing object includes multiple obstruction areas to which the obstructing object belongs before the first moment. The obstruction area trajectory that is in an active state includes multiple temporally consecutive obstructions that successfully match the detection area. If any obstruction area fails to successfully match the detection area, the obstruction area trajectory including the obstruction area can be deemed not to be in an active state. The obstruction area trajectory that is in an active state means that multiple temporally consecutive obstructions in the obstruction area trajectory have all been successfully matched with the detection area in previous predictions. Therefore, the lighting system's prediction result for the obstruction area is deemed to be of reference value, and the obstruction area still falls within the range for continued prediction.

[0120] If the number of consecutive first images with a plurality of second overlaps that are less than the second overlap threshold is greater than the first number threshold, it is considered that there are many prediction errors for the occluded area of ​​the occluded object in the historical records, and the occluded area track to which the occluded object belongs is not in an activated state. The occluded area can be considered as an unmatched track frame (unmatched track), and the information of the second occluded area including the occluded object can be deleted from the track queue. The prediction result of this time will no longer be referenced in the next prediction, thereby reducing the possibility of prediction errors. After deleting the second occluded area, the third occluded area of ​​the first image is obtained, that is, the other second occluded area of ​​the first image, and the third occluded area is determined as the target occluded area, and the information of the third occluded area is further determined as the target occluded area.

[0121] Alternatively, based on the first number being less than or equal to the first number threshold, a second number of the plurality of second overlaps being less than a third overlap threshold is determined; based on the second number being greater than the second number threshold, the second obscured region is deleted to obtain a fourth obscured region of the first image; and information about the target obscured region is determined based on information about the fourth obscured region. The third overlap threshold and the second number threshold can be set when the lighting system is tested at the factory or can be specified by user input.

[0122] If the number of consecutive first overlaps less than the second overlap threshold among the multiple second overlaps is greater than the first number threshold, it is considered that the prediction errors for the occluded area of ​​the occluded object in the historical records are relatively small. However, if the number of second overlaps less than the third overlap threshold among the multiple second overlaps is greater than the second number threshold, it is considered that the accuracy of the prediction for the occluded area of ​​the occluded object is not stable, or the occluded object has been lost. Therefore, the information of the second occluded area of ​​the occluded object can be deleted from the trajectory queue, and the prediction result of this time will no longer be referenced in the next prediction, thereby reducing the possibility of prediction errors. After deleting the second occluded area, the fourth occluded area of ​​the first image is obtained, that is, the other second occluded area of ​​the first image, and the fourth occluded area is determined as the target occluded area, and the information of the fourth occluded area is further determined as the target occluded area.

[0123] However, if the second number is less than or equal to the second number threshold, the second shielded area can be determined as the target shielded area, and information about the target shielded area is determined based on the information about the second shielded area. If the second number of the plurality of second overlaps that is less than the third overlap threshold is less than or equal to the second number threshold, it is considered that the accuracy of the prediction of the shielded area for the shielding object is relatively stable, and the second shielded area can continue to be determined as the target shielded area, thereby achieving shielding of the shielding object.

[0124] In addition, after determining the information of the target shielding area, the position and state of the shielding object can be updated through Kalman filtering to achieve continuous tracking of the shielding object.

[0125] After determining the information of the target shielding area, the processor 14 can determine the brightness of multiple light sources based on the information of the target shielding area, and control the multiple light sources to form areas of different brightness in the second environment according to the corresponding brightness. For example, the processor 14 can determine the positions that can be illuminated by each light source based on the external parameter calibration results between the target detection module 11 and the lighting lamp 13, and determine the positions that need to be shielded and the positions that do not need to be shielded in the second environment, and then determine the brightness of the light source used to illuminate the position that needs to be shielded as the first brightness, and the brightness of the light source used to illuminate other positions that do not need to be shielded as the second brightness. The first brightness can be lower than the second brightness.

[0126] Taking a traffic scenario as an example, the lighting system is applied to a vehicle, and the lighting lamp 13 can be the vehicle's headlight. The first brightness can be, for example, the brightness of a low beam, or the first brightness can be 0, so that the light source used to illuminate the area that needs to be shielded does not illuminate. The second brightness can be, for example, the brightness of a high beam. Multiple light sources are controlled to illuminate at corresponding brightness levels to form areas of varying brightness in the second environment. The illuminated and shielded areas are compensated in real time to avoid offsets in the illuminated and shielded areas caused by jitter in the lighting system.

[0127] Referring to Figures 4 and 5 , a schematic diagram of the lighting control process and a flowchart of the lighting control process are respectively shown. The high-frequency gyroscope in the shake detection module 12 calculates the attitude angle change; the camera in the target detection module 11 obtains image input and detects information about the second detection area in the second image; based on whether the mask frame successfully matches the detection frame in the historical records, the two cases are divided into successful matching and unsuccessful matching. Different prediction models can be used for each case to predict the information of the second masked area based on the information of the first masked area and the attitude angle change; the information of the second detection area is matched with the information of the second masked area using the Hungarian matching algorithm, and the information of the target masked area is updated based on the matching results; the light source corresponding to the target masked area in the lighting fixture 13 is calculated, and light shielding is achieved by adjusting the brightness of different light sources.

[0128] Referring to FIG6 , a schematic diagram of the effect of lighting control is shown. In part (1) of FIG6 , vehicle A shakes, but the lighting system provided by the embodiment of the present application is not installed on vehicle A, so the shielding area (the area within the dotted line) in the lighting area (the area within the solid line) of the multiple light sources of vehicle A is offset, that is, the shielding area of ​​vehicle A fails to align with the area to which vehicle B belongs, and vehicle B is not effectively shielded. The lower half of vehicle B is still located in the lighting area formed by the multiple light sources of vehicle A, which is prone to safety hazards. In part (2) of FIG6 , vehicle A shakes, and the lighting system provided by the embodiment of the present application is installed on vehicle A, so the shielding area in the lighting area of ​​the multiple light sources of vehicle A does not shift due to the shaking of the vehicle, achieving effective shielding of vehicle B and avoiding safety hazards.

[0129] To summarize, the present application perceives the environment in which the lighting system is located at the first moment through the first image, perceives whether the lighting system has an attitude angle offset through the attitude angle change, and accurately predicts the second environment of the lighting system at the second moment after the offset based on the first environment indicated by the first image and the offset of the lighting system indicated by the attitude angle change, and then accurately predicts the information of the target shielding area that needs to be shielded by light in the second environment, and determines the brightness of multiple light sources, so that at the second moment, multiple light sources can be controlled in time according to the brightness of multiple light sources to form areas with different brightness in the second environment, and the target shielding area can be shielded in time and accurately, thereby achieving precise lighting control, and the lighting control has high robustness.

[0130] In addition, the present application can predict the information of the target shading area at the third moment between the first moment and the second moment, and change the brightness of each light source according to the prediction result. There is no need to wait for the second image to be acquired before performing lighting control. The prediction result can be quickly inserted between the acquisition moments of the two frames of images, and the lighting control is more timely and flexible.

[0131] The present application also provides a lighting control method, which can be applied to the lighting system shown in Figure 1 or Figure 2. The flowchart of the method can be seen in Figure 7, and the method includes but is not limited to the following S701 to S704.

[0132] S701: Acquire a first image through a target detection module and send it to a processor. The first image is an image of a first environment where a lighting system is located at a first moment.

[0133] S702: Acquire the attitude angle change of the jitter detection module through the jitter detection module and send it to the processor.

[0134] S703: Receive the first image and the attitude angle change through the processor; and predict information about the target shielding area in the second image based on the first image and the attitude angle change.

[0135] Optionally, the lighting system also includes a distance sensing module, which is connected to the processor; before predicting the information of the target shading area in the second image based on the first image and the attitude angle change, it also includes: obtaining and sending the distance information between the shielding object and the lighting lamp to the processor through the distance sensing module, where the shielding object is the shielding object in the first environment; receiving the distance information through the processor; obtaining the movement speed of the lighting lamp; predicting the information of the target shading area in the second image based on the first image and the attitude angle change, including: predicting the information of the target shading area based on the movement speed, distance information, the first image and the attitude angle change through the processor.

[0136] In one possible implementation, the lighting system also includes a speed sensing module, which is connected to the processor; before obtaining the movement speed of the lighting lamp, it also includes: obtaining the movement speed through the speed sensing module and sending the movement speed to the processor; obtaining the movement speed of the lighting lamp includes: receiving the movement speed through the processor.

[0137] Exemplarily, the first image includes information about a first shielding area, which indicates a shielding area formed by multiple light sources in the first environment; based on the first image and the attitude angle change, information about a target shielding area in the second image is predicted, including: predicting, by a processor, information about a second shielding area in the second image based on the information about the first shielding area and the attitude angle change; detecting the second image to obtain information about a second detection area in the second image, the second detection area being the area of ​​the shielding object in the second image; determining information about the target shielding area based on the information about the second shielding area and the information about the second detection area, the target shielding area indicating a shielding area formed by multiple light sources in the second environment.

[0138] For example, based on the information of the first shielded area and the change in attitude angle, the information of the second shielded area in the second image is predicted, including: determining, by a processor, the coordinate system change between the first coordinate system and the second coordinate system based on the change in attitude angle, the first coordinate system is a coordinate system determined based on the attitude angle of the target detection module at the first moment, and the second coordinate system is a coordinate system determined based on the attitude angle of the target detection module at the second moment; calculating the mapping position of the first shielded area in the second coordinate system, the change between the position of the first shielded area in the first coordinate system and the mapping position is the same as the coordinate system change; and predicting the information of the second shielded area based on the mapping position.

[0139] Optionally, the coordinate system change between the first coordinate system and the second coordinate system is determined based on the attitude angle change of the jitter detection module, including: calculating, by a processor, the attitude angle change of the target detection module based on the external parameter calibration result between the jitter detection module and the target detection module and the attitude angle change of the jitter detection module; and calculating the coordinate system change based on the attitude angle change of the target detection module.

[0140] For example, based on the external parameter calibration results between the jitter detection module and the target detection module and the attitude angle change of the jitter detection module, the attitude angle change of the target detection module is calculated, including: solving the attitude angle change of the jitter detection module by a processor to obtain the rotation matrix of the jitter detection module; calculating the rotation matrix of the target detection module based on the external parameter calibration results and the rotation matrix of the jitter detection module; and calculating the attitude angle change of the target detection module according to the rotation matrix of the target detection module.

[0141] In one possible implementation, information of the target shielding area is determined based on information of the second shielding area and information of the second detection area, including: determining, by a processor, a first degree of overlap between the second shielding area and a second detection area corresponding to the second shielding area; determining the second shielding area as the target shielding area based on the first degree of overlap being greater than a first degree of overlap threshold; and determining information of the target shielding area based on the information of the second shielding area.

[0142] Optionally, after determining the first degree of overlap between the second masked area and the second detection area corresponding to the second masked area, the process further includes: determining, by a processor, whether the first masked object in the second detection area includes a masked object that does not belong to the first image based on the first degree of overlap being less than or equal to a first degree of overlap threshold, to obtain a first determination result; and when the first determination result is that the first masked object in the second detection area includes a masked object that does not belong to the first image, updating the information of the second masked area according to the information of the second detection area to obtain the information of the target masked area.

[0143] Exemplarily, after obtaining the first judgment result, it also includes: obtaining, through the processor, multiple second coincidences corresponding to the first concealing object when the first judgment result is that the first concealing object in the second detection area belongs to the concealing object in the first image, any second coincidence in the multiple second coincidences is the coincidence between any first detection area including the first concealing object and the corresponding first concealing area; determining the continuous first number of the multiple second coincidences that is less than the second coincidence threshold; deleting the second concealing area based on the first number being greater than the first number threshold to obtain the third concealing area of ​​the first image; and determining the information of the target concealing area based on the information of the third concealing area.

[0144] After determining the continuous first number of multiple second overlaps that is less than the second overlap threshold, the processor can also determine the second number of multiple second overlaps that is less than the third overlap threshold based on the first number being less than or equal to the first number threshold; based on the second number being greater than the second number threshold, delete the second masked area to obtain the fourth masked area of ​​the first image; and determine the information of the target masked area based on the information of the fourth masked area.

[0145] After determining that the second number of multiple second overlaps is less than the third overlap threshold, the processor can also determine the second shielded area as the target shielded area based on the second number being less than or equal to the second number threshold; and determine the information of the target shielded area based on the information of the second shielded area.

[0146] S704, determining the brightness of multiple light sources based on information of the target shielding area, controlling the multiple light sources to form areas of different brightness in the second environment according to the corresponding brightness, wherein the second image is an image of the second environment in which the lighting system is located at a second moment, and the second moment is after the first moment.

[0147] See Figures 8 and 9, both of which show schematic diagrams of the lighting control method. The gyroscope in the shake detection module acquires angular velocity. The shake detection module calculates the attitude angle change of the shake detection module based on the adjacent angular velocities, allowing the processor to calculate the coordinate system change based on the attitude angle change. The processor obtains the historical trajectory and determines whether the intersection of union (IOU) between the detection frame and the mask frame in the previous frame image is greater than a threshold, that is, whether the trajectory frame matches the detection frame in the previous frame. If it is greater than the threshold, the linear motion model is selected; if it is less than or equal to the threshold, the camera motion compensation model is selected. Regardless of which model is selected, Kalman filtering is performed based on the selected model to obtain information about the mask frame of the current frame. Object detection is performed on the image of the current frame captured by the camera in the object detection module to obtain information about the detection frame in the current frame image. Hungarian matching is then performed on the detection frame and the mask frame in the current frame image to determine whether the IOU between the detection frame and the mask frame in the current frame is greater than a threshold. If the association is successful, that is, the IOU is greater than the threshold, the trajectory is updated based on the Kalman filter. If the association fails, determine whether there are any unmatched detection frames. If so, determine whether it is a new track. If so, initialize it as a new mask frame track, add it to the track queue, and output the target mask frame information corresponding to the track. If it is not a new track, determine whether the unmatched track is in the active state. If not, delete the track from the track queue. Otherwise, determine whether the number of frames lost by the track exceeds the threshold, that is, determine whether the number of unmatched frames for the track exceeds the threshold. If so, delete the track from the track queue. Otherwise, retain the track and output the final target mask frame information.

[0148] The methods executed by each module in the lighting control method provided in the embodiment of the present application can refer to the above description of the roles and functions of each module of the lighting system, and the beneficial effects of the lighting control method can also refer to the beneficial effects of the above lighting system, which will not be repeated here.

[0149] The embodiment of the present application further provides an electronic device, as shown in Figure 10 , which is a schematic diagram of the composition of the electronic device. The electronic device is equipped with the above-mentioned lighting system. Optionally, the electronic device may include a vehicle, a drone, or a robot.

[0150] Taking a vehicle as an example of an electronic device, see the schematic diagram of the vehicle composition shown in Figure 11. The gyroscope in the vibration detection module can be integrated with the headlights on the same circuit. The gyroscope can collect data such as angular velocity and send the data such as angular velocity to the processor. The processor calculates the brightness of multiple light sources on the headlights based on the angular velocity and other data, and sends the brightness of multiple light sources to the headlights, so that the headlights form areas with different brightness in the driving environment, thereby achieving light shielding.

[0151] An embodiment of the present application further provides a computer-readable storage medium, in which at least one instruction is stored. The instruction is loaded and executed by a processor to enable the electronic device to implement any of the lighting control methods described above.

[0152] The embodiments of the present application also provide a computer program (product), which, when executed by a computer, can enable a processor or electronic device to execute the corresponding steps and / or processes in the above method embodiments.

[0153] An embodiment of the present application further provides a chip, which includes a processor for calling and executing instructions stored in a memory from the memory, so that an electronic device equipped with the chip executes any of the lighting control methods described above.

[0154] An embodiment of the present application also provides another chip, including: an input interface, an output interface, a processor and a memory, wherein the input interface, the output interface, the processor and the memory are connected via an internal connection path, and the processor is used to execute the code in the memory. When the code is executed, the processor is used to execute any of the lighting control methods described above.

[0155] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments may be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described herein are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated therein. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive).

[0156] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the posture changes involved in this application are all obtained with full authorization.

[0157] Those skilled in the art will appreciate that the various method steps and modules described in conjunction with the embodiments disclosed herein can be implemented in software, hardware, firmware, or any combination thereof. In order to clearly illustrate the interchangeability of hardware and software, the steps and components of each embodiment have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0158] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or may be accomplished by a program to instruct the relevant hardware, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0159] When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer program instructions. As an example, the method of the embodiment of the present application can be described in the context of a machine executable instruction, and the machine executable instruction is such as included in the program module executed in the device on the real or virtual processor of the target. Generally speaking, a program module includes a routine, a program, a library, an object, a class, a component, a data structure, etc., which performs a specific task or realizes a specific abstract data structure. In various embodiments, the function of the program module can be merged or split between the described program modules. The machine executable instruction for the program module can be executed in a local or distributed device. In a distributed device, the program module can be located in both a local and a remote storage medium.

[0160] The computer program code for implementing the method of the embodiment of the application can be written in one or more programming languages. These computer program codes can be provided to the processor of a general-purpose computer or a special-purpose computer so that when the program code is executed by the computer, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on a computer, partially on a computer, as an independent software package, partially on a computer and partially on a remote computer, or entirely on a remote computer or server.

[0161] In the context of the embodiments of the present application, computer program code or related data may be carried by any suitable carrier to enable a device, apparatus, or processor to perform the various processes and operations described above. Examples of carriers include signals, computer-readable media, and the like.

[0162] Examples of signals may include electrical, optical, radio, acoustic or other forms of propagated signals, such as carrier waves, infrared signals, etc.

[0163] A machine-readable medium may be any tangible medium that contains or stores a program for or in connection with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination thereof. More detailed examples of machine-readable storage media include an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0164] Those skilled in the art will clearly understand that, for the sake of convenience and brevity of description, the specific working processes of the above-described systems, devices, and modules can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0165] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the modules is merely a logical function division. In actual implementation, there may be other division methods, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, or can be electrical, mechanical or other forms of connection.

[0166] Modules described as separate components may or may not be physically separate, and components displayed as modules may or may not be physical modules, that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0167] In addition, the functional modules in the various embodiments of the present application may be integrated into a processing module, or each module may exist physically separately, or two or more modules may be integrated into a single module. The above-mentioned integrated modules may be implemented in the form of hardware or software functional modules.

[0168] If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0169] In this application, the terms "first", "second", etc. are used to distinguish between identical or similar items that have substantially the same effects and functions. It should be understood that there is no logical or temporal dependency between "first", "second", and "nth", nor is there any limitation on quantity or order of execution. It should also be understood that although the following description uses the terms first, second, etc. to describe various elements, these elements should not be limited by the terms. These terms are only used to distinguish one element from another. For example, without departing from the scope of the various described examples, a first image may be referred to as a second image, and similarly, a second image may be referred to as a first image. Both the first image and the second image may be images, and in some cases, may be separate and different images.

[0170] It should also be understood that in the various embodiments of the present application, the size of the serial number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0171] In this application, the term "at least one" means one or more, and the term "plurality" means two or more. For example, "plurality of second messages" means two or more second messages. The terms "system" and "network" are often used interchangeably herein.

[0172] It should be understood that the terminology used in the description of the various examples herein is for the purpose of describing particular examples only and is not intended to be limiting. As used in the description of the various examples and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

[0173] It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the listed items. The term "and / or" describes an association between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " in this application generally indicates that the associated objects are in an "or" relationship.

[0174] It will also be understood that the term “comprise” (also known as “includes,” “including,” “comprises,” and / or “comprising”) when used in this specification specifies the presence of stated features, integers, steps, operations, elements, and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0175] It should also be understood that the terms “if” and “if” may be interpreted to mean “when” or “upon” or “in response to determining” or “in response to detecting.” Similarly, the phrases “if it is determined that ” or “if [stated condition or event] is detected” may be interpreted to mean “upon determining ” or “in response to determining ” or “upon detecting [stated condition or event]” or “in response to detecting [stated condition or event],” depending on the context.

[0176] It should be understood that determining B based on A does not mean determining B based solely on A. B can also be determined based on A and / or other information.

[0177] It should also be understood that references throughout this specification to "one embodiment," "an embodiment," or "one possible implementation" mean that specific features, structures, or characteristics associated with that embodiment or implementation are included in at least one embodiment of the present application. Therefore, the appearance of "in one embodiment," "in an embodiment," or "one possible implementation" throughout this specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

Claims

1. A lighting system, characterized in that: The lighting system comprises a lighting lamp, a target detection module, a jitter detection module and a processor, wherein the lighting lamp, the target detection module and the jitter detection module are all connected to the processor, and the lighting lamp comprises a plurality of light sources; The target detection module is used to acquire and send a first image to the processor, where the first image is an image of a first environment where the lighting system is located at a first moment; The jitter detection module is used to obtain and send to the processor the attitude angle change of the jitter detection module; The processor is used to receive the first image and the attitude angle change, predict information of a target shielding area in a second image according to the first image and the attitude angle change, determine the brightness of the multiple light sources based on the information of the target shielding area, and control the multiple light sources to form areas with different brightness in a second environment according to corresponding brightness, wherein the second image is an image of a second environment where the lighting system is located at a second moment, and the second moment is after the first moment.

2. The lighting system according to claim 1, characterized in that The lighting system further comprises a distance sensing module, wherein the distance sensing module is connected to the processor; The distance sensing module is used to obtain and send to the processor the distance information between the shielding object and the lighting lamp, wherein the shielding object is a shielding object in the first environment; The processor is used to receive the distance information, obtain the movement speed of the lighting lamp, and predict the information of the target shielding area according to the movement speed, the distance information, the first image and the attitude angle change.

3. The lighting system according to claim 2, characterized in that The lighting system further comprises a speed perception module, wherein the speed perception module is connected to the processor; The speed perception module is used to obtain and send the movement speed to the processor; The processor is used to receive the movement speed.

4. The lighting system according to any one of claims 1 to 3, characterized in that: The first image includes information of a first shielding area, where the first shielding area indicates a shielding area formed by the multiple light sources in the first environment; The processor is configured to predict information of a second shielded area in the second image according to information of the first shielded area and the amount of change in the attitude angle; Detecting the second image to obtain information of a second detection area in the second image, where the second detection area is an area of ​​the shielded object in the second image; Information of the target shielding area is determined according to information of the second shielding area and information of the second detection area, where the target shielding area indicates a shielding area formed by the multiple light sources in the second environment.

5. The lighting system according to claim 4, characterized in that: The processor is used to determine a coordinate system change amount between a first coordinate system and a second coordinate system according to the attitude angle change amount, wherein the first coordinate system is a coordinate system determined based on the attitude angle of the target detection module at the first moment, and the second coordinate system is a coordinate system determined based on the attitude angle of the target detection module at the second moment; Calculate the mapping position of the first shielded area in the second coordinate system, the change between the position of the first shielded area in the first coordinate system and the mapping position is the same as the change of the coordinate system; predict information of the second shielded area based on the mapping position.

6. The lighting system according to claim 5, characterized in that The processor is used to calculate the attitude angle change of the target detection module based on the external parameter calibration result between the jitter detection module and the target detection module and the attitude angle change of the jitter detection module; and calculate the coordinate system change according to the attitude angle change of the target detection module.

7. The lighting system according to claim 6, characterized in that The processor is used to calculate the attitude angle change of the jitter detection module to obtain the rotation matrix of the jitter detection module; based on the external parameter calibration result and the rotation matrix of the jitter detection module, the rotation matrix of the target detection module is calculated; The attitude angle change of the target detection module is calculated according to the rotation matrix of the target detection module.

8. The lighting system according to any one of claims 4 to 7, characterized in that: The processor is used to determine a first degree of overlap between the second shielded area and a second detection area corresponding to the second shielded area; based on the first degree of overlap being greater than a first degree of overlap threshold, determine the second shielded area as the target shielded area; and determine information of the target shielded area based on information of the second shielded area.

9. The lighting system according to claim 8, characterized in that The processor is further used to determine whether the first obscured object in the second detection area includes an obscured object that does not belong to the first image based on the first overlap being less than or equal to the first overlap threshold, and obtain a first determination result; when the first determination result is that the first obscured object in the second detection area includes an obscured object that does not belong to the first image, update the information of the second obscured area according to the information of the second detection area, and obtain the information of the target obscured area.

10. The lighting system according to claim 9, characterized in that The processor is further configured to, when the first determination result is that the first shielding object in the second detection area belongs to the shielding object in the first image, obtain a plurality of second coincidences corresponding to the first shielding object, wherein any second coincidence among the plurality of second coincidences is a coincidence between any first detection area including the first shielding object and the corresponding first shielding area; Determine a first number of the multiple second overlaps that are smaller than a second overlap threshold; based on the first number being greater than the first number threshold, delete the second masked area to obtain a third masked area of ​​the first image; and determine information about the target masked area based on information about the third masked area.

11. The lighting system according to claim 10, characterized in that The processor is also used to determine a second number of the multiple second overlaps that is less than a third overlap threshold based on the first number being less than or equal to the first number threshold; delete the second masked area to obtain a fourth masked area of ​​the first image based on the second number being greater than the second number threshold; and determine information about the target masked area based on information about the fourth masked area.

12. The lighting system according to claim 11, characterized in that The processor is further configured to determine the second shielded area as the target shielded area based on the second number being less than or equal to the second number threshold; and determine information of the target shielded area according to information of the second shielded area.

13. A lighting control method, characterized in that: The method is applied to a lighting system, the lighting system comprising a lighting lamp, a target detection module, a jitter detection module and a processor, the lighting lamp, the target detection module and the jitter detection module are all connected to the processor, the lighting lamp comprises a plurality of light sources, and the method comprises: Acquire a first image through the target detection module and send it to the processor, where the first image is an image of a first environment where the lighting system is located at a first moment; Acquire the attitude angle change of the jitter detection module through the jitter detection module and send it to the processor; The first image and the attitude angle change are received by the processor; information of a target shielding area in a second image is predicted based on the first image and the attitude angle change; the brightness of the multiple light sources is determined based on the information of the target shielding area, and the multiple light sources are controlled to form areas with different brightness in a second environment according to corresponding brightness, wherein the second image is an image of a second environment where the lighting system is located at a second moment, and the second moment is after the first moment.

14. The method according to claim 13, characterized in that The lighting system further includes a distance sensing module, which is connected to the processor; before predicting information of a target shielding area in the second image according to the first image and the attitude angle change, the system further includes: Acquiring, by the distance sensing module, distance information between the shielding object and the illuminating lamp and sending it to the processor, wherein the shielding object is a shielding object in the first environment; Receiving the distance information through the processor; acquiring the movement speed of the lighting lamp; The predicting, according to the first image and the attitude angle change, information of the target shielding area in the second image includes: The processor predicts information of the target shielding area according to the movement speed, the distance information, the first image and the attitude angle change.

15. The method according to claim 14, characterized in that The lighting system further comprises a speed perception module, wherein the speed perception module is connected to the processor; Before obtaining the movement speed of the lighting lamp, the method further includes: Acquiring the movement speed through the speed perception module and sending the movement speed to the processor; The obtaining the movement speed of the lighting lamp comprises: The speed of movement is received by the processor.

16. The method according to any one of claims 13 to 15, characterized in that: The first image includes information of a first shielding area, where the first shielding area indicates a shielding area formed by the multiple light sources in the first environment; The predicting, according to the first image and the attitude angle change, information of the target shielding area in the second image includes: The processor predicts information of a second shielding area in the second image based on information of the first shielding area and the amount of change in the attitude angle; detects the second image to obtain information of a second detection area in the second image, where the second detection area is an area of ​​a shielded object in the second image; and determines information of a target shielding area based on information of the second shielding area and information of the second detection area, where the target shielding area indicates a shielding area formed by the multiple light sources in the second environment.

17. The method according to claim 16, characterized in that The predicting, according to the information of the first shielded area and the attitude angle change, the information of the second shielded area in the second image comprises: The processor determines a coordinate system change amount between a first coordinate system and a second coordinate system according to the attitude angle change amount, wherein the first coordinate system is a coordinate system determined based on the attitude angle of the target detection module at the first moment, and the second coordinate system is a coordinate system determined based on the attitude angle of the target detection module at the second moment; calculates a mapping position of the first shielded area in the second coordinate system, and a change amount between the position of the first shielded area in the first coordinate system and the mapping position is the same as the coordinate system change amount; and predicts information of the second shielded area based on the mapping position.

18. The method according to claim 17, characterized in that Determining the coordinate system change amount between the first coordinate system and the second coordinate system according to the attitude angle change amount includes: The processor calculates the attitude angle change of the target detection module based on the external parameter calibration result between the jitter detection module and the target detection module and the attitude angle change of the jitter detection module; and calculates the coordinate system change according to the attitude angle change of the target detection module.

19. The method according to claim 18, characterized in that The step of calculating the attitude angle change of the target detection module based on the external parameter calibration result between the jitter detection module and the target detection module and the attitude angle change of the jitter detection module comprises: The processor calculates the attitude angle change of the jitter detection module to obtain the rotation matrix of the jitter detection module; based on the external parameter calibration result and the rotation matrix of the jitter detection module, the rotation matrix of the target detection module is calculated; according to the rotation matrix of the target detection module, the attitude angle change of the target detection module is calculated.

20. The method according to any one of claims 16 to 19, characterized in that: The determining the information of the target shielded area according to the information of the second shielded area and the information of the second detection area includes: The processor determines a first degree of overlap between the second shielded area and a second detection area corresponding to the second shielded area; based on the first degree of overlap being greater than a first degree of overlap threshold, the second shielded area is determined as the target shielded area; and information of the target shielded area is determined based on information of the second shielded area.

21. The method according to claim 20, characterized in that After determining the first overlap between the second shielded area and the second detection area corresponding to the second shielded area, the method further includes: The processor determines, based on the fact that the first overlap is less than or equal to the first overlap threshold, whether the first obscured object in the second detection area includes an obscured object that does not belong to the first image, to obtain a first determination result; when the first determination result is that the first obscured object in the second detection area includes an obscured object that does not belong to the first image, the information of the second obscured area is updated according to the information of the second detection area to obtain the information of the target obscured area.

22. The method according to claim 21, characterized in that After obtaining the first determination result, the method further includes: When the first judgment result is that the first obscured object in the second detection area belongs to the obscured object in the first image, the processor obtains multiple second overlaps corresponding to the first obscured object, and any second overlap among the multiple second overlaps is the overlap between any first detection area including the first obscured object and the corresponding first obscured area; determines the continuous first number of the multiple second overlaps that is less than the second overlap threshold; based on the first number being greater than the first number threshold, deletes the second obscured area to obtain the third obscured area of ​​the first image; and determines the information of the target obscured area according to the information of the third obscured area.

23. The method according to claim 22, characterized in that After determining the continuous first number of the plurality of second overlaps that are less than the second overlap threshold, the method further includes: The processor determines, based on the first number being less than or equal to the first number threshold, a second number of the multiple second overlaps that is less than a third overlap threshold; based on the second number being greater than the second number threshold, the second masked area is deleted to obtain a fourth masked area of ​​the first image; and information of the target masked area is determined based on the information of the fourth masked area.

24. The method according to claim 23, characterized in that After determining the second number of the plurality of second overlaps that is less than the third overlap threshold, the method further includes: The processor determines, based on the second number being less than or equal to the second number threshold, the second shielded area as the target shielded area; and determines information of the target shielded area according to information of the second shielded area.

25. An electronic device, characterized in that: The electronic device is installed with a lighting system as described in any one of claims 1-12.

26. The electronic device according to claim 25, characterized in that: The electronic device includes a vehicle, a drone or a robot.

Images