Ai intelligent control dynamic image projection lamp based on deep learning

Abstract

The invention discloses an Ai intelligent control dynamic image projection lamp based on deep learning. And based on the induction images of the multiple time points, detecting an observation area through an observation object detection network to obtain observation point features. And detecting the overall change of the observation projection based on the sensing positions of the plurality of time points to obtain a predicted sensing position. And adjusting observation point features based on the predicted sensing position to obtain a predicted zoom position. And adjusting the projection lamp based on the predicted zooming position. And outputting a dynamic image into a preset scene, decomposing a dynamic pattern into static patterns, and projecting the static patterns in sequence. Focusing is carried out and movement of the holder base is controlled in real time according to the position of an inductor, and real-time movement change of a projection pattern is achieved.

Description

Technical Field

[0001] The present invention relates to the field of smart home, and in particular to an AI intelligently controlled dynamic image projection lamp based on deep learning. Background Art

[0002] At present, projection lamps are widely used as tools for advertising and pattern display. In existing equipment, when projection lamps are set in advertising windows or floors, they usually display different pictures by rotating pattern pieces driven by synchronous motors. Traditional methods can only realize rotating projection of pictures or rotating projection of several pictures, and cannot display dynamic pictures. If dynamic patterns are to be displayed, projectors are generally used, but their waterproof performance and price limit their use. Summary of the invention

[0003] The purpose of the present invention is to provide an AI intelligent controlled dynamic image projection lamp based on deep learning to solve the above-mentioned problems existing in the prior art.

[0004] In a first aspect, an embodiment of the present invention provides an AI intelligent control dynamic image projection lamp based on deep learning, comprising a camera, a projection lamp, a pan-tilt base and a processor; the projection lamp has a built-in focusing device; The processor is used to perform the following steps: Acquire sensing images and corresponding sensing positions at multiple time points; the sensing position is the position of the sensing object from the sensing head of the projection lamp; the sensing image is an image containing the sensing object taken by the camera on the projection lamp; Based on the sensing images at multiple time points, the observation area is detected through the observation object detection network to obtain the observation point features; the observation point features represent the features of the observation area at the next time point predicted at the current time point; the observation area represents the area where the projection lamp is observed; Based on the sensing positions at multiple time points, the overall change of the observed projection is detected to obtain a predicted sensing position; the predicted sensing position is a two-dimensional position; Based on the predicted sensing position, the observation point feature is adjusted to obtain a predicted zoom position; the predicted zoom position is a three-dimensional position; Based on the predicted zoom position, the projection light is adjusted.

[0005] Optionally, adjusting the observation point feature based on the predicted sensing position to obtain the predicted zoom position includes: Based on the predicted position, the observation point features are adjusted to obtain an adjusted feature matrix; the adjusted feature matrix is ​​a three-dimensional matrix; According to the adjustment feature matrix, a predicted height is obtained; the predicted height is the height distance from the projection lamp at a time point next to the current time point; The point corresponding to the predicted sensing position and the predicted height is taken as the predicted zoom position.

[0006] Optionally, adjusting the projection light based on the predicted zoom position includes: Obtaining a current zoom position and a projection lamp position; the current zoom position indicates the position of the observation area at the current time point; the projection lamp position indicates the position of the projection lamp; Connecting the predicted zoom position with the projection lamp position to obtain a predicted offset line segment; Connecting the current zoom position with the projection lamp position to obtain a current offset line segment; The angle between the predicted offset line segment and the current offset line segment is taken as the rotation angle; Based on the rotation angle and the current time point, get the rotation speed; Adjust the light based on the rotation speed, the predicted offset segment, and the current offset segment.

[0007] Optionally, the sensing images based on multiple time points detect the observation area through an observation object detection network to obtain observation point features, including: Obtaining a first two-dimensional convolution kernel; the length of the first two-dimensional convolution kernel is equal to the length of the sensed image; the length of the first two-dimensional convolution kernel is 2; With a step size of 1, the first two-dimensional convolution kernel is convolved in the width direction of the sensing image to obtain the first eigenvector; The first feature vector represents a change feature of pixel values ​​in a wide direction of the sensing image; Inputting the sensed image into a second convolutional network to obtain a second feature map; the convolution kernel of the second convolutional network is a 2*2 two-dimensional convolution kernel; The width of the second feature map is equal to the number of elements of the first feature vector; The sensing images at multiple time points correspond to obtaining first feature vectors at multiple time points and second feature maps at multiple time points; Based on the first feature vectors and the second feature graphs of the multiple time points, the features of the observation area at the next time point of the current time point are predicted to obtain the observation point features.

[0008] Optionally, the detecting and observing the overall change of the projection based on the sensing positions at multiple time points to obtain the predicted sensing position includes: The sensing position is input into the LSTM neural network, and the characteristics of the sensing position changing over time are found to obtain a first predicted position; Fitting the sensing positions at multiple time points into a curve to obtain a second curve; Obtain the position of the next time point of the current time point in the second curve to obtain a second predicted position; The midpoint between the first predicted position and the second predicted position is used as the predicted sensing position.

[0009] Optionally, the predicting the features of the observation area at the next time point of the current time point based on the first feature vectors and the second feature graphs of the multiple time points to obtain the observation point features includes: Inputting a value of the first feature vectors at multiple time points into the first temporal convolutional network to obtain a first prediction feature vector; and obtaining multiple first prediction feature vectors corresponding to multiple values ​​in the first feature vector; Inputting a row of the second feature map in multiple times into the second time convolutional network to obtain a second prediction feature vector; obtaining multiple second prediction feature vectors corresponding to multiple rows of the second feature map; The multiple first prediction features are fused with the corresponding second prediction features to obtain observation point features.

[0010] Optionally, a marked observation area is obtained; the marked observation area represents a marked position for observing the projection light; The observation point features are trained by marking the observation area.

[0011] Optionally, adjusting the projection light based on the rotation speed, the predicted offset line segment and the current offset line segment includes: According to the rotation speed, the pan / tilt base is adjusted to uniformly adjust the projection light from the direction of the current offset line segment to the direction of the predicted offset line segment; The length of the predicted offset line segment is used as the predicted offset distance; Adjust the focus of the focusing device inside the projector lamp according to the predicted offset distance.

[0012] Optionally, obtaining the rotation speed based on the rotation angle and the current time point includes: The length between two adjacent time points of the captured sensing images is taken as the rotation time length; The rotation angle is divided by the rotation time length to obtain the rotation speed; the rotation speed is the angular velocity.

[0013] Optionally, the camera is fixed in a vertical direction on the front side of the projection lamp; Obtain the dynamic image to be projected; Decomposing the dynamic image into a plurality of static images and storing them in a projection lamp; The focusing device comprises a double-screw motor and four optical axes.

[0014] Compared with the prior art, the embodiments of the present invention achieve the following beneficial effects: The embodiment of the present invention provides an AI intelligent controlled dynamic image projection lamp based on deep learning.

[0015] In the present invention, a dynamic image is output to a preset scene, the dynamic pattern is decomposed into a static pattern, and the static patterns are projected in sequence. By pausing human vision, a complete dynamic image will be presented to the human eye. At the same time, in order to realize that the projection pattern follows the movement of people / objects. A front camera and a sensor are set, and the moving person / object is sensed by the camera, and the position of the observation projection lamp is detected by the AI ​​algorithm. The three-dimensional position of the observation area at the next time point is predicted. And according to the position, the resistance focusing is performed, and a double-screw motor is used to drive to ensure that the focusing is not stuck. Four optical axes are set around to ensure that the focusing process is not offset, ensuring smooth focusing and clear projection patterns. And by adjusting the pan-tilt base, the entire projection lamp can realize X and Y axis rotation, and the projection pattern is moved in real time according to the position of the sensing object. To improve the display effect and interactive fun of the projection lamp. The technical effect of real-time focusing of the projection lamp due to the change of the projection distance during the movement is achieved, ensuring the clarity of the pattern during the projection process and increasing the interactivity and fun. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a flow chart of a method for AI intelligent control of dynamic image projection lamp based on deep learning provided by an embodiment of the present invention.

[0017] Figure 2 It is a structural schematic diagram of a projection lamp, a camera and a sensor head of an AI intelligent control dynamic image projection lamp based on deep learning provided by an embodiment of the present invention.

[0018] Figure 3 It is a schematic diagram of a pan-tilt base of an AI intelligently controlled dynamic image projection lamp based on deep learning provided by an embodiment of the present invention.

[0019] Figure 4 It is a structural schematic diagram of a focusing device of an AI intelligently controlled dynamic image projection lamp based on deep learning provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0020] The present invention will be described in detail below in conjunction with the accompanying drawings.

[0021] Example 1

[0022] like Figure 1 As shown, an embodiment of the present invention provides an AI intelligent control dynamic image projection lamp based on deep learning, including a camera, a projection lamp, a pan-tilt base and a processor; the projection lamp has a built-in focusing device; The processor is used to perform the following steps: S101: Acquire sensing images and corresponding sensing positions at multiple time points; the sensing position is the position of the sensing object from the sensing head of the projection lamp; the sensing image is an image containing the sensing object taken by the camera on the projection lamp.

[0023] The structural diagram of the projection lamp, camera and sensor head is as follows: Figure 2 shown.

[0024] The sensing objects include human bodies and other objects that can observe the projection light.

[0025] The projection lamp is equipped with a camera and a sensor head, and the shooting direction of the camera is the same as the direction of the image in the projection lamp. The camera can shoot people / objects. Through the AI ​​algorithm, the projection pattern can follow the movement of people / objects, which increases interactivity and fun.

[0026] Wherein, a sensor head is fixed on the projection lamp, and the sensor head is used to obtain a sensing position from the projection lamp. The sensing position is a two-dimensional position from the projection lamp, indicating the left-right distance and the front-back distance from the projection lamp.

[0027] S102: Based on the sensing images at multiple time points, the observation area is detected through the observation object detection network to obtain observation point features; the observation point features represent the features of the observation area at the next time point predicted from the current time point; the observation area represents the area where the projection light is observed.

[0028] In this embodiment, the observation area is the area where the eyes of the human body are located.

[0029] S103: Based on the sensing positions at multiple time points, detect and observe the overall change of the projection to obtain a predicted sensing position; the predicted sensing position is a two-dimensional position.

[0030] S104: Based on the predicted sensing position, adjusting the observation point features to obtain a predicted zoom position; the predicted zoom position is a three-dimensional position.

[0031] S105: Adjusting the projection light based on the predicted zoom position.

[0032] Optionally, adjusting the observation point feature based on the predicted sensing position to obtain the predicted zoom position includes: Based on the predicted position, the observation point features are adjusted to obtain an adjusted feature matrix; the adjusted feature matrix is ​​a three-dimensional matrix; According to the adjustment feature matrix, a predicted height is obtained; the predicted height is the height distance from the projection lamp at a time point next to the current time point; The point corresponding to the predicted sensing position and the predicted height is taken as the predicted zoom position.

[0033] Optionally, adjusting the projection light based on the predicted zoom position includes: Obtaining a current zoom position and a projection lamp position; the current zoom position indicates the position of the observation area at the current time point; the projection lamp position indicates the position of the projection lamp; Connecting the predicted zoom position with the projection lamp position to obtain a predicted offset line segment; Connecting the current zoom position with the projection lamp position to obtain a current offset line segment; The angle between the predicted offset line segment and the current offset line segment is taken as the rotation angle; Based on the rotation angle and the current time point, the rotation speed is obtained.

[0034] Wherein, the rotation speed is an angular velocity.

[0035] Adjust the light based on the rotation speed, the predicted offset segment, and the current offset segment.

[0036] Optionally, the sensing images based on multiple time points detect the observation area through the observation object detection network to obtain the observation point features; the observation point features represent the features of the observation area; the observation area represents the area where the projection light is observed, including: Obtaining a first two-dimensional convolution kernel; the length of the first two-dimensional convolution kernel is equal to the length of the sensed image; the length of the first two-dimensional convolution kernel is 2; With a step size of 1, the first two-dimensional convolution kernel is convolved in the width direction of the sensing image to obtain the first eigenvector; The first feature vector represents a change feature of pixel values ​​in a wide direction of the sensing image; The sensed image is input into a second convolutional network to obtain a second feature map; the convolution kernel of the second convolutional network is a 2*2 two-dimensional convolution kernel.

[0037] Among them, the second convolutional network is a convolutional neural network (CNN).

[0038] Wherein, the second feature is a two-dimensional feature map; The width of the second feature map is equal to the number of elements of the first feature vector; The sensing images at multiple time points correspond to obtaining first feature vectors at multiple time points and second feature maps at multiple time points; Based on the first feature vectors and the second feature graphs of the multiple time points, the features of the observation area at the next time point of the current time point are predicted to obtain the observation point features.

[0039] Optionally, the detecting and observing the overall change of the projection based on the sensing positions at multiple time points to obtain the predicted sensing position includes: The sensing position is input into the LSTM neural network, and the characteristics of the sensing position changing over time are found to obtain a first predicted position; The sensing positions at multiple time points are fitted into a curve to obtain a second curve.

[0040] In this embodiment, a polynomial fitting method is used for fitting.

[0041] The second curve represents the movement route of the observation area.

[0042] Wherein, in the second curve, the time point is the horizontal coordinate and the corresponding sensing position is the vertical coordinate.

[0043] Obtain the position of the next time point of the current time point in the second curve to obtain a second predicted position; The midpoint between the first predicted position and the second predicted position is used as the predicted sensing position.

[0044] Optionally, the predicting the features of the observation area at the next time point of the current time point based on the first feature vectors and the second feature graphs of the multiple time points to obtain the observation point features includes: A value in the first feature vectors at multiple time points is input into the first temporal convolutional network to obtain a first predicted feature vector; and multiple values ​​in the first feature vector correspond to obtain multiple first predicted feature vectors.

[0045] In this embodiment, the first temporal convolutional network is a temporal convolutional network (TCN).

[0046] The number of elements in the first prediction feature vector depends on the number of neurons in the last layer of the first time convolutional network.

[0047] A row of the second feature map in multiple times is input into the second time convolutional network to obtain a second prediction feature vector; and multiple rows of the second feature map correspond to multiple second prediction feature vectors.

[0048] In this embodiment, the second temporal convolutional network is a temporal convolutional network (TCN) having a different structure from the first temporal convolutional network.

[0049] The number of elements in the second prediction feature vector depends on the number of neurons in the last layer of the second temporal convolutional network.

[0050] Among them, the number of neurons in the last layer of the first time convolutional network is equal to the number of neurons in the last layer of the second time convolutional network.

[0051] The multiple first prediction feature vectors are fused with the corresponding second prediction feature vectors to obtain observation point features.

[0052] In this embodiment, the first value of the first prediction feature vector and the first value of the second prediction feature vector are averaged and fused to obtain the first value of the observation point feature. The second value of the first prediction feature vector and the second value of the second prediction feature vector are averaged and fused to obtain the second value of the observation point feature.

[0053] Optionally, a marked observation area is obtained; the marked observation area represents a marked position for observing the projection light; The observation point features are trained by marking the observation area.

[0054] The observation point features are input into a neural network to obtain a predicted observation area, and the predicted observation area and the annotated observation area are compared to obtain a loss through a cross entropy loss function.

[0055] Optionally, adjusting the projection light based on the rotation speed, the predicted offset line segment and the current offset line segment includes: According to the rotation speed, the pan / tilt base is adjusted to uniformly adjust the projection light from the direction of the current offset line segment to the direction of the predicted offset line segment.

[0056] The structural diagram of the pan / tilt base is as follows: Figure 3 shown.

[0057] The length of the predicted offset line segment is used as the predicted offset distance; Adjust the focus of the focusing device inside the projector lamp according to the predicted offset distance.

[0058] Optionally, obtaining the rotation speed based on the rotation angle and the current time point includes: The length between two adjacent time points of the captured sensing images is taken as the rotation time length; The rotation angle is divided by the rotation time length to obtain the rotation speed; the rotation speed is the angular velocity.

[0059] Optionally, the camera is fixed in a vertical direction on the front side of the projection lamp.

[0060] The vertical direction is the direction perpendicular to the ground and the shooting direction of the camera is the vertical direction.

[0061] Obtain the dynamic image to be projected; The dynamic image is decomposed into a plurality of static images and stored in a projection lamp.

[0062] In this embodiment, the dynamic pattern is decomposed into 19 static patterns, and the 19 patterns are projected in sequence.

[0063] The focusing device comprises a double-screw motor and four optical axes.

[0064] Wherein, the structure of the focusing device is as follows Figure 4 shown.

[0065] The algorithm and display provided herein are not inherently related to any particular computer, virtual system or other device. Various general purpose systems can also be used together with the teachings based on this. According to the above description, it is obvious that the structure required for constructing such systems. In addition, the present invention is not directed to any specific programming language either. It should be understood that various programming languages ​​can be utilized to realize the content of the present invention described herein, and the description of the above specific languages ​​is for disclosing the best mode of the present invention.

[0066] Similarly, it should be understood that in order to streamline the present disclosure and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting the intention that the claimed invention requires more features than those explicitly recited in each claim. More specifically, as reflected in the claims below, inventive aspects lie in less than all the features of the individual embodiments disclosed above. Therefore, the claims that follow the specific embodiment are hereby expressly incorporated into the specific embodiment, with each claim itself serving as a separate embodiment of the present invention.

[0067] It is understood in the art that the modules in the devices in the embodiments may be adaptively changed and set in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and further may be divided into multiple submodules or subunits or subcomponents. All features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner may be combined in any combination, except that at least some of such features and / or processes or units are mutually exclusive. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature that provides the same, equivalent or similar purpose.

Claims

1. An AI intelligent control dynamic image projection lamp based on deep learning, characterized in that: It includes a camera, a projection lamp, a pan / tilt base and a processor; the projection lamp has a built-in focusing device; The processor is used to perform the following steps: Acquire sensing images and corresponding sensing positions at multiple time points; the sensing position is the position of the sensing object from the sensing head of the projection lamp; the sensing image is an image containing the sensing object taken by the camera on the projection lamp; Based on the sensing images at multiple time points, the observation area is detected through the observation object detection network to obtain the observation point features; the observation point features represent the features of the observation area at the next time point predicted at the current time point; the observation area represents the area where the projection lamp is observed; Based on the sensing positions at multiple time points, the overall change of the observed projection is detected to obtain a predicted sensing position; the predicted sensing position is a two-dimensional position; Based on the predicted sensing position, the observation point feature is adjusted to obtain a predicted zoom position; the predicted zoom position is a three-dimensional position; Based on the predicted zoom position, the projection light is adjusted.

2. According to claim 1, the AI ​​intelligent control dynamic image projection lamp based on deep learning is characterized in that: The step of adjusting the observation point feature based on the predicted sensing position to obtain the predicted zoom position includes: Based on the predicted position, the observation point features are adjusted to obtain an adjusted feature matrix; the adjusted feature matrix is ​​a three-dimensional matrix; According to the adjustment feature matrix, a predicted height is obtained; the predicted height is the height distance from the projection lamp at a time point next to the current time point; The point corresponding to the predicted sensing position and the predicted height is taken as the predicted zoom position.

3. According to claim 1, the AI ​​intelligent control dynamic image projection lamp based on deep learning is characterized in that: The step of adjusting the projection light based on the predicted zoom position comprises: Obtaining a current zoom position and a projection lamp position; the current zoom position indicates the position of the observation area at the current time point; the projection lamp position indicates the position of the projection lamp; Connecting the predicted zoom position with the projection lamp position to obtain a predicted offset line segment; Connecting the current zoom position with the projection lamp position to obtain a current offset line segment; The angle between the predicted offset line segment and the current offset line segment is taken as the rotation angle; Based on the rotation angle and the current time point, get the rotation speed; Adjust the light based on the rotation speed, the predicted offset segment, and the current offset segment.

4. According to claim 1, the AI ​​intelligent control dynamic image projection lamp based on deep learning is characterized in that: The sensing images based on multiple time points detect the observation area through the observation object detection network to obtain the observation point features, including: Obtaining a first two-dimensional convolution kernel; the length of the first two-dimensional convolution kernel is equal to the length of the sensed image; the length of the first two-dimensional convolution kernel is 2; With a step size of 1, the first two-dimensional convolution kernel is convolved in the width direction of the sensing image to obtain the first eigenvector; The first feature vector represents a change feature of pixel values ​​in a wide direction of the sensing image; Inputting the sensed image into a second convolutional network to obtain a second feature map; the convolution kernel of the second convolutional network is a 2*2 two-dimensional convolution kernel; The width of the second feature map is equal to the number of elements of the first feature vector; The sensing images at multiple time points correspond to obtaining first feature vectors at multiple time points and second feature maps at multiple time points; Based on the first feature vectors and the second feature graphs of the multiple time points, the features of the observation area at the next time point of the current time point are predicted to obtain the observation point features.

5. According to claim 1, the AI ​​intelligent control dynamic image projection lamp based on deep learning is characterized in that: The detecting and observing the overall change of the projection based on the sensing positions at multiple time points to obtain the predicted sensing positions includes: The sensing position is input into the LSTM neural network, and the characteristics of the sensing position changing over time are found to obtain a first predicted position; Fitting the sensing positions at multiple time points into a curve to obtain a second curve; Obtain the position of the next time point of the current time point in the second curve to obtain a second predicted position; The midpoint between the first predicted position and the second predicted position is used as the predicted sensing position.

6. According to claim 4, the AI ​​intelligent control dynamic image projection lamp based on deep learning is characterized in that: The feature of the observation area at the next time point of the current time point is predicted based on the first feature vector and the second feature map of the multiple time points to obtain the observation point feature, including: Inputting a value of the first feature vectors at multiple time points into the first temporal convolutional network to obtain a first prediction feature vector; and obtaining multiple first prediction feature vectors corresponding to multiple values ​​in the first feature vector; Inputting a row of the second feature map in multiple times into the second time convolutional network to obtain a second prediction feature vector; obtaining multiple second prediction feature vectors corresponding to multiple rows of the second feature map; The multiple first prediction features are fused with the corresponding second prediction features to obtain observation point features.

7. The deep learning-based AI intelligent control dynamic image projection lamp according to claim 6, characterized in that: Acquire a marked observation area; the marked observation area represents a marked position for observing a projection lamp; The observation point features are trained by marking the observation area.

8. The deep learning-based AI intelligent control dynamic image projection lamp according to claim 3, characterized in that: The step of adjusting the projection light based on the rotation speed, the predicted offset line segment and the current offset line segment comprises: According to the rotation speed, the pan / tilt base is adjusted to uniformly adjust the projection light from the direction of the current offset line segment to the direction of the predicted offset line segment; The length of the predicted offset line segment is used as the predicted offset distance; Adjust the focus of the focusing device inside the projector lamp according to the predicted offset distance.

9. The AI ​​intelligent control dynamic image projection lamp based on deep learning according to claim 3 is characterized in that: The method of obtaining the rotation speed based on the rotation angle and the current time point includes: The length between two adjacent time points of the captured sensing images is taken as the rotation time length; The rotation angle is divided by the rotation time length to obtain the rotation speed; the rotation speed is the angular velocity.

10. The deep learning-based AI intelligent control dynamic image projection lamp according to claim 1, characterized in that: The camera is fixed in the vertical direction of the front side of the projection lamp; Obtain the dynamic image to be projected; Decomposing the dynamic image into a plurality of static images and storing them in a projection lamp; The focusing device comprises a double-screw motor and four optical axes.

Images