A dynamic image projection lamp with AI intelligent control based on deep learning

Abstract

The present invention discloses an AI intelligent control dynamic image projection lamp based on deep learning. Based on the sensed images at multiple time points, the observation area is detected through an object detection network to obtain the observation point features. Based on the sensed positions at multiple time points, the overall change of the observed projection is detected to obtain the predicted sensed position. Based on the predicted sensed position, the observation point features are adjusted to obtain the predicted zoom position. Based on the predicted zoom position, the projection lamp is adjusted. A dynamic image is output into a preset scene, the dynamic pattern is decomposed into static patterns, and the static patterns are projected in sequence. The focus is adjusted in real time according to the position of the sensed object, and the movement of the pan-tilt base is controlled to achieve real-time movement and change of the projected pattern.

Description

Technical Field

[0001] The present invention relates to the field of smart home, and more particularly, to an Ai intelligent control dynamic image projection lamp based on deep learning. Background Art

[0002] Currently, projection lamps are widely used as tools for advertising and pattern display. In existing devices, when the projection lamp installed in an advertising window or on the floor for publicity works, it usually rotates a pattern piece driven by a synchronous motor to display different pictures. The traditional method can only achieve rotational projection of pictures or rotation projection of several pictures, and cannot display dynamic pictures. If you want to display dynamic patterns, a projector is generally used, but its waterproof performance and price limit its use. Summary of the Invention

[0003] The purpose of the present invention is to provide an Ai intelligent control dynamic image projection lamp based on deep learning to solve the above problems existing in the prior art.

[0004] In a first aspect, an embodiment of the present invention provides an Ai intelligent control dynamic image projection lamp based on deep learning, including a camera, a projection lamp, a pan-tilt base, and a processor; a focusing device is built into the projection lamp;

[0005] The processor is used to perform the following steps:

[0006] Obtain induction images at multiple time points and corresponding induction positions; the induction position is the position of the induction object from the induction head of the projection lamp; the induction image is an image containing the induction object captured by the camera on the projection lamp;

[0007] Based on the induction images at multiple time points, through an observation object detection network, detect the observation area to obtain observation point features; the observation point features represent the features of the predicted observation area at the next time point of the current time point; the observation area represents the area where the projection lamp is observed;

[0008] Based on the induction positions at multiple time points, detect the overall change of the observed projection to obtain a predicted induction position; the predicted induction position is a two-dimensional position;

[0009] Based on the predicted induction position, adjust the observation point features to obtain a predicted zoom position; the predicted zoom position is a three-dimensional position;

[0010] Based on the predicted zoom position, adjust the focal length of the pan-tilt base and the focusing device inside the projection lamp.

[0011] Optionally, the step of adjusting the observation point features based on the predicted induction position to obtain a predicted zoom position includes:

[0012] Based on the predicted induction position, adjust the characteristics of the observation point to obtain an adjusted characteristic matrix; the adjusted characteristic matrix is a three-dimensional matrix;

[0013] According to the adjusted characteristic matrix, obtain the predicted height; the predicted height is the distance from the next time point after the current time point to the height of the projection lamp;

[0014] Take the point corresponding to the predicted induction position and the predicted height as the predicted zoom position.

[0015] Optionally, the adjusting the focal lengths of the adjusting pan-tilt base and the in-focus device of the projection lamp based on the predicted zoom position includes:

[0016] Obtain the current zoom position and the projection lamp position; the current zoom position represents the position of the observation area at the current time point; the projection lamp position represents the position where the projection lamp is located;

[0017] Connect the predicted zoom position and the projection lamp position to obtain a predicted offset line segment;

[0018] Connect the current zoom position and the projection lamp position to obtain a current offset line segment;

[0019] Take the angle between the predicted offset line segment and the current offset line segment as the rotation angle;

[0020] Based on the rotation angle and the current time point, obtain the rotation speed;

[0021] Based on the rotation speed, the predicted offset line segment and the current offset line segment, adjust the focal lengths of the pan-tilt base and the in-focus device of the projection lamp.

[0022] Optionally, the detecting the observation area through an observation object detection network based on the induction images at multiple time points to obtain the characteristics of the observation point includes:

[0023] Obtain a first two-dimensional convolution kernel; the length of the first two-dimensional convolution kernel is equal to the length of the induction image; the width of the first two-dimensional convolution kernel is 2;

[0024] With a step size of 1, convolve the first two-dimensional convolution kernel in the width direction of the induction image to obtain a first feature vector;

[0025] The first feature vector represents the change characteristics of the pixel values in the width direction of the induction image;

[0026] Input the induction image into a second convolution network to obtain a second feature map; the convolution kernel of the second convolution network is a 2*2 two-dimensional convolution kernel;

[0027] The width of the second feature map is equal to the number of elements of the first feature vector;

[0028] The sensed images at multiple time points respectively obtain the first feature vectors at multiple time points and the second feature maps at multiple time points;

[0029] Based on the first feature vectors and the second feature maps at the multiple time points, predict the features of the observation area at the next time point of the current time point to obtain the observation point features.

[0030] Optionally, the detecting the overall change of the observation projection based on the sensing positions at multiple time points to obtain the predicted sensing position includes:

[0031] Input the sensing position into an LSTM neural network, find the features of the change of the sensing position over time, and obtain the first predicted position;

[0032] Fit the sensing positions at multiple time points into a curve to obtain a second curve;

[0033] Obtain the position at the next time point of the current time point in the second curve to obtain a second predicted position;

[0034] Take the midpoint of the first predicted position and the second predicted position as the predicted sensing position.

[0035] Optionally, the predicting the features of the observation area at the next time point of the current time point based on the first feature vectors and the second feature maps at the multiple time points to obtain the observation point features includes:

[0036] Input a value in the first feature vectors at multiple time points into a first temporal convolutional network to obtain a first predicted feature vector; multiple values in the first feature vectors respectively obtain multiple first predicted feature vectors;

[0037] Input a row of the second feature maps at multiple times into a second temporal convolutional network to obtain a second predicted feature vector; multiple rows of the second feature maps respectively obtain multiple second predicted feature vectors;

[0038] Fuse the multiple first predicted features and the corresponding second predicted features to obtain the observation point features.

[0039] Optionally, obtain a labeled observation area; the labeled observation area represents the labeled position for observing the projection lamp;

[0040] Train the observation point features through the labeled observation area.

[0041] Optionally, the adjusting the focal lengths of the pan-tilt base and the in-focus device in the projection lamp based on the rotation speed, the predicted offset line segment, and the current offset line segment includes:

[0042] Adjust the pan-tilt base according to the rotation speed, and uniformly adjust the projection lamp from the direction of the current offset line segment to the direction of the predicted offset line segment;

[0043] Take the length of the predicted offset line segment as the predicted offset distance;

[0044] Adjust the focal length of the internal focusing device of the projection lamp according to the predicted offset distance.

[0045] Optionally, obtaining the rotation speed based on the rotation angle and the current time point includes:

[0046] Take the length between two adjacent time points of the captured induction image as the rotation time length;

[0047] Divide the rotation angle by the rotation time length to obtain the rotation speed; the rotation speed is the angular velocity.

[0048] Optionally, the camera is fixed in the vertical direction on the front of the projection lamp;

[0049] Obtain the dynamic image to be projected;

[0050] Decompose the dynamic image into multiple static images and store them in the projection lamp;

[0051] The focusing device includes a double lead screw motor and four optical axes.

[0052] Compared with the prior art, the embodiments of the present invention achieve the following beneficial effects:

[0053] The embodiments of the present invention provide an AI intelligent control dynamic image projection lamp based on deep learning.

[0054] In the present invention, a dynamic image is output to a preset scene, the dynamic pattern is decomposed into static patterns, and the static patterns are projected out in sequence. Utilizing the visual pause of people, the final complete dynamic image will be presented in people's eyes. At the same time, in order to make the projected pattern follow the movement of people / objects. A front camera and a sensor are set. The moving people / objects are sensed by the camera, and the position of the projection lamp is detected through an AI algorithm. The three-dimensional position of the observed area at the next time point is predicted. And resistance focusing is performed according to the position, driven by a double lead screw motor to ensure that the focusing is not stuck, and four optical axes are arranged around to ensure that the focusing process does not deviate, ensuring smooth focusing and clear projected patterns. And by adjusting the pan-tilt base, the entire projection lamp can rotate on the X and Y axes, and the projected pattern can be moved in real time according to the position of the sensing object. To improve the display effect and interactive interest of the projection lamp. It achieves the technical effect of real-time focusing according to the change of the projection distance during the movement of the projection lamp, ensuring the clarity of the pattern during the projection process and increasing the interactivity and interest. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 This is a flowchart of a method for an AI intelligent control dynamic image projection lamp based on deep learning provided by an embodiment of the present invention.

[0056] Figure 2 This is a schematic structural diagram of a projection lamp, a camera, and an induction head of an AI intelligent control dynamic image projection lamp based on deep learning provided by an embodiment of the present invention.

[0057] Figure 3 This is a schematic diagram of a pan-tilt base of an AI intelligent control dynamic image projection lamp based on deep learning provided by an embodiment of the present invention.

[0058] Figure 4 This is a schematic structural diagram of a focusing device of an AI intelligent control dynamic image projection lamp based on deep learning provided by an embodiment of the present invention.

[0059] Markings in the figure: front cover 1; glass 2; focusing assembly 3; pattern sheet 401; pattern disc 402; condenser lens group 501; copper column 502; lamp bead board 503; lamp body 504; pan-tilt base 6; camera 701; induction head 702; power supply and control board assembly 8; rear cover 9; optical axis 301; motor fixing plate 302; lens 303; optical axis sleeve 304; focusing bottom plate 305; lead screw motor 306; lead screw sleeve 307; focusing fixing plate 308. Detailed implementation manners

[0060] The present invention will be described in detail below with reference to the accompanying drawings.

[0061] Embodiment 1

[0062] As Figure 1 shown, an embodiment of the present invention provides an AI intelligent control dynamic image projection lamp based on deep learning, including a camera, a projection lamp, a pan-tilt base, and a processor; a focusing device is built in the projection lamp;

[0063] The processor is used to execute the following steps:

[0064] S101: Obtain induction images and corresponding induction positions at multiple time points; the induction position is the position of the induction object from the induction head of the projection lamp; the induction image is an image containing the induction object captured by the camera on the projection lamp.

[0065] Among them, the schematic structural diagram of the projection lamp, the camera, and the induction head is as Figure 2 shown.

[0066] Among them, the induction object includes a human body and other objects that can observe the projection lamp.

[0067] Among them, a camera and a sensor head are installed on the projection lamp, and the shooting direction of the camera is the same as the direction of the straight surface of the image in the projection lamp. The camera can capture people / objects. Through the AI algorithm, the projection pattern is realized to move following people / objects, increasing interactivity and interest.

[0068] Among them, a sensor head is fixed on the projection lamp, and the sensor head is used to obtain the sensing position from the projection lamp. The sensing position is a two-dimensional position from the projection lamp, indicating the left-right distance and the front-back distance from the projection lamp.

[0069] S102: Based on the sensing images at multiple time points, through the observation object detection network, detect the observation area to obtain the observation point features; the observation point features represent the features of the observation area at the next time point predicted for the current time point; the observation area represents the area for observing the projection lamp.

[0070] Among them, as in this embodiment, the observation area is the area where the eyes of the human body are located.

[0071] S103: Based on the sensing positions at multiple time points, detect the overall change of the observed projection to obtain the predicted sensing position; the predicted sensing position is a two-dimensional position.

[0072] S104: Based on the predicted sensing position, adjust the observation point features to obtain the predicted zoom position; the predicted zoom position is a three-dimensional position.

[0073] S105: Based on the predicted zoom position, adjust the focal lengths of the adjustment pan-tilt base and the in-focus device in the projection lamp.

[0074] Optionally, the adjusting the observation point features based on the predicted sensing position to obtain the predicted zoom position includes:

[0075] Based on the predicted sensing position, adjust the observation point features to obtain an adjustment feature matrix; the adjustment feature matrix is a three-dimensional matrix;

[0076] According to the adjustment feature matrix, obtain the predicted height; the predicted height is the distance from the projection lamp in height at the next time point of the current time point;

[0077] Take the point corresponding to the predicted sensing position and the predicted height as the predicted zoom position.

[0078] Optionally, the adjusting the focal lengths of the adjustment pan-tilt base and the in-focus device in the projection lamp based on the predicted zoom position includes:

[0079] Obtain the current zoom position and the projection lamp position; the current zoom position represents the position of the observation area at the current time point; the projection lamp position represents the position where the projection lamp is located;

[0080] Connect the predicted zoom position with the projection lamp position to obtain a predicted offset line segment;

[0081] Connect the current zoom position with the projection lamp position to obtain a current offset line segment;

[0082] Take the angle between the predicted offset line segment and the current offset line segment as the rotation angle;

[0083] Based on the rotation angle and the current time point, obtain the rotation speed.

[0084] Wherein, the rotation speed is the angular velocity.

[0085] Based on the rotation speed, the predicted offset line segment and the current offset line segment, adjust the focal lengths of the pan-tilt base and the in-focus device of the projection lamp.

[0086] Optionally, based on the sensed images at multiple time points, through an observation object detection network, detect the observation area to obtain observation point features; the observation point features represent the features of the observation area; the observation area represents the area for observing the projection lamp, including:

[0087] Obtain a first two-dimensional convolution kernel; the length of the first two-dimensional convolution kernel is equal to the length of the sensed image; the width of the first two-dimensional convolution kernel is 2;

[0088] With a step size of 1, perform convolution of the first two-dimensional convolution kernel in the width direction of the sensed image to obtain a first feature vector;

[0089] The first feature vector represents the change feature of the pixel values in the width direction of the sensed image;

[0090] Input the sensed image into a second convolution network to obtain a second feature map; the convolution kernel of the second convolution network is a 2*2 two-dimensional convolution kernel.

[0091] Wherein, the second convolution network is a Convolutional Neural Networks (CNN).

[0092] Wherein, the second feature is a two-dimensional feature map;

[0093] The width of the second feature map is equal to the number of elements of the first feature vector;

[0094] The sensed images at multiple time points respectively obtain the first feature vectors at multiple time points and the second feature maps at multiple time points;

[0095] Based on the first feature vectors and the second feature maps at the multiple time points, predict the features of the observation area at the next time point of the current time point to obtain the observation point features.

[0096] Optionally, detecting the overall change of the observed projection based on the induction positions at multiple time points to obtain a predicted induction position, including:

[0097] Inputting the induction position into an LSTM neural network, finding the characteristics of the change of the induction position over time, and obtaining a first predicted position;

[0098] Fitting the induction positions at multiple time points into a curve to obtain a second curve.

[0099] Among them, in this embodiment, a polynomial fitting method is used for fitting.

[0100] Among them, the second curve represents the movement route of the observation area.

[0101] Among them, in the second curve, the time point is used as the abscissa and the corresponding induction position is used as the ordinate.

[0102] Obtaining the position of the next time point of the current time point in the second curve to obtain a second predicted position;

[0103] Taking the midpoint of the first predicted position and the second predicted position as the predicted induction position.

[0104] Optionally, predicting the characteristics of the observation area at the next time point of the current time point based on the first feature vector and the second feature map at the multiple time points to obtain an observation point feature, including:

[0105] Inputting a value in the first feature vectors at multiple time points into a first temporal convolutional network to obtain a first predicted feature vector; multiple values in the first feature vectors respectively obtain multiple first predicted feature vectors.

[0106] Among them, in this embodiment, the first temporal convolutional network is a temporal convolutional network (TCN).

[0107] Among them, the number of elements in the first predicted feature vector depends on the number of neurons in the last layer of the first temporal convolutional network.

[0108] Inputting a row of the second feature maps at multiple times into a second temporal convolutional network to obtain a second predicted feature vector; multiple rows of the second feature maps respectively obtain multiple second predicted feature vectors.

[0109] Among them, in this embodiment, the second temporal convolutional network is a temporal convolutional network (TCN) with a different structure from the first temporal convolutional network.

[0110] Among them, the number of elements in the second predicted feature vector depends on the number of neurons in the last layer of the second temporal convolutional network.

[0111] Among them, the number of neurons in the last layer of the first temporal convolutional network is equal to the number of neurons in the last layer of the second temporal convolutional network.

[0112] Fuse the multiple first prediction feature vectors with the corresponding second prediction feature vectors to obtain the observation point features.

[0113] Among them, in this embodiment, the first value of the first prediction feature vector and the first value of the second prediction feature vector are averaged to perform fusion to obtain the first value of the observation point features. The second value of the first prediction feature vector and the second value of the second prediction feature vector are averaged to perform fusion to obtain the second value of the observation point features.

[0114] Optionally, obtain a labeled observation area; the labeled observation area represents the labeled position for observing the projection lamp.

[0115] Train the observation point features through the labeled observation area.

[0116] Among them, the observation point features are input into a neural network to obtain a predicted observation area, and the loss is obtained by using the cross-entropy loss function for the predicted observation area and the labeled observation area.

[0117] Optionally, the adjusting the pan-tilt base and the focal length of the in-focus device inside the projection lamp based on the rotation speed, the predicted offset line segment, and the current offset line segment includes:

[0118] Adjust the pan-tilt base according to the rotation speed, and uniformly adjust the projection lamp from the direction of the current offset line segment to the direction of the predicted offset line segment.

[0119] Among them, the structural schematic diagram of the pan-tilt base is as Figure 3 shown.

[0120] Take the length of the predicted offset line segment as the predicted offset distance;

[0121] Adjust the focal length of the in-focus device inside the projection lamp according to the predicted offset distance.

[0122] Optionally, the obtaining the rotation speed based on the rotation angle and the current time point includes:

[0123] Take the length between two adjacent time points of capturing the induction image as the rotation time length;

[0124] Divide the rotation angle by the rotation time length to obtain the rotation speed; the rotation speed is the angular velocity.

[0125] Optionally, the camera is fixed in the vertical direction on the front of the projection lamp.

[0126] Wherein, the direction perpendicular to the ground is taken as the vertical direction. The shooting direction of the camera is taken as the vertical direction.

[0127] Obtain the dynamic image to be projected;

[0128] Decompose the dynamic image into multiple static images and store them in the projection lamp.

[0129] Wherein, in this embodiment, the dynamic pattern is decomposed into 19 static patterns, and the 19 patterns are projected in sequence.

[0130] The focusing device includes a double lead screw motor and four optical axes.

[0131] Wherein, the structure of the focusing device is as Figure 4 shown.

[0132] The algorithms and displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used in conjunction with the teachings herein. Based on the above description, the structure required to construct such a system is obvious. In addition, the present invention is not directed to any particular programming language. It should be understood that the content of the present invention described herein can be implemented using various programming languages, and the description of the specific language above is for disclosing the best mode of the present invention.

[0133] Similarly, it should be understood that, in order to streamline the present disclosure and assist in understanding one or more of the various inventive aspects, in the above description of the exemplary embodiments of the present invention, the various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, the disclosed method should not be construed as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the following claims, the inventive aspects lie in less than all of the features of the preceding disclosed single embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into the detailed description, where each claim stands on its own as a separate embodiment of the present invention.

[0134] Those skilled in the art can understand that the modules in the devices in the embodiments can be adaptively changed and arranged in one or more devices different from those embodiments. The modules or units or components in the embodiments can be combined into one module or unit or component, and in addition, they can be divided into multiple sub-modules or sub-units or sub-components. Except that at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all the features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all the processes or units of any method or device thus disclosed. Unless otherwise explicitly stated, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) can be replaced by an alternative feature that provides the same, equivalent, or similar purpose.

Claims

1. An Ai intelligent control dynamic image projection lamp based on deep learning, characterized in that, It includes a camera, a projection lamp, a pan-tilt base, and a processor; a focusing device is built into the projection lamp; The processor is used to perform the following steps: Obtain sensing images and corresponding sensing positions at multiple time points; the sensing position is the position of the sensed object from the sensing head of the projection lamp; the sensing image is an image containing the sensed object captured by the camera on the projection lamp; Obtain a first two-dimensional convolution kernel; the length of the first two-dimensional convolution kernel is equal to the length of the sensing image; the width of the first two-dimensional convolution kernel is 2; With a step size of 1, perform convolution of the first two-dimensional convolution kernel in the width direction of the sensing image to obtain a first feature vector; The first feature vector represents the change feature of the pixel values in the width direction of the sensing image; Input the sensing image into a second convolutional network to obtain a second feature map; the convolutional kernel of the second convolutional network is a 2*2 two-dimensional convolutional kernel; The width of the second feature map is equal to the number of elements of the first feature vector; Sensing images at multiple time points respectively obtain first feature vectors at multiple time points and second feature maps at multiple time points; Input one value in the first feature vectors at multiple time points into a first temporal convolutional network to obtain a first predicted feature vector; Multiple values in the first feature vectors respectively obtain multiple first predicted feature vectors; Input one row of the second feature maps at multiple times into a second temporal convolutional network to obtain a second predicted feature vector; multiple rows of the second feature maps respectively obtain multiple second predicted feature vectors; Fuse the multiple first predicted features with the corresponding second predicted features to obtain an observation point feature; the observation point feature represents the feature of the predicted observation area at the next time point of the current time point; the observation area represents the area for observing the projection lamp; Input the sensing position into an LSTM neural network to find the change feature of the sensing position over time and obtain a first predicted position; Fit the sensing positions at multiple time points into a curve to obtain a second curve; Obtain the position at the next time point of the current time point in the second curve to obtain a second predicted position; Take the midpoint of the first predicted position and the second predicted position as the predicted sensing position; the predicted sensing position is a two-dimensional position; Based on the predicted sensing position, adjust the observation point feature to obtain an adjusted feature matrix; the adjusted feature matrix is a three-dimensional matrix; According to the adjusted feature matrix, obtain a predicted height; the predicted height is the distance from the projection lamp in terms of height at the next time point of the current time point; Take the point corresponding to the predicted sensing position and the predicted height as the predicted zoom position; Based on the predicted zoom position, adjust the focal lengths of the pan-tilt base and the focusing device in the projection lamp.

2. The Ai intelligent control dynamic image projection lamp based on deep learning according to claim 1, wherein, The adjusting the focal lengths of the pan-tilt base and the focusing device in the projection lamp based on the predicted zoom position includes: Obtain the current zoom position and the projection lamp position; the current zoom position represents the position of the observation area at the current time point; the projection lamp position represents the position where the projection lamp is located; Connect the predicted zoom position and the projection lamp position to obtain a predicted offset line segment; Connect the current zoom position and the projection lamp position to obtain a current offset line segment; Take the angle between the predicted offset line segment and the current offset line segment as the rotation angle; Based on the rotation angle and the current time point, obtain the rotation speed; Based on the rotation speed, the predicted offset line segment and the current offset line segment, adjust the focal lengths of the pan-tilt base and the in-focus device of the projection lamp.

3. An Ai intelligent control dynamic image projection lamp based on deep learning according to claim 1, characterized in that, Obtain the marked observation area; the marked observation area represents the marked position for observing the projection lamp; Train the observation point features through the marked observation area.

4. An Ai intelligent control dynamic image projection lamp based on deep learning according to claim 2, characterized in that, The adjusting the focal lengths of the pan-tilt base and the in-focus device of the projection lamp based on the rotation speed, the predicted offset line segment and the current offset line segment includes: Adjust the pan-tilt base according to the rotation speed, and uniformly adjust the projection lamp from the direction of the current offset line segment to the direction of the predicted offset line segment; Take the length of the predicted offset line segment as the predicted offset distance; Adjust the focal length of the in-focus device of the projection lamp according to the predicted offset distance.

5. The Ai intelligent control dynamic image projection lamp based on deep learning according to claim 2, characterized in that, The obtaining the rotation speed based on the rotation angle and the current time point includes: Take the length between two adjacent time points of the captured induction image as the rotation time length; Divide the rotation angle by the rotation time length to obtain the rotation speed; the rotation speed is the angular velocity.

6. The Ai intelligent control dynamic image projection lamp based on deep learning according to claim 1, characterized in that, The camera is fixed in the vertical direction on the front of the projection lamp; Obtain the dynamic image to be projected; Decompose the dynamic image into multiple static images and store them in the projection lamp; The in-focus device includes a double lead screw motor and four optical axes.

Images