Intelligent energy consumption control method and system based on dynamic distribution of building personnel

Abstract

The embodiment of the present invention discloses an intelligent energy consumption control method and system based on the dynamic distribution of building personnel, wherein the method includes: obtaining video data of a first area through multiple cameras; detecting whether there are personnel in the first area, and if so, determining N personnel in the first area, where N is a positive integer; performing the following steps for each of the N personnel: determining N visual coverage ranges corresponding to the N personnel; controlling the illumination angles and illumination brightness of multiple illumination lamps to increase the brightness of the information display area. The implementation of the embodiment of the present application can improve the intelligent degree of brightness adjustment and save power loss.

Description

Technical Field

[0001] The present invention relates to the field of resource scheduling, specifically to the field of intelligent energy consumption control based on dynamic distribution of building personnel, and in particular to an intelligent energy consumption control method, system, electronic equipment and storage medium based on dynamic distribution of building personnel. Background Art

[0002] Currently, in museums, art galleries, large conference rooms, etc., the lights of the entire venue are usually turned on to maximum power, resulting in the consumption of electricity in the venue regardless of whether there are visitors or not, causing a waste of electricity bills in the venue; currently, the screen brightness can be adjusted based on the gaze point position of a single person; but in the case of multiple people, the brightness cannot be accurately and efficiently adjusted based on the gaze point of everyone; therefore, under the advocacy of green energy conservation, how to save electricity in large venues has become a technical problem that needs to be solved urgently. Summary of the invention

[0003] The embodiments of the present invention provide an intelligent energy consumption control method, system, electronic device and storage medium based on the dynamic distribution of building personnel, which can improve the intelligence level of brightness adjustment and save power loss.

[0004] On the one hand, the present invention is an embodiment that provides an intelligent energy consumption control method based on the dynamic distribution of building personnel, and the method is applied to an intelligent energy consumption control system based on the dynamic distribution of building personnel, and the intelligent energy consumption control system based on the dynamic distribution of building personnel is deployed in a target building, and the target building is equipped with multiple cameras and multiple lighting lamps, and the lighting angles of the multiple lighting lamps are adjustable. The method includes: acquiring video data of a first area through the multiple cameras; detecting whether there are personnel in the first area, and if so, determining N personnel in the first area, where N is a positive integer; performing the following steps for each of the N personnel: determining N visual coverage ranges corresponding to the N personnel; determining M visual coverage ranges that intersect with each other among the N visual coverage ranges corresponding to the N personnel, and judging whether M is greater than or equal to a preset threshold; if M is greater than or equal to a preset threshold, threshold, determine the union of the M visual coverage ranges; determine the information display area in the union; control the illumination angles and illumination brightness of the multiple illumination lamps to increase the brightness of the information display area; if the M is less than the preset threshold, determine the M persons corresponding to the M visual coverage ranges, track the real-time gaze points of the M persons through the multiple cameras, and determine whether the gaze points fall in the information display area; if so, control the illumination angles and illumination brightness of the multiple illumination lamps to increase the brightness of the information display area; for the first visual coverage range among the N visual coverage ranges that does not have an intersection, track the real-time gaze points of the persons corresponding to the first visual coverage range through the multiple cameras, and determine whether the gaze points fall in the information display area; if so, control the illumination angles and illumination brightness of the multiple illumination lamps to increase the brightness of the information display area.

[0005] It should be noted that the target building can be a library, art gallery, exhibition hall, large conference hall, etc.; the surrounding walls of the target building can be provided with information display areas, such as calligraphy, art paintings, and item display introduction areas. Currently, large exhibition halls or large conference halls illuminate calligraphy, art paintings, and other areas at maximum brightness regardless of whether there are people there for easy browsing by users.

[0006] It should be noted that the preset threshold is a preset value, for example, the preset threshold is 5.

[0007] It can be seen that in the embodiment of the present application, when it is detected that the visual coverage ranges of multiple people fall within the same area, it means that the probability of this area being browsed by people is relatively high, and the brightness of all information display areas within the union area of ​​this part of people is increased to the maximum; when it is detected that the number of people corresponding to the intersection of the visual coverage ranges is small, or there is no intersection in the visual coverage ranges between the people, the person may be a tour guide or a single viewer, and it is determined whether the person browses the information display area based on the person's gaze point. If so, the brightness of the corresponding information display area is increased to achieve customized brightness adjustment for the person, thereby avoiding waste of electric energy.

[0008] In a possible implementation, determining the N visual coverage ranges corresponding to the N persons includes: performing feature point detection on each person, obtaining a group of feature points on both sides of the shoulders of each person, the group of feature points being at the same level, constructing a first plane based on the group of feature points, the first plane being perpendicular to the horizontal plane, and the group of feature points being located in the first plane; detecting facial feature points of each person; determining a first space where the facial feature points of each person are located relative to the first plane, constructing a vertical line perpendicular to the first plane in the first space with the center point of the group of feature points as the starting point; constructing a first ray and a second ray with the center point as the starting point; the first ray, the second ray and the vertical line are in the same plane and are located on the same side of the first plane; the angles between the first ray and the second ray and the vertical line are both preset angles; constructing a second plane and a third plane based on the first ray and the second ray respectively; the second plane and the third plane are perpendicular to the horizontal plane; determining a first intersection line and a second intersection line of the second plane and the third plane with the object in the first space; the area between the first intersection line and the second intersection line is the visual coverage range of each person.

[0009] The first ray is located in the second plane, and the second ray is located in the third plane.

[0010] Among them, the first plane divides the space into two parts, and the first space is the side where the facial feature points are located; specifically, it can be the side where most of the faces are located or the side where the feature points of the nose of the face are located.

[0011] In the prior art, the visual coverage range is determined based on the human face. However, since the human head often twists, the determined visual coverage range is inaccurate, and the calculation process of real-time tracking and detecting the face orientation is computationally intensive.

[0012] It can be seen that in the embodiment of the present application, the orientation of a person can be determined based on the position of the feature points of the shoulders and the feature points of the face, and then the visual coverage range of the person can be determined, which improves the accuracy of determining the visual coverage and the efficiency of determining the visual coverage.

[0013] In a possible implementation, the feature point detection is performed on each person to obtain a set of feature points on both shoulders of each person, including: after determining N persons in the first area, inputting the video data of the first area into a first feature extraction module for feature extraction to obtain global features; inputting the video data of the first area into a second feature extraction module for feature extraction to obtain local features; splicing the global features with the local features, and sequentially inputting the spliced ​​features into a self-attention module and an average pooling layer to obtain feature points; wherein the first feature extraction module includes a two-dimensional convolutional neural network, The two-dimensional convolutional neural network includes a spatiotemporal feature extraction module; the video data of the first area is input into the first feature extraction module for feature extraction to obtain global features, including: the two-dimensional convolutional neural network includes a first branch, a second branch and a third branch, the first branch includes a convolution layer with a convolution kernel of 1*1, a convolution layer with a convolution kernel of 3*3, and a convolution layer with a convolution kernel of 3*3 in sequence, the second branch includes a convolution layer with a convolution kernel of 1*1, and a convolution layer with a convolution kernel of 3*3 in sequence; the third branch includes a convolution layer with a convolution kernel of 1*1 and the spatiotemporal feature extraction module in sequence; the first branch, the second branch and the third branch are input into the first feature extraction module for feature extraction to obtain global features, the first branch, the second branch and the third branch are input into the second ... The output of the first branch and the output of the third branch are fused to obtain a global feature; the output of the convolution layer with a convolution kernel of 1*1 in the third branch is a second feature; the second feature is input into a spatiotemporal feature extraction module to obtain an output feature; the second feature is input into a spatiotemporal feature extraction module to obtain an output feature, including: performing a two-dimensional convolution on the second feature to obtain a spatiotemporal feature (C, T, W, H), where C is the number of channels, W and H are spatial dimensions, and T is a time dimension, and the spatiotemporal feature is divided into a first spatiotemporal feature and a second spatiotemporal feature based on the channel; a first gating feature is obtained from the first spatiotemporal feature, and a gating feature is obtained from the second spatiotemporal feature. to the second gated feature; time-shift the first gated feature to obtain a forward time-shift feature, and time-shift the second gated feature to obtain a backward time-shift feature; subtract the first gated feature from the first spatiotemporal feature to obtain a first residual feature, and subtract the second gated feature from the second spatiotemporal feature to obtain a second residual feature; the first residual feature and the forward time-shifted feature are fused to obtain the output of the first spatiotemporal feature extraction module, and the second residual feature and the backward time-shifted feature are fused to obtain the output of the second spatiotemporal feature extraction module; the output of the first spatiotemporal feature extraction module and the output of the second spatiotemporal feature extraction module together constitute the output feature of the spatiotemporal feature extraction module;

[0014] The first gating feature and the second gating feature are obtained in the following manner:

[0015] Y1=tanh(W1*X1)⊙X1;

[0016] Y2=tanh(W2*X2)⊙X2;

[0017] Among them, Y1 and Y2 are the first gated feature and the second gated feature respectively, X1 is the first spatiotemporal feature, X2 is the second spatiotemporal feature, X1 and X2 are segmented based on channels, W1 and W2 are 3*3*3 convolution kernels respectively, * is convolution, and ⊙ is Hadamard product.

[0018] In a possible implementation, the video data of the first area is input into a second feature extraction module for feature extraction to obtain local features, including: defining a vocabulary table related to entities of a set of feature points, and merging them into a text prompt; encoding the text prompt through a pre-trained BERT to obtain vocabulary features; inputting the video data of the first area into a pre-trained visual encoder for processing to obtain image region of interest features; aligning and fusing vocabulary features and region of interest features to obtain point of interest entity features; and extracting features of the point of interest entity features through an attention mechanism to obtain local features.

[0019] In a possible implementation, the multiple cameras are four cameras, and the four cameras are located on the ceiling; the lines connecting the position points of the four cameras form a rectangle, and the area covered by the rectangle is larger than the first area.

[0020] In one possible implementation, after determining N persons in the first area, before executing the following steps for each of the N persons, the method also includes: determining whether N is greater than a preset number of persons, and if so, detecting the position movement changes and body rotation changes of each of the N persons within a preset time period; calculating the comprehensive movement change degree and the comprehensive rotation change degree based on the position movement changes and body rotation changes of each person; if the comprehensive movement change degree is greater than the preset movement change degree or the comprehensive rotation change degree is greater than the preset rotation change degree, controlling the multiple illumination lamps to increase the brightness of the information display area corresponding to the first area; if the comprehensive movement change degree is less than or equal to the preset movement change degree or the comprehensive rotation change degree is less than or equal to the preset rotation change degree; then executing the following steps for each of the N persons.

[0021] Among them, whether the people are focused on the information display area is determined based on the comprehensive movement change degree and the comprehensive rotation change degree. If the comprehensive movement change degree or the comprehensive rotation change degree is large, it means that the people are not focused on the content of the information display area, and are more likely to be talking or moving, and the subsequent brightness enhancement step will not be executed.

[0022] It can be seen that in the embodiment of the present application, the attention of the personnel to the information display area can be judged based on the status of the personnel, and the subsequent steps will not be executed when the attention is low, thereby reducing unnecessary waste of electricity.

[0023] In a possible implementation, the detecting the position movement changes and body rotation changes of each of the N persons within a preset time period; calculating the comprehensive movement change degree and the comprehensive rotation change degree based on the position movement changes and body rotation changes of each person, includes: determining the position movement distance of each person based on the starting point and the end point of the preset time period, determining a first proportion of the position movement distances of the N persons that are less than a preset distance, and determining the comprehensive movement change degree according to the first proportion; recording the body posture of each person at the starting point of the preset time period, and calculating the rotation time of each person's body rotation angle greater than the preset rotation angle within the preset time period based on the body posture of each person at the starting point; determining a second proportion of the rotation time less than the preset time among the N persons, and determining the comprehensive rotation change degree according to the second proportion.

[0024] On the other hand, an embodiment of the present invention provides an intelligent energy consumption control system based on dynamic distribution of building personnel, wherein the intelligent energy consumption control system based on dynamic distribution of building personnel is deployed in a target building, wherein the target building is equipped with multiple cameras and multiple illumination lamps, wherein the illumination angles of the multiple illumination lamps are adjustable, and the intelligent energy consumption control system based on dynamic distribution of building personnel is used to perform the following method steps:

[0025] Acquire video data of the first area through the multiple cameras;

[0026] Detect whether there is a person in the first area, and if so, determine N persons in the first area, where N is a positive integer;

[0027] For each of the N persons, perform the following steps:

[0028] Determine N visual coverage areas corresponding to the N persons;

[0029] Determine M visual coverage areas that intersect with each other among the N visual coverage areas corresponding to the N persons;

[0030] Determine whether M is greater than or equal to a preset threshold;

[0031] If M is greater than or equal to a preset threshold, determining the union of the M visual coverage areas;

[0032] Determine the information display area in the union; control the illumination angle and illumination brightness of the multiple illumination lamps to increase the brightness of the information display area;

[0033] If M is less than a preset threshold, the M persons corresponding to the M visual coverage areas are determined, and the real-time gaze points of the M persons are tracked by the multiple cameras to determine whether the gaze points fall in the information display area; if so, the illumination angles and illumination brightness of the multiple illumination lamps are controlled to increase the brightness of the information display area;

[0034] For the first visual coverage range among the N visual coverage ranges that does not have an intersection, the real-time gaze point of the person corresponding to the first visual coverage range is tracked by the multiple cameras to determine whether the gaze point falls in the information display area; if so, the illumination angle and illumination brightness of the multiple illumination lamps are controlled to increase the brightness of the information display area.

[0035] On the other hand, an embodiment of the present invention provides an electronic device, comprising: a processor, suitable for implementing one or more instructions; and a computer storage medium, the computer storage medium storing one or more instructions, the one or more instructions being suitable for being loaded by the processor and executing the above-mentioned intelligent energy consumption control method based on the dynamic distribution of building personnel.

[0036] On the other hand, an embodiment of the present invention provides a computer storage medium, which stores one or more instructions, and the one or more instructions are suitable for being loaded by a processor and executing the above-mentioned intelligent energy consumption control method based on the dynamic distribution of building personnel.

[0037] It can be seen that the present invention can dynamically adjust the brightness of the information display area based on the crowd's field of vision coverage; by extracting information from global features and using supplementary local information to enrich features, a two-dimensional convolutional neural network and a spatiotemporal module are used for global feature extraction to reduce the amount of calculation while maintaining the spatiotemporal context feature extraction relationship. When extracting local features, a pre-trained language encoder is combined with a visual encoder to obtain more accurate local features. In summary, the present application can quickly and dynamically adjust the brightness of an area based on image data processing in the case of multiple people, thereby improving the speed of response and achieving a more accurate energy consumption control effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0039] Figure 1 is a schematic diagram of a scenario provided by an embodiment of the present invention;

[0040] Figure 2 It is a flow chart of an intelligent energy consumption control method based on dynamic distribution of building personnel provided by an embodiment of the present invention;

[0041] Figure 3 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0042] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0043] It should be noted that the terms "first", "second", etc. in the description and claims of the embodiments of the present invention and the above-mentioned drawings are used to distinguish different objects rather than to describe a specific order.

[0044] like Figure 1 As shown, Figure 1 is a schematic diagram of a scenario provided in an embodiment of the present application. An intelligent energy consumption control method based on dynamic distribution of building personnel provided in an embodiment of the present application can be used in Figure 1 In the scene diagram shown, the scene includes four target persons, a wall W1 and a visual coverage range C1, and the four target persons are located in the first area; the intelligent energy consumption control system based on the dynamic distribution of building personnel can determine the corresponding space based on the feature points of the shoulders of the four persons, and determine the visual coverage range of each person within the space. When there is an intersection between the visual coverage ranges and the corresponding number of people is greater than a preset threshold, the union of the visual coverage ranges is determined, and the brightness of the information display area of ​​the union area is increased. The specific preset threshold can be set to 3, that is, when there are more than 3 people, the brightness of the information display area of ​​the union area is increased.

[0045] Based on the above description, an embodiment of the present invention proposes an intelligent energy consumption control method based on dynamic distribution of building personnel. The intelligent energy consumption control method based on dynamic distribution of building personnel can be executed by an electronic device. The method is applied to an intelligent energy consumption control system based on dynamic distribution of building personnel. The intelligent energy consumption control system based on dynamic distribution of building personnel is deployed in a target building. The target building is equipped with multiple cameras and multiple illumination lamps, and the illumination angles of the multiple illumination lamps are adjustable. Please refer to Figure 2 The intelligent energy consumption control method based on dynamic distribution of building personnel may include the following steps S201-S204:

[0046] S201: Acquire video data of a first area through the multiple cameras.

[0047] S202, detecting whether there are people in the first area, and if so, determining N people in the first area, where N is a positive integer.

[0048] S203, performing the following steps for each of the N persons: determining N visual coverage ranges corresponding to the N persons; and determining M visual coverage ranges that intersect with each other among the N visual coverage ranges corresponding to the N persons.

[0049] S204, determining whether M is greater than or equal to a preset threshold.

[0050] S205, if M is greater than or equal to a preset threshold, determining the union of the M visual coverage areas;

[0051] Determine the information display area in the union; control the illumination angle and illumination brightness of the multiple illumination lamps to increase the brightness of the information display area;

[0052] Among them, the preset threshold can be set based on the actual situation, for example, equal to 3, 5, 10, etc.; when there is an intersection between the visual coverage ranges of multiple people, and the number is greater than the preset threshold, it can be determined that there is an information display area on one side of the crowd. Therefore, the brightness of the information display area in the union of the visual coverage ranges can be increased.

[0053] S206, if M is less than a preset threshold, determine the M persons corresponding to the M visual coverage areas, track the real-time gaze points of the M persons through the multiple cameras, and determine whether the gaze points fall in the information display area; if so, control the illumination angles and illumination brightness of the multiple illumination lamps, and increase the brightness of the information display area;

[0054] If there are fewer people at the intersection, it means that the M people are smaller in number than the entire crowd. In a large venue, they may be staff members, such as tour guides. Therefore, the real-time gaze points of the M people are tracked by cameras to determine whether they are viewing the information display area, thereby achieving precise brightness control and saving energy.

[0055] S207, for a first visual coverage range that has no intersection among the N visual coverage ranges, tracking the real-time gaze point of the person corresponding to the first visual coverage range through the multiple cameras, and determining whether the gaze point falls on the information display area; if so, controlling the illumination angle and illumination brightness of the multiple illumination lamps to increase the brightness of the information display area;

[0056] If there is no intersection of people, it means that the number of people is small compared to the entire crowd, for example, there is only one person; in a large venue, they may be staff members, such as lecturers, etc. Therefore, the real-time gaze point of the corresponding person is tracked by the camera in real time to determine whether he is viewing the information display area, thereby achieving precise brightness control and saving energy.

[0057] In a possible implementation, determining the N visual coverage ranges corresponding to the N persons includes: performing feature point detection on each person, obtaining a group of feature points on both sides of the shoulders of each person, the group of feature points being at the same level, constructing a first plane based on the group of feature points, the first plane being perpendicular to the horizontal plane, and the group of feature points being located in the first plane; detecting facial feature points of each person; determining a first space where the facial feature points of each person are located relative to the first plane, constructing a vertical line perpendicular to the first plane in the first space with the center point of the group of feature points as the starting point; constructing a first ray and a second ray with the center point as the starting point; the first ray, the second ray and the vertical line are in the same plane and are located on the same side of the first plane; the angles between the first ray and the second ray and the vertical line are both preset angles; constructing a second plane and a third plane based on the first ray and the second ray respectively; the second plane and the third plane are perpendicular to the horizontal plane; determining a first intersection line and a second intersection line of the second plane and the third plane with the object in the first space; the area between the first intersection line and the second intersection line is the visual coverage range of each person.

[0058] Among them, the first plane divides the space into two parts, and the first space is the side where the facial feature points are located; specifically, it can be the side where most of the faces are located or the side where the feature points of the nose of the face are located.

[0059] In the prior art, the visual coverage range is determined based on the human face. However, since the human head often twists, the determined visual coverage range is inaccurate, and the calculation process of real-time tracking and detecting the face orientation is computationally intensive.

[0060] It can be seen that in the embodiment of the present application, the orientation of a person can be determined based on the position of the feature points of the shoulders and the feature points of the face, and then the visual coverage range of the person can be determined, thereby avoiding the problem of large fluctuations and errors in the visual coverage range caused directly based on the orientation of the face; while improving the accuracy of determining the visual coverage, the efficiency of determining the visual coverage range is also improved.

[0061] In a possible implementation, the feature point detection is performed on each person to obtain a set of feature points on both shoulders of each person, including: after determining N persons in the first area, inputting the video data of the first area into a first feature extraction module for feature extraction to obtain global features; inputting the video data of the first area into a second feature extraction module for feature extraction to obtain local features; splicing the global features with the local features, and sequentially inputting the spliced ​​features into a self-attention module and an average pooling layer to obtain feature points; wherein the first feature extraction module includes a two-dimensional convolutional neural network, The two-dimensional convolutional neural network includes a spatiotemporal feature extraction module; the video data of the first area is input into the first feature extraction module for feature extraction to obtain global features, including: the two-dimensional convolutional neural network includes a first branch, a second branch and a third branch, the first branch includes a convolution layer with a convolution kernel of 1*1, a convolution layer with a convolution kernel of 3*3, and a convolution layer with a convolution kernel of 3*3 in sequence, the second branch includes a convolution layer with a convolution kernel of 1*1, and a convolution layer with a convolution kernel of 3*3 in sequence; the third branch includes a convolution layer with a convolution kernel of 1*1 and the spatiotemporal feature extraction module in sequence; the first branch, the second branch and the third branch are input into the first feature extraction module for feature extraction to obtain global features, the first branch, the second branch and the third branch are input into the second ... The output of the first branch and the output of the third branch are fused to obtain a global feature; the output of the convolution layer with a convolution kernel of 1*1 in the third branch is a second feature; the second feature is input into a spatiotemporal feature extraction module to obtain an output feature; the second feature is input into a spatiotemporal feature extraction module to obtain an output feature, including: performing a two-dimensional convolution on the second feature to obtain a spatiotemporal feature (C, T, W, H), where C is the number of channels, W and H are spatial dimensions, and T is a time dimension, and the spatiotemporal feature is divided into a first spatiotemporal feature and a second spatiotemporal feature based on the channel; a first gating feature is obtained from the first spatiotemporal feature, and a gating feature is obtained from the second spatiotemporal feature. to the second gated feature; time-shift the first gated feature to obtain a forward time-shift feature, and time-shift the second gated feature to obtain a backward time-shift feature; subtract the first gated feature from the first spatiotemporal feature to obtain a first residual feature, and subtract the second gated feature from the second spatiotemporal feature to obtain a second residual feature; the first residual feature and the forward time-shifted feature are fused to obtain the output of the first spatiotemporal feature extraction module, and the second residual feature and the backward time-shifted feature are fused to obtain the output of the second spatiotemporal feature extraction module; the output of the first spatiotemporal feature extraction module and the output of the second spatiotemporal feature extraction module together constitute the output feature of the spatiotemporal feature extraction module;

[0062] The first gating feature and the second gating feature are obtained in the following manner:

[0063] Y1=tanh(W1*X1)⊙X1;

[0064] Y2=tanh(W2*X2)⊙X2;

[0065] Among them, Y1 and Y2 are the first gated feature and the second gated feature respectively, X1 is the first spatiotemporal feature, X2 is the second spatiotemporal feature, X1 and X2 are segmented based on channels, W1 and W2 are 3*3*3 convolution kernels respectively, * is convolution, and ⊙ is Hadamard product.

[0066] In a possible implementation, the video data of the first area is input into a second feature extraction module for feature extraction to obtain local features, including: defining a vocabulary table related to entities of a set of feature points, and merging them into a text prompt; encoding the text prompt through a pre-trained BERT to obtain vocabulary features; inputting the video data of the first area into a pre-trained visual encoder for processing to obtain image region of interest features; aligning and fusing vocabulary features and region of interest features to obtain point of interest entity features; and extracting features of the point of interest entity features through an attention mechanism to obtain local features.

[0067] For example, the entity-related vocabulary may be shoulders, hands, and so on of a human body.

[0068] It can be seen that the embodiments of the present invention can dynamically adjust the brightness of the information display area based on the coverage of the crowd's field of vision; by extracting information from global features and using supplementary local information to enrich features, a two-dimensional convolutional neural network and a spatiotemporal module are used for global feature extraction to maintain the spatiotemporal context feature extraction relationship while reducing the amount of calculation; when extracting local features, a pre-trained language encoder is combined with a visual encoder to obtain more accurate local features. In summary, the embodiments of the present application can dynamically adjust the brightness of the area based on rapid image data processing, thereby improving the speed of response and achieving a more accurate energy consumption control effect.

[0069] In a possible implementation, the multiple cameras are four cameras, and the four cameras are located on the ceiling; the lines connecting the position points of the four cameras form a rectangle, and the area covered by the rectangle is larger than the first area.

[0070] In one possible implementation, after determining N persons in the first area, before executing the following steps for each of the N persons, the method also includes: determining whether N is greater than a preset number of persons, and if so, detecting the position movement changes and body rotation changes of each of the N persons within a preset time period; calculating the comprehensive movement change degree and the comprehensive rotation change degree based on the position movement changes and body rotation changes of each person; if the comprehensive movement change degree is greater than the preset movement change degree or the comprehensive rotation change degree is greater than the preset rotation change degree, controlling the multiple illumination lamps to increase the brightness of the information display area corresponding to the first area; if the comprehensive movement change degree is less than or equal to the preset movement change degree or the comprehensive rotation change degree is less than or equal to the preset rotation change degree; then executing the following steps for each of the N persons.

[0071] Among them, whether the people are focused on the information display area is determined based on the comprehensive movement change degree and the comprehensive rotation change degree. If the comprehensive movement change degree or the comprehensive rotation change degree is large, it means that the people are not focused on the content of the information display area, and are more likely to be talking or moving, and the subsequent brightness enhancement step will not be executed.

[0072] Among them, the preset movement change degree and the preset rotation change degree can be set based on actual needs.

[0073] It can be seen that in the embodiment of the present application, the attention of the personnel to the information display area can be judged based on the status of the personnel, and the subsequent steps will not be executed when the attention is low, thereby reducing unnecessary waste of electricity.

[0074] In a possible implementation, the detecting the position movement changes and body rotation changes of each of the N persons within a preset time period; calculating the comprehensive movement change degree and the comprehensive rotation change degree based on the position movement changes and body rotation changes of each person, includes: determining the position movement distance of each person based on the starting point and the end point of the preset time period, determining a first proportion of the position movement distances of the N persons that are less than a preset distance, and determining the comprehensive movement change degree according to the first proportion; recording the body posture of each person at the starting point of the preset time period, and calculating the rotation duration of each person's body rotation angle greater than the preset rotation angle within the preset time period based on the body posture of each person at the starting point; determining a second proportion of the rotation duration greater than the preset rotation duration among the N persons, and determining the comprehensive rotation change degree according to the second proportion.

[0075] Specifically, the larger the first ratio is, the smaller the overall movement of the N persons is; the larger the second ratio is, the longer the duration of the person's body conversion visual coverage within a certain period of time is.

[0076] It should be noted that the preset duration, preset distance, preset rotation angle, and preset rotation duration can all be based on actual needs; for example, the preset duration can be 5 minutes, the preset distance can be 2 meters, the preset rotation angle can be 30 degrees, the preset rotation duration can be 2 minutes, and the preset movement change degree can be 80%. When the proportion of N personnel with a movement distance of less than 2 meters is less than 80%, it is considered that the comprehensive movement change degree is greater than the preset movement change degree, otherwise, the comprehensive movement change degree is less than or equal to the preset movement change degree; when the proportion of N personnel with a rotation angle greater than 30 degrees and a holding time of more than 2 minutes is greater than 80%, it indicates that the comprehensive rotation change degree is greater than the preset rotation change degree; otherwise, the comprehensive rotation change degree is less than or equal to the preset rotation change degree.

[0077] Based on the description of the generation method embodiment described in the above text, the embodiment of the present invention also discloses an intelligent energy consumption control system based on dynamic distribution of building personnel.

[0078] The intelligent energy consumption control system based on dynamic distribution of building personnel is deployed in a target building, and the target building is equipped with multiple cameras and multiple illumination lamps, and the illumination angles of the multiple illumination lamps are adjustable. The intelligent energy consumption control system based on dynamic distribution of building personnel is used to perform the following method steps:

[0079] Acquire video data of the first area through the multiple cameras;

[0080] Detect whether there is a person in the first area, and if so, determine N persons in the first area, where N is a positive integer;

[0081] For each of the N persons, perform the following steps:

[0082] Determine N visual coverage areas corresponding to the N persons;

[0083] Determine M visual coverage areas that intersect with each other among the N visual coverage areas corresponding to the N persons;

[0084] Determine whether M is greater than or equal to a preset threshold;

[0085] If M is greater than or equal to a preset threshold, determining the union of the M visual coverage areas;

[0086] Determine the information display area in the union; control the illumination angle and illumination brightness of the multiple illumination lamps to increase the brightness of the information display area;

[0087] If M is less than a preset threshold, the M persons corresponding to the M visual coverage areas are determined, and the real-time gaze points of the M persons are tracked by the multiple cameras to determine whether the gaze points fall in the information display area; if so, the illumination angles and illumination brightness of the multiple illumination lamps are controlled to increase the brightness of the information display area;

[0088] For the first visual coverage range among the N visual coverage ranges that does not have an intersection, the real-time gaze point of the person corresponding to the first visual coverage range is tracked by the multiple cameras to determine whether the gaze point falls in the information display area; if so, the illumination angle and illumination brightness of the multiple illumination lamps are controlled to increase the brightness of the information display area.

[0089] The embodiment of the present invention also provides a computer storage medium (Memory), which is a memory device in an electronic device for storing programs and data. It is understandable that the computer storage medium here can include both built-in storage media in the electronic device and, of course, extended storage media supported by the electronic device. The computer storage medium provides a storage space, which stores the operating system of the electronic device. In addition, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that the computer storage medium here can be a high-speed RAM memory, or a non-volatile memory (non-volatile memory), such as at least one disk storage; optionally, it can also be at least one computer storage medium located away from the aforementioned processor.

[0090] Figure 3 3 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The device includes: at least one processor 301, such as a central processing unit (CPU), at least one memory 302, and at least one bus 303.

[0091] The memory 302 may store program instructions, and the processor 301 may be used to call program instructions to execute an intelligent energy consumption control system and method based on dynamic distribution of building personnel.

[0092] A person of ordinary skill in the art can understand that all or part of the steps in various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electrically-erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM), a solid state disk (SSD) or other optical disk storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0093] It should be noted that all steps of the embodiments of the present application are performed under legal and compliant conditions, that is, all steps of the embodiments of the present application are performed with authorization.

[0094] The above disclosure is only the preferred embodiment of the present invention, which certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.

Claims

1. An intelligent energy consumption control method based on dynamic distribution of building personnel, characterized in that: The method is applied to an intelligent energy consumption control system based on dynamic distribution of building personnel. The intelligent energy consumption control system based on dynamic distribution of building personnel is deployed in a target building. The target building is equipped with multiple cameras and multiple irradiation lamps. The irradiation angles of the multiple irradiation lamps are adjustable. The method includes: Acquire video data of the first area through the multiple cameras; Detect whether there is a person in the first area, and if so, determine N persons in the first area, where N is a positive integer; For each of the N persons, perform the following steps: Determine N visual coverage areas corresponding to the N persons; Determine M visual coverage areas that intersect with each other among the N visual coverage areas corresponding to the N persons, Determine whether M is greater than or equal to a preset threshold; If M is greater than or equal to a preset threshold, determining the union of the M visual coverage areas; Determine the information display area in the union; control the illumination angle and illumination brightness of the multiple illumination lamps to increase the brightness of the information display area; If M is less than a preset threshold, the M persons corresponding to the M visual coverage areas are determined, and the real-time gaze points of the M persons are tracked by the multiple cameras to determine whether the gaze points fall in the information display area; if so, the illumination angles and illumination brightness of the multiple illumination lamps are controlled to increase the brightness of the information display area; For a first visual coverage range that does not have an intersection among the N visual coverage ranges, the multiple cameras are used to track the real-time gaze point of the person corresponding to the first visual coverage range, and it is determined whether the gaze point falls on the information display area; if so, the illumination angle and illumination brightness of the multiple illumination lamps are controlled to increase the brightness of the information display area; The determining of N visual coverage areas corresponding to the N persons includes: Perform feature point detection on each person, obtain a group of feature points on both sides of the shoulders of each person, the group of feature points are at the same level, and construct a first plane based on the group of feature points, the first plane is perpendicular to the horizontal plane, and the group of feature points are located in the first plane; Detecting facial feature points of each person; Determine a first space where the facial feature points of each person are located compared to the first plane, and construct a vertical line perpendicular to the first plane in the first space with the center point of the group of feature points as a starting point; A first ray and a second ray are constructed with the center point as a starting point; the first ray, the second ray and the vertical line are in the same plane and are located on the same side of the first plane; and the angles between the first ray and the second ray and the vertical line are both preset angles; Constructing a second plane and a third plane based on the first ray and the second ray respectively; The second plane and the third plane are perpendicular to the horizontal plane; determining a first intersection line and a second intersection line of the second plane and the third plane with the object in the first space; The area between the first intersection line and the second intersection line is the visual coverage range of each person.

2. The method according to claim 1, characterized in that The step of performing feature point detection on each person to obtain a set of feature points on both sides of the shoulders of each person includes: After determining the N persons in the first area, inputting the video data of the first area into a first feature extraction module for feature extraction to obtain a global feature; Inputting the video data of the first area into a second feature extraction module to extract features and obtain local features; The global feature is concatenated with the local feature, and the concatenated features are sequentially input into the self-attention module and the average pooling layer to obtain feature points; Wherein, the first feature extraction module includes a two-dimensional convolutional neural network, and the two-dimensional convolutional neural network includes a spatiotemporal feature extraction module; Inputting the video data of the first area into a first feature extraction module to extract features to obtain global features, including: The two-dimensional convolutional neural network includes a first branch, a second branch and a third branch, wherein the first branch includes a convolution layer with a convolution kernel of 1*1, a convolution layer with a convolution kernel of 3*3, and a convolution layer with a convolution kernel of 3*3 in sequence, and the second branch includes a convolution layer with a convolution kernel of 1*1 and a convolution layer with a convolution kernel of 3*3 in sequence; the third branch includes a convolution layer with a convolution kernel of 1*1 and the spatiotemporal feature extraction module in sequence; the outputs of the first branch, the second branch and the third branch are fused to obtain a global feature; the output of the convolution layer with a convolution kernel of 1*1 of the third branch is the second feature; Inputting the second feature into a spatiotemporal feature extraction module to obtain an output feature; The step of inputting the second feature into a spatiotemporal feature extraction module to obtain an output feature includes: performing a two-dimensional convolution on the second feature to obtain a spatiotemporal feature (C, T, W, H), where C is the number of channels, W and H are spatial dimensions, and T is a time dimension, and dividing the spatiotemporal feature into a first spatiotemporal feature and a second spatiotemporal feature based on the channel; Obtaining a first gating feature from the first spatiotemporal feature, and obtaining a second gating feature from the second spatiotemporal feature; Performing time shift on the first gated feature to obtain a forward time shift feature, and performing time shift on the second gated feature to obtain a backward time shift feature; The first residual feature is obtained by subtracting the first gating feature from the first spatiotemporal feature, and the second residual feature is obtained by subtracting the second gating feature from the second spatiotemporal feature; The first residual feature and the forward time-shift feature are fused to obtain the output of the first spatiotemporal feature extraction module, and the second residual feature and the backward time-shift feature are fused to obtain the output of the second spatiotemporal feature extraction module; The output of the first spatiotemporal feature extraction module and the output of the second spatiotemporal feature extraction module together constitute the output features of the spatiotemporal feature extraction module.

3. The method according to claim 2, characterized in that The step of inputting the video data of the first area into a second feature extraction module for feature extraction to obtain local features includes: Define a set of entity-related vocabularies of feature points and incorporate them into the text prompt; Encoding the text prompts through the pre-trained BERT to obtain vocabulary features; Inputting the video data of the first area into a pre-trained visual encoder for processing to obtain features of the image region of interest; Align and fuse vocabulary features and region of interest features to obtain point of interest entity features; The entity features of the interest point are extracted through an attention mechanism to obtain the local features.

4. The method according to claim 1, characterized in that The multiple cameras are 4 cameras, and the 4 cameras are located on the ceiling; the lines connecting the position points of the 4 cameras form a rectangle, and the area covered by the rectangle is larger than the first area.

5. The method according to claim 4, characterized in that After determining N persons in the first area, and before performing the following steps for each of the N persons, the method further includes: Determine whether N is greater than a preset number of persons, and if so, detect the position movement change and body rotation change of each of the N persons within a preset time period; calculate the comprehensive movement change degree and the comprehensive rotation change degree based on the position movement change and body rotation change of each person; If the comprehensive movement change degree is greater than the preset movement change degree or the comprehensive rotation change degree is greater than the preset rotation change degree, controlling the multiple illumination lamps to increase the brightness of the information display area corresponding to the first area; If the comprehensive movement change degree is less than or equal to the preset movement change degree or the comprehensive rotation change degree is less than or equal to the preset rotation change degree; then execute the following steps for each of the N persons.

6. The method according to claim 5, characterized in that The detecting of the position movement change and body rotation change of each of the N persons within a preset time period; Calculating the comprehensive movement change degree and the comprehensive rotation change degree based on the position movement change and body rotation change of each person includes: Determine the position movement distance of each person based on the starting point and the end point of each person in the preset time period, determine a first proportion of the position movement distances of the N people that are less than the preset distance, and determine the comprehensive movement change degree according to the first proportion; The body posture of each person at the starting point of the preset duration is recorded, and based on the body posture of each person at the starting point, the rotation duration of each person's body rotation angle greater than the preset rotation angle within the preset duration is calculated; a second proportion of the N persons whose rotation duration is greater than the preset duration is determined, and the comprehensive rotation change degree is determined according to the second proportion.

7. An intelligent energy consumption control system based on dynamic distribution of building personnel, characterized in that: The intelligent energy consumption control system based on dynamic distribution of building personnel is deployed in a target building, and the target building is equipped with multiple cameras and multiple illumination lamps, and the illumination angles of the multiple illumination lamps are adjustable. The intelligent energy consumption control system based on dynamic distribution of building personnel is used to perform the following method steps: Acquire video data of the first area through the multiple cameras; Detect whether there is a person in the first area, and if so, determine N persons in the first area, where N is a positive integer; For each of the N persons, perform the following steps: Determine N visual coverage areas corresponding to the N persons; Determine M visual coverage areas that intersect with each other among the N visual coverage areas corresponding to the N persons; Determine whether M is greater than or equal to a preset threshold; If M is greater than or equal to a preset threshold, determining the union of the M visual coverage areas; Determine the information display area in the union; control the illumination angle and illumination brightness of the multiple illumination lamps to increase the brightness of the information display area; If M is less than a preset threshold, the M persons corresponding to the M visual coverage areas are determined, and the real-time gaze points of the M persons are tracked by the multiple cameras to determine whether the gaze points fall in the information display area; if so, the illumination angles and illumination brightness of the multiple illumination lamps are controlled to increase the brightness of the information display area; For a first visual coverage range that has no intersection among the N visual coverage ranges, tracking the real-time gaze point of a person corresponding to the first visual coverage range through the multiple cameras, and determining whether the gaze point falls within the information display area; If yes, controlling the illumination angles and illumination brightness of the multiple illumination lamps to increase the brightness of the information display area; The determining of N visual coverage areas corresponding to the N persons includes: Perform feature point detection on each person, obtain a group of feature points on both sides of the shoulders of each person, the group of feature points are at the same level, and construct a first plane based on the group of feature points, the first plane is perpendicular to the horizontal plane, and the group of feature points are located in the first plane; Detecting facial feature points of each person; Determine a first space where the facial feature points of each person are located compared to the first plane, and construct a vertical line perpendicular to the first plane in the first space with the center point of the group of feature points as a starting point; A first ray and a second ray are constructed with the center point as a starting point; the first ray, the second ray and the vertical line are in the same plane and are located on the same side of the first plane; and the angles between the first ray and the second ray and the vertical line are both preset angles; Constructing a second plane and a third plane based on the first ray and the second ray respectively; The second plane and the third plane are perpendicular to the horizontal plane; determining a first intersection line and a second intersection line of the second plane and the third plane with the object in the first space; The area between the first intersection line and the second intersection line is the visual coverage range of each person.

8. An electronic device, characterized in that: include: a processor adapted to implement one or more instructions; as well as, A computer storage medium storing one or more instructions, wherein the one or more instructions are suitable for being loaded by the processor and executed by the intelligent energy consumption control method based on the dynamic distribution of building personnel as described in any one of claims 1 to 6.

9. A computer storage medium, characterized in that The computer storage medium stores one or more instructions, and the one or more instructions are suitable for being loaded by a processor and executing the intelligent energy consumption control method based on dynamic distribution of building personnel as described in any one of claims 1-6.

Images