A multifunctional linked intelligent street light control method, system and storage medium
Through the multi-functional intelligent street light control system, the use of visible light, infrared and audio data, combined with smart cloud platform and edge controller, dynamically adjusts the status of street lights, solving the problem of low intelligence level of existing street light control systems and achieving more accurate energy consumption management and safety monitoring.
Patent Information
- Application Number
- CN202211634635.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-12-19
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2042-12-19
AI Technical Summary
The existing street light control system has low intelligence level and cannot effectively respond to emergencies, resulting in the inability to accurately control the status of street lights, affecting energy consumption management and safety monitoring.
Adopt a multi-functional linkage smart street light control system to collect data through visible light collectors, infrared sensors and radios, combine smart cloud platforms and edge controllers to construct multi-level data interfaces and environmental processing models, generate street light control strategies, and dynamic adjustments The brightness and direction of the projected street lights are adjusted in real time according to the environmental complexity and attention position.
It improves the intelligence level of street light control, can respond to emergencies more accurately, improves energy consumption management and safety monitoring effects, provides clear image data, and enhances the monitoring ability of illegal behavior.
Smart Images

Figure CN115835457B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of artificial intelligence, and in particular to a multifunctional linked smart street lamp control method, system and storage medium. Background Art
[0002] Currently, street lamps are mostly controlled by induction, that is, the brightness is increased when there are people, and the brightness is reduced when there are fewer people and cars, thereby reducing energy consumption. However, this control method from the perspective of energy saving brings a poor experience, cannot cope with emergencies, and has a low level of intelligence.
[0003] Therefore, it is necessary to provide a street lamp control system with a high level of intelligence. Summary of the invention
[0004] The embodiments of this specification provide a multifunctional linked smart street lamp control method, system and storage medium for accurately controlling the state of street lamps and improving the intelligence level of street lamps.
[0005] The embodiment of this specification provides a multifunctional linked smart street light control system, including:
[0006] A multifunctional street lamp, having a visible light collector, an infrared sensor, a receiver, a fixed projection street lamp, and an adjustable projection street lamp, for collecting visible light images, infrared images, and ambient audio, transmitting them to an edge controller, and controlling the working states of the fixed projection street lamp and the adjustable projection street lamp by control instructions sent by the edge controller;
[0007] A smart cloud platform, used to construct a data interface with multiple levels, construct a multifunctional environment processing model according to the multiple levels of data interfaces, deploy the multifunctional environment processing model on the edge controller and the smart cloud platform, use the multifunctional environment processing model to process the visible light image, infrared image, ambient audio or compressed features thereof collected by the multifunctional street lamp, generate a street lamp control strategy, and execute it through the edge controller to control the working status of the fixed projection street lamp and the adjustable projection street lamp;
[0008] The edge controller is used to obtain visible light images, infrared images, and ambient audio collected by multiple multifunctional street lamps, input them into the multifunctional environment processing model, and calculate the environmental complexity according to the visible light images, infrared images, and ambient audio, and select the corresponding level of data interface according to the environmental complexity to output the compressed features to the smart cloud platform, and the smaller the level is, the greater the environmental complexity is;
[0009] The smart cloud platform is also used to: continue to process the compressed features from the corresponding level: combine the compressed features of multiple multifunctional street lamps, mine the attention position according to the arm direction, the direction of the target sound source, and the facial orientation after turning the head when the head turning speed is greater than a threshold, and generate a street lamp control strategy in combination with the attention position and the complexity of the environment to adjust the working status of the fixed projection street lamp and the adjustable projection street lamp.
[0010] Optionally, the combined compression features of multiple multifunctional street lamps are used to mine the attention position according to the arm direction, the direction of the target sound source, and the face direction after turning the head when the head turning speed is greater than a threshold, including:
[0011] A three-dimensional virtual space is constructed according to the relative positions of each street lamp, and visible light images and infrared images collected by multiple multifunctional street lamps are placed in the three-dimensional virtual space. The first intersection is calculated according to the arm pointing and the facial orientation after turning the head when the head turning speed is greater than a threshold. The ambient audio collected by each multifunctional street lamp is separated into components according to the frequency domain distribution characteristics of different component sounds. The ambient audio separation results of different multifunctional street lamps are classified. The second intersection is determined according to the loudness difference and time difference of the audio of the target category component collected under at least three street lamps, and the attention position is determined between the first intersection and the second intersection.
[0012] Optionally, the environmental complexity is calculated according to the visible light image, the infrared image, and the environmental audio, and a data interface of a corresponding level is selected according to the environmental complexity to output compression features to the smart cloud platform, and a smaller level is selected as the environmental complexity increases, and further includes:
[0013] Calculate the time complexity of the current computing task of the multifunctional environmental processing model, determine the future moment based on the time complexity, predict the environmental complexity at the future moment, and move forward or backward the data interface level of the current computing task of the multifunctional environmental processing model in real time according to the predicted future complexity.
[0014] Optionally, the generating of a streetlight control strategy in combination with the attention position and the environmental complexity to adjust the working states of the fixed projection streetlight and the adjustable projection streetlight includes:
[0015] The projection direction of the adjustable projection streetlight is adjusted according to the attention position, the brightness of the fixed projection streetlight and the adjustable projection streetlight is adjusted according to the complexity of the environment, and this strategy is sent to the edge controller for execution.
[0016] Optionally, the environmental complexity s=m / l nv, where m is the number of moving objects, v is the average moving speed, and the temporary brightness of the fixed projection streetlight and the adjustable projection streetlight L=s 1 *L 0 / s,L0 is the reference brightness, s 1 is the reference environment complexity.
[0017] Optionally, each block node under the smart cloud platform stores a street light control strategy respectively;
[0018] When the authenticity of the visible light video stream to be verified as evidence needs to be verified later, each block node will vote by consensus: determine the projection direction timing and brightness timing according to the stored street light control strategy, extract the projection direction features and brightness features in the visible light video stream to be verified, and match them with the projection direction features and brightness features in the stored street light control strategy. If the similarity is less than the threshold, the visible light video stream to be verified is determined to be a forged video.
[0019] Optionally, each block node is further used for:
[0020] The correspondence between moving objects in visible light images and infrared images collected by adjacent street lamps is determined according to the three-dimensional virtual space, and whether the visible light images and infrared images collected by adjacent street lamps are coherent is determined according to the correspondence between moving objects in the visible light images and infrared images collected by adjacent street lamps.
[0021] The embodiment of this specification provides a smart street lamp control method, including:
[0022] Visible light images, infrared images, and ambient audio are collected by visible light collectors, infrared sensors, and microphones in the multifunctional street lamps, and transmitted to the edge controllers. The working states of the fixed projection street lamps and the adjustable projection street lamps are controlled by control instructions sent by the edge controllers. The multifunctional street lamps include fixed projection street lamps and adjustable projection street lamps.
[0023] Construct a data interface with multiple levels in the smart cloud platform, construct a multifunctional environment processing model according to the data interface of the multiple levels, deploy the multifunctional environment processing model on the edge controller and the smart cloud platform, use the multifunctional environment processing model to process the visible light image, infrared image, ambient audio or compressed features thereof collected by the multifunctional street lamp, generate a street lamp control strategy, and execute it through the edge controller to control the working status of the fixed projection street lamp and the adjustable projection street lamp;
[0024] Visible light images, infrared images, and ambient audio collected by multiple multifunctional street lamps are obtained through the edge controller, and input into the multifunctional environment processing model, and the environment complexity is calculated according to the visible light images, infrared images, and ambient audio. According to the environment complexity, the corresponding level of data interface is selected to output the compressed features to the smart cloud platform, and the smaller the level is, the greater the environment complexity is;
[0025] The smart cloud platform continues to process the compressed features from the corresponding level: combining the compressed features of multiple multifunctional street lamps, mining the attention position according to the arm direction, the direction of the target sound source, and the facial orientation after turning the head when the head turning speed is greater than the threshold, and generating a street lamp control strategy based on the attention position and the complexity of the environment to adjust the working status of the fixed projection street lamp and the adjustable projection street lamp.
[0026] The embodiment of the present specification also provides an electronic device, wherein the electronic device includes:
[0027] processor; and,
[0028] A memory storing computer executable instructions, which when executed cause the processor to perform any of the above methods.
[0029] An embodiment of the present specification also provides a computer-readable storage medium, wherein the computer-readable storage medium stores one or more programs, and when the one or more programs are executed by a processor, any of the above methods is implemented.
[0030] The various technical solutions provided in the embodiments of this specification construct a data interface with multiple levels in the smart cloud platform, construct a model, and redundantly deploy it on the edge controller and the smart cloud platform. The edge controller obtains the visible light image, infrared image, and ambient audio collected by the street lamp, and inputs them into the model. The environmental complexity is calculated according to the visible light image, infrared image, and ambient audio. The data interface of the corresponding level is selected according to the environmental complexity to output the compressed features to the smart cloud platform. The larger the environmental complexity, the smaller the level is selected. The smart cloud platform continues to perform feature processing on the compressed features from the corresponding level: combining the compressed features of multiple street lamps, mining the attention position according to the arm pointing, the direction of the target sound source, and the facial orientation when the head turning speed is greater than the threshold, and generating a street lamp control strategy based on the attention position and environmental complexity to adjust the working state of fixed and adjustable projection street lamps. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0032] Figure 1 A schematic diagram of the structure of a multifunctional linked smart street light control system provided in the embodiments of this specification;
[0033] Figure 2 A schematic diagram of the principle of a multifunctional linked intelligent street light control method provided in the embodiments of this specification;
[0034] Figure 3A schematic diagram of the structure of an electronic device provided in an embodiment of this specification;
[0035] Figure 4 A schematic diagram of a computer-readable medium provided for an embodiment of this specification. DETAILED DESCRIPTION
[0036] Exemplary embodiments of the present invention will now be described more fully with reference to the accompanying drawings. However, exemplary embodiments can be implemented in a variety of forms, and should not be construed as limiting the present invention to the embodiments set forth herein. On the contrary, providing these exemplary embodiments enables the present invention to be more comprehensive and complete, and is more convenient for fully conveying the inventive concept to those skilled in the art. The same reference numerals in the figures represent the same or similar elements, components or parts, and thus their repeated description will be omitted.
[0037] Under the premise of being consistent with the technical concept of the present invention, the features, structures, characteristics or other details described in a specific embodiment do not exclude that they can be combined in one or more other embodiments in a suitable manner.
[0038] In the description of specific embodiments, the features, structures, characteristics or other details described in the present invention are intended to enable those skilled in the art to fully understand the embodiments. However, it does not exclude that those skilled in the art can practice the technical solutions of the present invention without one or more of the specific features, structures, characteristics or other details.
[0039] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.
[0040] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.
[0041] The term "and / or" or "and / or" includes all combinations of any one or more of the associated listed items.
[0042] Figure 1 A schematic diagram of a multifunctional linked smart street light control system provided in an embodiment of this specification, the system includes:
[0043] The multifunctional street lamp 103 has a visible light collector 1031, an infrared sensor 1032, a receiver 1033, a fixed projection street lamp 1034, and an adjustable projection street lamp 1035, and is used to collect visible light images, infrared images, and ambient audio, transmit them to the edge controller, and control the working states of the fixed projection street lamp and the adjustable projection street lamp by control instructions sent by the edge controller;
[0044] The smart cloud platform 101 is used to construct a data interface with multiple levels, construct a multifunctional environment processing model according to the data interface of the multiple levels, deploy the multifunctional environment processing model on the edge controller and the smart cloud platform, use the multifunctional environment processing model to process the visible light image, infrared image, environmental audio or its compressed features collected by the multifunctional street lamp, generate a street lamp control strategy, and execute it through the edge controller to control the working state of the fixed projection street lamp and the adjustable projection street lamp. The platform may also have a block node 1011;
[0045] The edge controller 102 is used to obtain visible light images, infrared images, and ambient audio collected by multiple multifunctional street lamps, input them into the multifunctional environment processing model, and calculate the environmental complexity according to the visible light images, infrared images, and ambient audio, and select the corresponding level of data interface according to the environmental complexity to output the compressed features to the smart cloud platform, and the smaller the level is, the greater the environmental complexity is;
[0046] The smart cloud platform is also used to: continue to process the compressed features from the corresponding level: combine the compressed features of multiple multifunctional street lamps, mine the attention position according to the arm direction, the direction of the target sound source, and the facial orientation after turning the head when the head turning speed is greater than a threshold, and generate a street lamp control strategy in combination with the attention position and the complexity of the environment to adjust the working status of the fixed projection street lamp and the adjustable projection street lamp.
[0047] The system constructs a model by constructing a data interface with multiple levels in the smart cloud platform, and redundantly deploys it on the edge controller and the smart cloud platform. The edge controller obtains the visible light image, infrared image, and ambient audio collected by the street lamp, and inputs them into the model. The environmental complexity is calculated based on the visible light image, infrared image, and ambient audio. The data interface of the corresponding level is selected according to the environmental complexity to output the compressed features to the smart cloud platform. The larger the environmental complexity, the smaller the level is selected. The smart cloud platform continues to process the compressed features from the corresponding level: combining the compressed features of multiple street lamps, mining the attention position according to the arm pointing, the direction of the target sound source, and the facial orientation when the head turning speed is greater than the threshold, and generating a street lamp control strategy based on the attention position and environmental complexity to adjust the working state of fixed and adjustable projection street lamps.
[0048] Considering that many criminals tend to commit crimes on less crowded or darker roads, and they often look for shadow areas of street lights to commit crimes, it is often difficult to obtain clear images when solving cases.
[0049] Considering that the attention position of pedestrians can usually reflect the location of criminals, this solution can obtain a clear image by mining the attention position and then adjusting the working status of the fixed projection street lamp and the adjustable projection street lamp, while collecting images.
[0050] Specifically, by mining the attention position based on the arm direction, the direction of the target sound source, and the facial orientation after turning the head when the head turning speed is greater than a threshold, the attention position can be accurately mined.
[0051] Radars can be installed under street lights to identify the direction of the target sound source, or the audio collected by multiple street lights can be combined to calculate the direction of the target sound source.
[0052] Taking into account the limitations of computing power and data congestion, edge controllers and smart cloud platforms can be used to dynamically adjust computing resources and adjust computing entities in real time.
[0053] Specifically, by constructing a multifunctional environmental processing model, the multifunctional environmental processing model is a neural network structure, and each layer has an input data interface and an output data interface. In this way, as long as there is an intermediate compression feature, the calculation can be started from the corresponding level. The multifunctional environmental processing model is redundantly deployed on the edge controller and the smart cloud platform to achieve joint processing, that is, the previous processing steps are implemented on the edge controller, and the subsequent steps are implemented on the smart cloud platform. The step that needs to be completed by the edge controller is adjusted in real time according to the complexity of the environment, and the compression features are output to the smart cloud platform using the data interface of the corresponding level of this step to complete the subsequent steps.
[0054] Among them, compressed features refer to features with smaller data volume obtained through feature extraction and recognition.
[0055] The feature extraction and recognition here can be performed by a processing layer obtained through machine learning training.
[0056] Among them, by identifying the head turning speed, it is determined whether the head turning speed is greater than a threshold, and then the facial orientation after turning the head at this time is obtained. Since the head turning speed can often reflect the degree of urgency at the scene, if the head turning speed is greater than the threshold, it means that an emergency has occurred at the scene, thereby avoiding frequent projection of street lights and improving the accuracy of directional projection of street lights.
[0057] Among them, generating a street light control strategy based on the attention position and the environmental complexity may refer to:
[0058] The temporary brightness of the street lamp and the duration of the temporary brightness are calculated according to the complexity of the environment, and the projection direction of the adjustable projection street lamp is determined according to the attention position.
[0059] The smaller the environmental complexity, the more it can characterize the flow of people and vehicles, as well as the speed. The faster the flow of people, the more complex the environment, and the more difficult it is to capture the key scenes, so the brightness needs to be increased. The fewer the flow of people, the smaller the environmental complexity, which means the environment is more dangerous, so the brightness also needs to be increased. That is, the smaller the environmental complexity, the greater the temporary brightness.
[0060] In the embodiment of this specification, the environment complexity s=m / l nv, where m is the number of moving objects, v is the average moving speed, and the temporary brightness of the fixed projection street lamp and the adjustable projection street lamp L=s 1 *L 0 / s,L 0 is the reference brightness, s 1 is the reference environment complexity.
[0061] The multifunctional environment processing model can identify the positional relationship of multifunctional street lamps, and determine the positions of moving objects in visible light images, infrared images, and the positions of sound sources in ambient audio based on the positional relationship as compressed features.
[0062] The multifunctional environment processing model can also be supplemented with a reinforcement learning module to automatically generate and screen street light control strategies that are suitable for the environment based on feature extraction and recognition results.
[0063] In the embodiment of this specification, the compression features of the combined multiple multifunctional street lamps are used to mine the attention position according to the arm direction, the direction of the target sound source, and the face direction after turning the head when the head turning speed is greater than the threshold, including:
[0064] A three-dimensional virtual space is constructed according to the relative positions of each street lamp, and visible light images and infrared images collected by multiple multifunctional street lamps are placed in the three-dimensional virtual space. The first intersection is calculated according to the arm pointing and the facial orientation after turning the head when the head turning speed is greater than a threshold. The ambient audio collected by each multifunctional street lamp is separated into components according to the frequency domain distribution characteristics of different component sounds. The ambient audio separation results of different multifunctional street lamps are classified. The second intersection is determined according to the loudness difference and time difference of the audio of the target category component collected under at least three street lamps, and the attention position is determined between the first intersection and the second intersection.
[0065] The three-dimensional virtual space can have the coordinates of each street lamp and moving objects.
[0066] The target category components may be screaming, explosions, dog barking, and other sound components that indicate danger.
[0067] The facial orientation after turning the head may be that of a police dog or that of a person.
[0068] Since the propagation of sound is affected by distance, the distance from the sound source to each street lamp can be calculated based on the propagation speed of sound and the loss rate of sound during propagation. Then the position of the sound source can be calculated based on the relative positions of the three street lamps and the three distances.
[0069] In the embodiment of the present specification, the environmental complexity is calculated according to the visible light image, the infrared image, and the environmental audio, and the data interface of the corresponding level is selected according to the environmental complexity to output the compressed features to the smart cloud platform, and the smaller the level is, the greater the environmental complexity is, the further including:
[0070] Calculate the time complexity of the current computing task of the multifunctional environmental processing model, determine the future moment based on the time complexity, predict the environmental complexity at the future moment, and move forward or backward the data interface level of the current computing task of the multifunctional environmental processing model in real time according to the predicted future complexity.
[0071] In the application scenario, through monitoring, we can see that, for example, for a model with 10 processing layers, corresponding to 10 processing steps, at a certain moment, the edge controller uses the first four steps for processing and transmits the processing results to the platform. The platform calls the next six processing layers to continue processing. Subsequently, the complexity of the environment increases. Therefore, the edge controller only performs simple processing and leaves the remaining large amount of computing to the platform. Through real-time adjustment, the computing power can be well balanced and adapted to the environment.
[0072] In the embodiment of this specification, the generation of a street light control strategy in combination with the attention position and the environmental complexity to adjust the working states of the fixed projection street light and the adjustable projection street light includes:
[0073] The projection direction of the adjustable projection streetlight is adjusted according to the attention position, the brightness of the fixed projection streetlight and the adjustable projection streetlight is adjusted according to the complexity of the environment, and this strategy is sent to the edge controller for execution.
[0074] In the embodiment of this specification, each block node under the smart cloud platform stores the street light control strategy respectively;
[0075] When the authenticity of the visible light video stream to be verified as evidence needs to be verified later, each block node will vote by consensus: determine the projection direction timing and brightness timing according to the stored street light control strategy, extract the projection direction features and brightness features in the visible light video stream to be verified, and match them with the projection direction features and brightness features in the stored street light control strategy. If the similarity is less than the threshold, the visible light video stream to be verified is determined to be a forged video.
[0076] In this way, the block consensus voting method can prevent the evidence from being tampered with, and improve the credibility of the visible light video stream used as evidence.
[0077] In the embodiments of the present specification, each street lamp can also use the power consumption timing of the previous time period to hash, use the hash result as the private key to encrypt the power consumption timing of the current time period, and upload the encrypted result to each block node for storage. During verification, each block node hashes the power consumption timing of the previous time period, calculates the expected asymmetric public key, decrypts the power consumption timing of the next time period, calculates the brightness of the time period based on the decryption result, and compares it with the brightness feature in the street lamp control strategy. If they are consistent and the similarity is less than a threshold, the visible light video stream to be verified is determined to be a real video. If they are inconsistent or the similarity is less than a threshold, the visible light video stream to be verified is determined to be a forged video.
[0078] In the embodiment of this specification, each block node is also used for:
[0079] The correspondence between moving objects in visible light images and infrared images collected by adjacent street lamps is determined according to the three-dimensional virtual space, and whether the visible light images and infrared images collected by adjacent street lamps are coherent is determined according to the correspondence between moving objects in the visible light images and infrared images collected by adjacent street lamps.
[0080] By judging whether the visible light image and the infrared image are coherent, it is further determined whether the visible light video stream to be verified is a forged video, thereby improving credibility.
[0081] The position of the moving object in the three-dimensional virtual space can be extracted based on the visible light image and infrared image, and the visible light images and infrared images originating from different street lamps can be matched with the moving objects. When the same moving object is matched, the position time series of the moving object can be constructed, and the similarity of the position time series of the moving objects originating from different street lamps can be calculated. If the similarity exceeds a threshold, it is judged to be coherent. If there is a time difference in the time series, it is judged to be incoherent.
[0082] In the embodiments of this specification, it is also used for:
[0083] If the moving object is a vehicle, the displacement of the moving object at a future time is predicted based on multiple visible light images, infrared images and multiple visible light images, infrared images, and the brightness of the street lamps within the displacement range is increased, wherein the future time is the braking time, specifically, the displacement at the future time in, Q is the process noise covariance, u k is the system control equation, representing the acceleration, A k is the state transition matrix of the configuration, B kis the control input matrix of the configuration, H k is the transformation matrix from state vector to observation vector, Z k is the measured value of the current displacement, R is the process noise covariance matrix, I is the identity matrix, P k is the posterior estimated covariance matrix, the covariance between the true value and the optimal estimate, which becomes the input of the next iteration after update. The a priori estimated covariance matrix, the covariance between the true value and the predicted value, is used to minimize the error by minimizing its trace (minimizing the sum of variances) to obtain the Kalman gain K. k , used to correct the prior estimate k-1 represents the iteration time of the last iteration, and k represents the current iteration time, so as to obtain the displacement estimation value at the future time;
[0084] If the moving object is a vehicle or a pedestrian, the corresponding street lamp is determined according to a preset distance in the moving direction of the pedestrian and in the opposite direction, and its brightness is increased.
[0085] Figure 2 A schematic diagram of the principle of a multifunctional linked smart street light control method provided in an embodiment of this specification, the method may include:
[0086] S101: collect visible light images, infrared images, and ambient audio through the visible light collector, infrared sensor, and microphone in the multifunctional street lamp, transmit them to the edge controller, and control the working states of the fixed projection street lamp and the adjustable projection street lamp according to the control instructions sent by the edge controller, wherein the multifunctional street lamp includes the fixed projection street lamp and the adjustable projection street lamp;
[0087] S102: constructing a data interface with multiple levels in the smart cloud platform, constructing a multifunctional environment processing model according to the data interface of the multiple levels, deploying the multifunctional environment processing model on the edge controller and the smart cloud platform, using the multifunctional environment processing model to process the visible light image, infrared image, ambient audio or compressed features thereof collected by the multifunctional street lamp, generating a street lamp control strategy, and executing it through the edge controller to control the working status of the fixed projection street lamp and the adjustable projection street lamp;
[0088] S103: Obtain visible light images, infrared images, and ambient audio collected by multiple multifunctional street lamps through the edge controller, input them into the multifunctional environment processing model, and calculate the environmental complexity according to the visible light images, infrared images, and ambient audio, and select the corresponding level of data interface according to the environmental complexity to output compressed features to the smart cloud platform, and the smaller the level is, the greater the environmental complexity is;
[0089] S104: The smart cloud platform continues to perform feature processing on the compressed features from the corresponding level: combining the compressed features of multiple multifunctional street lamps, mining the attention position according to the arm direction, the direction of the target sound source, and the facial orientation after turning the head when the head turning speed is greater than the threshold, and generating a street lamp control strategy based on the attention position and the complexity of the environment to adjust the working status of the fixed projection street lamp and the adjustable projection street lamp.
[0090] The method constructs a model by constructing a data interface with multiple levels in the smart cloud platform, and redundantly deploys it on the edge controller and the smart cloud platform. The edge controller obtains the visible light image, infrared image, and ambient audio collected by the street lamp, and inputs them into the model. The environmental complexity is calculated according to the visible light image, infrared image, and ambient audio. The data interface of the corresponding level is selected according to the environmental complexity to output the compressed features to the smart cloud platform. The larger the environmental complexity, the smaller the level is selected. The smart cloud platform continues to process the compressed features from the corresponding level: the compressed features of multiple street lamps are combined, and the attention position is mined according to the arm pointing, the direction of the target sound source, and the facial orientation when the head turning speed is greater than the threshold. The street lamp control strategy is generated in combination with the attention position and the environmental complexity to adjust the working state of the fixed and adjustable projection street lamps.
[0091] Based on the same inventive concept, an embodiment of this specification also provides an electronic device.
[0092] The following describes an electronic device embodiment of the present invention, which can be regarded as a specific physical implementation of the method and device embodiments of the present invention. The details described in the electronic device embodiment of the present invention should be regarded as a supplement to the above method or device embodiments; details not disclosed in the electronic device embodiment of the present invention can be implemented with reference to the above method or device embodiments.
[0093] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this specification. Figure 3 The electronic device 300 according to the embodiment of the present invention is described. Figure 3 The electronic device 300 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.
[0094] like Figure 3 As shown, the electronic device 300 is in the form of a general computing device. The components of the electronic device 300 may include, but are not limited to: at least one processing unit 310, at least one storage unit 320, a bus 330 connecting different system components (including the storage unit 320 and the processing unit 310), a display unit 340, etc.
[0095] The storage unit stores program codes, which can be executed by the processing unit 310, so that the processing unit 310 performs the steps according to various exemplary embodiments of the present invention described in the above processing method section of this specification. For example, the processing unit 310 can perform the following steps: Figure 1 Steps shown.
[0096] The storage unit 320 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 3201 and / or a cache memory unit 3202 , and may further include a read-only memory unit (ROM) 3203 .
[0097] The storage unit 320 may also include a program / utility 3204 having a set (at least one) of program modules 3205, such program modules 3205 including but not limited to: an operating system, one or more application programs, other program modules and program data, each of which or some combination may include the implementation of a network environment.
[0098] Bus 330 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.
[0099] The electronic device 300 may also communicate with one or more external devices 400 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), one or more devices that enable a user to interact with the electronic device 300, and / or any device that enables the electronic device 300 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed through an input / output (I / O) interface 350. Furthermore, the electronic device 300 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 360. The network adapter 360 may communicate with other modules of the electronic device 300 through the bus 330. It should be understood that although Figure 3 Not shown, other hardware and / or software modules may be used in conjunction with the electronic device 300, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.
[0100] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the exemplary embodiments described in the present invention can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation method of the present invention can be embodied in the form of a software product, which can be stored in a computer-readable storage medium (which can be a CD-ROM, USB flash drive, mobile hard disk, etc.) or on a network, including a number of instructions to enable a computing device (which can be a personal computer, server, or network device, etc.) to execute the above method according to the present invention. When the computer program is executed by a data processing device, the computer-readable medium can implement the above method of the present invention, that is: Figure 1 The method shown.
[0101] Figure 4 A schematic diagram of a computer-readable medium provided for an embodiment of this specification.
[0102] accomplish Figure 1 The computer program of the method shown can be stored on one or more computer readable media. The computer readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
[0103] The computer readable storage medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by an instruction execution system, an apparatus, or a device or used in combination with it. The program code contained on the readable storage medium may be transmitted with any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.
[0104] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).
[0105] In summary, the present invention can be implemented in hardware, or in a software module running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that general data processing devices such as microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components in the embodiments of the present invention. The present invention can also be implemented as a device or apparatus program (e.g., a computer program and a computer program product) for executing part or all of the methods described herein. Such a program implementing the present invention can be stored on a computer-readable medium, or can have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.
[0106] The specific embodiments described above further describe the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the present invention is not inherently related to any specific computer, virtual device or electronic device, and various general devices can also implement the present invention. The above description is only a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
[0107] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.
[0108] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims
1. A multifunctional linked intelligent street light control system, characterized in that: include: A multifunctional street lamp, having a visible light collector, an infrared sensor, a receiver, a fixed projection street lamp, and an adjustable projection street lamp, for collecting visible light images, infrared images, and ambient audio, transmitting them to an edge controller, and controlling the working states of the fixed projection street lamp and the adjustable projection street lamp by control instructions sent by the edge controller; A smart cloud platform, used to construct a data interface with multiple levels, construct a multifunctional environment processing model according to the multiple levels of data interfaces, deploy the multifunctional environment processing model on the edge controller and the smart cloud platform, use the multifunctional environment processing model to process the visible light image, infrared image, ambient audio or compressed features thereof collected by the multifunctional street lamp, generate a street lamp control strategy, and execute it through the edge controller to control the working status of the fixed projection street lamp and the adjustable projection street lamp; The edge controller is used to obtain visible light images, infrared images, and ambient audio collected by multiple multifunctional street lamps, input them into the multifunctional environment processing model, and calculate the environmental complexity according to the visible light images, infrared images, and ambient audio, and select the corresponding level of data interface according to the environmental complexity to output the compressed features to the smart cloud platform, and the smaller the level is, the greater the environmental complexity is; The smart cloud platform is also used to: continue to process the compressed features from the corresponding level: combine the compressed features of multiple multifunctional street lamps, mine the attention position according to the arm direction, the direction of the target sound source, and the facial orientation after turning the head when the head turning speed is greater than a threshold, and generate a street lamp control strategy in combination with the attention position and the complexity of the environment to adjust the working status of the fixed projection street lamp and the adjustable projection street lamp.
2. The system according to claim 1, characterized in that The combined compression features of multiple multifunctional street lamps are used to mine the attention position according to the arm direction, the direction of the target sound source, and the face direction after turning the head when the head turning speed is greater than a threshold, including: A three-dimensional virtual space is constructed according to the relative positions of each street lamp, and visible light images and infrared images collected by multiple multifunctional street lamps are placed in the three-dimensional virtual space. The first intersection is calculated according to the arm pointing and the facial orientation after turning the head when the head turning speed is greater than a threshold. The ambient audio collected by each multifunctional street lamp is separated into components according to the frequency domain distribution characteristics of different component sounds. The ambient audio separation results of different multifunctional street lamps are classified. The second intersection is determined according to the loudness difference and time difference of the audio of the target category component collected under at least three street lamps, and the attention position is determined between the first intersection and the second intersection.
3. The system according to claim 1, characterized in that The environmental complexity is calculated according to the visible light image, the infrared image, and the environmental audio, and a data interface of a corresponding level is selected according to the environmental complexity to output compression features to the smart cloud platform, and a smaller level is selected as the environmental complexity increases, and further includes: Calculate the time complexity of the current computing task of the multifunctional environmental processing model, determine the future moment based on the time complexity, predict the environmental complexity at the future moment, and move forward or backward the data interface level of the current computing task of the multifunctional environmental processing model in real time according to the predicted future complexity.
4. The system according to claim 1, characterized in that The generating of the street light control strategy by combining the attention position and the environmental complexity to adjust the working states of the fixed projection street light and the adjustable projection street light includes: The projection direction of the adjustable projection streetlight is adjusted according to the attention position, the brightness of the fixed projection streetlight and the adjustable projection streetlight is adjusted according to the complexity of the environment, and this strategy is sent to the edge controller for execution.
5. The system according to claim 1, characterized in that The environmental complexity s=m / lnv, where m is the number of moving objects, v is the average moving speed, and the temporary brightness of the fixed projection street lamp and the adjustable projection street lamp is L=s1*L0 / s, where L0 is the reference brightness and s1 is the reference environmental complexity.
6. The system according to claim 1, characterized in that Each block node under the smart cloud platform stores the street light control strategy respectively; When the authenticity of the visible light video stream to be verified as evidence needs to be verified later, each block node will vote by consensus: determine the projection direction timing and brightness timing according to the stored street light control strategy, extract the projection direction features and brightness features in the visible light video stream to be verified, and match them with the projection direction features and brightness features in the stored street light control strategy. If the similarity is less than the threshold, the visible light video stream to be verified is determined to be a forged video.
7. The system according to claim 6, characterized in that Each block node is also used for: The correspondence between moving objects in visible light images and infrared images collected by adjacent street lamps is determined according to the three-dimensional virtual space, and whether the visible light images and infrared images collected by adjacent street lamps are coherent is determined according to the correspondence between moving objects in the visible light images and infrared images collected by adjacent street lamps.
8. A smart street light control method executed by the system of claim 1, characterized in that: include: Visible light images, infrared images, and ambient audio are collected by visible light collectors, infrared sensors, and microphones in the multifunctional street lamps, and transmitted to the edge controllers. The working states of the fixed projection street lamps and the adjustable projection street lamps are controlled by control instructions sent by the edge controllers. The multifunctional street lamps include fixed projection street lamps and adjustable projection street lamps. Construct a data interface with multiple levels in the smart cloud platform, construct a multifunctional environment processing model according to the data interface of the multiple levels, deploy the multifunctional environment processing model on the edge controller and the smart cloud platform, use the multifunctional environment processing model to process the visible light image, infrared image, ambient audio or compressed features thereof collected by the multifunctional street lamp, generate a street lamp control strategy, and execute it through the edge controller to control the working status of the fixed projection street lamp and the adjustable projection street lamp; Visible light images, infrared images, and ambient audio collected by multiple multifunctional street lamps are obtained through the edge controller, and input into the multifunctional environment processing model, and the environment complexity is calculated according to the visible light images, infrared images, and ambient audio. According to the environment complexity, the corresponding level of data interface is selected to output the compressed features to the smart cloud platform, and the smaller the level is, the greater the environment complexity is; The smart cloud platform continues to process the compressed features from the corresponding level: combining the compressed features of multiple multifunctional street lamps, mining the attention position according to the arm direction, the direction of the target sound source, and the facial orientation after turning the head when the head turning speed is greater than the threshold, and generating a street lamp control strategy based on the attention position and the complexity of the environment to adjust the working status of the fixed projection street lamp and the adjustable projection street lamp.
9. A computer-readable storage medium, wherein: The computer-readable storage medium stores one or more programs, which, when executed by a processor, implement the method of claim 8.
