Intelligent control method and system for realizing shooting picture based on Internet of Things

Abstract

The invention relates to the technical field of camera shooting picture intelligent control, and discloses an intelligent control method for realizing camera shooting pictures based on the Internet of Things, and the method comprises the steps: collecting the environment parameters of a camera shooting monitoring area through an Internet of Things device, adjusting the parameters of camera shooting system equipment of the camera shooting monitoring area, and obtaining an adjusted camera shooting device; the method comprises the following steps: acquiring point cloud data of a camera monitoring area, performing element modeling on the camera monitoring area to obtain a camera monitoring area model, and performing secondary error measurement to obtain a target environment model; capturing image information of the camera monitoring area in real time to obtain an analysis target, and analyzing the target dynamic trend of the analysis target; the method comprises the following steps of: acquiring a dynamic picture of an analysis target, carrying out multivariate composite mapping on a target environment model to obtain a real model, carrying out conjoint analysis on the analysis target to obtain a conjoint analysis result, and carrying out real-time control on a camera picture of a camera monitoring area based on the conjoint analysis result. According to the invention, the control precision of the shot picture can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent control of camera images, and in particular to an intelligent control method and system for camera images based on the Internet of Things. Background Art

[0002] In today's digital age, the development of intelligent control technology for camera images has brought significant changes and optimizations to many fields, especially in the fields of monitoring, security and multimedia creation. Intelligent control methods for camera images based on advanced algorithms have attracted much attention due to their precision, efficiency and adaptability.

[0003] At present, traditional camera image control methods mainly rely on fixed parameter settings and simple logical judgments. Although these methods can achieve basic image control requirements in some scenarios, they have obvious limitations. Among them, fixed parameter settings cannot adapt to complex and changing environments and shooting objects, resulting in unstable image quality; simple logical judgments are difficult to accurately capture key information and dynamic changes in the picture, and are prone to missing important details. In addition, traditional methods perform poorly in dealing with different lighting conditions, shooting angles, and scene complexity, which may cause the picture to be too dark, too bright, blurred or distorted, resulting in low control accuracy of the camera image. Summary of the invention

[0004] In order to solve the above problems, the present invention provides an intelligent control method for camera images based on the Internet of Things, which can improve the control accuracy of camera images.

[0005] In a first aspect, the present invention provides an intelligent control method for camera images based on the Internet of Things, comprising:

[0006] An Internet of Things device is deployed in a video surveillance area, and after the Internet of Things device is connected to the video surveillance area through a communication connection, the environmental parameters of the video surveillance area are collected by using the Internet of Things device, and the parameters of the camera system equipment in the video surveillance area are adjusted based on the environmental parameters to obtain an adjusted camera device;

[0007] Collecting point cloud data of the camera monitoring area, using the point cloud data to perform element modeling on the camera monitoring area to obtain a camera monitoring area model, and performing a quadratic error measurement on the camera monitoring area model to obtain a target environment model;

[0008] Capturing the image information of the camera monitoring area in real time to obtain the analysis target, collecting the infrared image and the visible light image of the analysis target, and analyzing the target dynamic trend of the analysis target by using the infrared image and the visible light image;

[0009] Based on the dynamic trend of the target, the dynamic picture of the analysis target is collected, and the target environment model is mapped with a multivariate composite map using the target dynamic picture to obtain a real model. The analysis target is jointly analyzed using the real model to obtain a joint analysis result. Based on the joint analysis result, an intelligent control strategy for the camera device in the camera monitoring area is formulated to use the intelligent control strategy to perform real-time control of the camera picture in the camera monitoring area.

[0010] In a possible implementation manner of the first aspect, adjusting the parameters of the camera system equipment in the camera monitoring area based on the environmental parameters to obtain the adjusted camera device includes:

[0011] Determine the light intensity, ambient decibel and ambient humidity of the camera monitoring area according to the environmental parameters;

[0012] Based on the light intensity, the ambient decibel level and the ambient humidity, determining the ambient condition of the camera monitoring area;

[0013] When the environmental condition does not meet the preset environmental condition, the camera system equipment parameters of the camera monitoring area are adjusted to obtain an adjusted camera device.

[0014] In a possible implementation of the first aspect, element modeling of the camera monitoring area is performed using the point cloud data to obtain a camera monitoring area model, including:

[0015] Dividing the point cloud data into three-dimensional voxel grids;

[0016] Identifying a centroid point of the three-dimensional voxel grid;

[0017] Based on the centroid point, the three-dimensional voxel grid is de-densified to obtain a filtered point cloud;

[0018] Performing point cloud stitching on the filtered point cloud to obtain a stitched point cloud;

[0019] The environment model of the camera monitoring area is constructed using the spliced ​​point cloud to obtain a camera monitoring area model.

[0020] In a possible implementation manner of the first aspect, performing point cloud stitching on the filtered point cloud to obtain a stitched point cloud includes:

[0021] Extracting adjacent point clouds from the filtered point cloud;

[0022] Using the adjacent point clouds, a rotation matrix of the filtered point cloud is created, wherein the rotation matrix can be expressed in the following form:

[0023]

[0024] Among them, ω represents the rotation matrix, R represents the cross vector of adjacent point clouds, and Zr x , Z y , Z z Represents the three-dimensional space coordinates of the intersection vector of adjacent point clouds, θ represents the intersection angle of adjacent point clouds, and E represents the unit vector of adjacent point clouds;

[0025] After the ground normal vector angles of the filtered point clouds are rotated to be consistent using the rotation matrix, the filtered point clouds are spliced ​​to obtain a spliced ​​point cloud.

[0026] In a possible implementation manner of the first aspect, performing a quadratic error measurement on the camera monitoring area model to obtain a target environment model includes:

[0027] The monitoring coverage, image clarity, target recognition accuracy and object motion trajectory are used as key features of the camera monitoring area model;

[0028] Calculating a characteristic error of the key characteristic;

[0029] Based on the characteristic error, the comprehensive quadratic error of the camera monitoring area model is calculated using the following formula:

[0030] F 总 =W1E1+W2E2+W3E3+W4E4

[0031] Among them, E 总 represents the comprehensive quadratic error, E1 represents the monitoring coverage error in the feature error, E2 represents the image clarity error in the feature error, E3 represents the target recognition accuracy error in the feature error, E4 represents the object motion trajectory error in the feature error, W1 represents the weight corresponding to the monitoring coverage error in the feature error, W2 represents the weight corresponding to the image clarity error in the feature error, W3 represents the weight corresponding to the target recognition accuracy error in the feature error, and W4 represents the weight corresponding to the object motion trajectory error in the feature error;

[0032] Based on the comprehensive quadratic error, the camera monitoring area model is adjusted to obtain a target environment model.

[0033] In a possible implementation manner of the first aspect, calculating the feature error of the key feature includes:

[0034] The monitoring coverage error in the key feature is calculated using the following formula:

[0035] E1=(A actual -A model ) 2

[0036] Among them, E1 represents the monitoring coverage error, A actual Indicates the area of ​​the actual monitoring area, A model represents the area predicted by the model;

[0037] The image clarity error in the key feature is calculated using the following formula:

[0038] E2=∑ x ∑ y (P actual (x,y)-P model (x,y) 2

[0039] Among them, E2 represents the image clarity error, P actual (x, y) represents the pixel of the key feature corresponding to the coordinate point (x, y) in the original image, P model (x,y) represents the pixel at point (x,y) in the model;

[0040] The target recognition accuracy error in the key feature is calculated using the following formula:

[0041] E3=(N actual -N model ) 2

[0042] Among them, E3 represents the target recognition accuracy error, N actual Indicates the number of correctly identified targets, N model Indicates the correct number of targets predicted by the model;

[0043] The object motion trajectory error in the key feature is calculated using the following formula:

[0044]

[0045] Among them, E4 represents the object motion trajectory error, O represents the object motion coordinate sequence,

[0046] A coordinate sequence representing the motion trajectory of an object in the monitoring area,

[0047] A sequence of coordinates representing the trajectory of an object predicted by the model.

[0048] In a possible implementation manner of the first aspect, analyzing the target dynamic trend of the analysis target by using the infrared image and the visible light image includes:

[0049] Extracting image features of the infrared image and the visible light image to obtain a first feature and a second feature;

[0050] Performing feature fusion on the first feature and the second feature to obtain a fused feature;

[0051] Tracking the analysis target based on the fusion feature to obtain the target action trajectory;

[0052] The target dynamic trend of the analysis target is analyzed based on the target action trajectory.

[0053] In a possible implementation manner of the first aspect, analyzing the target dynamic trend of the analysis target based on the target action trajectory includes:

[0054] Segmenting the target motion trajectory to obtain a segmented motion trajectory;

[0055] Querying the action parameters of the segmented action trajectory;

[0056] Identifying trajectory features of the target action trajectory based on the action parameters;

[0057] Based on the trajectory characteristics, a curve simulation is performed on the target action trajectory to obtain a simulation curve; and based on the simulation curve, the target dynamic trend of the target is analyzed.

[0058] In a possible implementation manner of the first aspect, using the real model to perform a joint analysis on the analysis target to obtain a joint analysis result includes:

[0059] Extracting feature data from the dynamic picture of the analysis target;

[0060] Performing data integration on the feature data to obtain integrated data;

[0061] Initializing the real model using the integrated data to obtain an initialized model;

[0062] Use the initialization model to perform motion simulation on the analysis target, obtain the motion simulation result, and analyze the simulation deviation of the motion simulation result.

[0063] When the simulation deviation meets the preset deviation, the motion simulation result is used as the joint analysis result.

[0064] In a second aspect, the present invention provides an intelligent control system for realizing camera images based on the Internet of Things, the system comprising:

[0065] A device adjustment module is used to deploy an Internet of Things device in a camera monitoring area, establish a communication connection between the Internet of Things device and the camera monitoring area, collect environmental parameters of the camera monitoring area using the Internet of Things device, and adjust the parameters of the camera system equipment in the camera monitoring area based on the environmental parameters to obtain an adjusted camera device;

[0066] An environment model building module is used to collect point cloud data of the camera monitoring area, use the point cloud data to perform element modeling on the camera monitoring area to obtain a camera monitoring area model, perform quadratic error measurement on the camera monitoring area model to obtain a target environment model;

[0067] A target dynamic analysis module is used to capture the image information of the camera monitoring area in real time to obtain the analysis target, collect the infrared image and the visible light image of the analysis target, and analyze the target dynamic trend of the analysis target by using the infrared image and the visible light image;

[0068] The camera image intelligent control module is used to collect the dynamic image of the analysis target based on the dynamic trend of the target, use the target dynamic image to perform multivariate composite mapping on the target environment model to obtain a real model, use the real model to perform a joint analysis on the analysis target to obtain a joint analysis result, and based on the joint analysis result, formulate an intelligent control strategy for the camera device in the camera monitoring area to use the intelligent control strategy to perform real-time control of the camera image in the camera monitoring area.

[0069] Compared with the prior art, the technical principle and beneficial effects of this solution are:

[0070] The embodiment of the present invention can enhance the monitoring accuracy of the camera monitoring area by deploying the Internet of Things device in the camera monitoring area, realize intelligent early warning by linking with the camera monitoring, and collect the point cloud data of the camera monitoring area to understand the environmental information of the camera monitoring area in detail, such as the distribution of buildings, the layout of streets and the undulation of terrain, and provide intuitive and accurate data support for planning decisions. Further, the embodiment of the present invention can capture the picture information of the camera monitoring area in real time to obtain the analysis target, and collect the infrared image and visible light image of the analysis target to continuously obtain the image data of the monitoring area, and determine the specific object or target that needs to be analyzed in depth, and the infrared image can highlight the heat distribution of the target, such as the heating area of ​​the vehicle engine, and the visible light image can clearly show the appearance characteristics of the target. Furthermore, the embodiment of the present invention can deeply understand the behavior pattern of the target by collecting the dynamic picture of the analysis target based on the dynamic trend of the target, such as by collecting the dynamic trend picture of the vehicle, the traffic flow distribution in different time periods and sections can be understood. The embodiment of the present invention proposes a method and system for realizing intelligent control of camera images based on the Internet of Things, which can improve the control accuracy of camera images. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0072] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0073] Figure 1 A schematic diagram of a flow chart of an intelligent control method for camera images based on the Internet of Things provided by one embodiment of the present invention;

[0074] Figure 2 A schematic diagram of a module of an intelligent control system for realizing camera images based on the Internet of Things provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0075] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0076] The embodiment of the present invention provides an intelligent control method for camera images based on the Internet of Things, and the execution subject of the intelligent control method for camera images based on the Internet of Things includes but is not limited to at least one of the electronic devices such as a server and a terminal that can be configured to execute the method provided by the embodiment of the present invention. In other words, the intelligent control method for camera images based on the Internet of Things can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms.

[0077] See also Figure 1 FIG. 1 is a flow chart of an intelligent control method for realizing camera images based on the Internet of Things provided by an embodiment of the present invention. Figure 1 An intelligent control method for camera images based on the Internet of Things described in the specification includes:

[0078] S1. Deploy an Internet of Things device in a video surveillance area, establish a communication connection between the Internet of Things device and the video surveillance area, use the Internet of Things device to collect environmental parameters of the video surveillance area, and adjust the equipment parameters of the camera system of the video surveillance area based on the environmental parameters to obtain an adjusted camera device.

[0079] The embodiment of the present invention can enhance the monitoring accuracy of the video surveillance area by deploying the Internet of Things device in the video surveillance area, and realize intelligent early warning by linking with the video surveillance. The Internet of Things device can include sensors such as temperature sensors, humidity sensors, smoke sensors, light sensors, etc., for collecting environmental data and environmental monitoring equipment, remote sensing devices, etc.

[0080] Furthermore, after the embodiment of the present invention establishes a communication connection between the Internet of Things device and the camera monitoring area, the environmental parameters of the camera monitoring area are collected by the Internet of Things device to understand the environmental status of the camera monitoring area, and then the camera performance of the camera monitoring area is analyzed in real time, thereby facilitating real-time adjustment of the camera parameters of the camera device.

[0081] The environmental parameters can be obtained by collecting parameters such as light intensity, temperature and humidity of the camera monitoring area through temperature sensors, humidity sensors and light sensors.

[0082] Furthermore, the embodiment of the present invention adjusts the parameters of the camera system equipment of the camera monitoring area based on the environmental parameters, so that the camera device can be adjusted to better adapt to the on-site environment, thereby capturing high-quality images.

[0083] As an embodiment of the present invention, the camera system equipment parameters of the camera monitoring area are adjusted based on the environmental parameters to obtain an adjusted camera device, including: determining the light intensity, environmental decibels and environmental humidity of the camera monitoring area according to the environmental parameters, judging the environmental conditions of the camera monitoring area based on the light intensity, the environmental decibels and the environmental humidity, and when the environmental conditions do not meet the preset environmental conditions, adjusting the camera system equipment parameters of the camera monitoring area to obtain an adjusted camera device.

[0084] Optionally, the light intensity, ambient decibel and ambient humidity of the video surveillance area determined according to the environmental parameters can be obtained by directly querying the collected data of the environmental parameters. The environmental condition of the video surveillance area is judged based on the light intensity, the ambient decibel and the ambient humidity. For example, the light intensity of the current environment detected by the light sensor is 500 lux, the ambient temperature detected by the temperature sensor is 12 degrees Celsius, the humidity detected by the humidity sensor is 80%, and the sound intensity detected by the sound sensor is 60 decibels. Under this condition, the environmental condition of the video surveillance area is poor, and it is necessary to adjust the parameters of the camera equipment, such as exposure time, gain and white balance. The camera system equipment parameters of the video surveillance area are adjusted to obtain the adjusted camera device by comparing the environmental parameters with a preset threshold. For example, if the light intensity threshold is 800 lux and the light intensity detected in the environmental parameters is 500, it is necessary to adjust the exposure time and gain of the camera device. Assuming that in a dark environment, the exposure time is adjusted to 0.1 seconds and the gain is increased to 10 dB. The specific adjustment needs to be set in combination with the actual environmental information and the camera device.

[0085] S2. Collect point cloud data of the camera monitoring area, use the point cloud data to perform element modeling on the camera monitoring area to obtain a camera monitoring area model, perform quadratic error measurement on the camera monitoring area model to obtain a target environment model.

[0086] The embodiment of the present invention can obtain detailed information about the environment of the video surveillance area by collecting the point cloud data of the video surveillance area, such as the distribution of buildings, the layout of streets, and the undulations of the terrain, and provide intuitive and accurate data support for planning decisions. The point cloud data refers to a collection of a large number of spatial points obtained by scanning and measuring the surrounding environment through specific technical means. These spatial points contain information such as the position, shape, and size of various objects and spaces in the environment, and can present the three-dimensional characteristics of the environment in a high-precision and high-resolution manner. The point cloud data can indirectly generate a point cloud from a spectral image through an image derivation method.

[0087] Furthermore, the embodiment of the present invention uses the point cloud data to perform element modeling on the camera surveillance area, and obtains a camera surveillance area model, which can provide a user with a clear and comprehensive understanding, so as to effectively plan, manage and evaluate the performance and coverage effect of the surveillance system. The camera surveillance area model refers to a virtual digital copy constructed for the camera surveillance area.

[0088] As an embodiment of the present invention, the method of using the point cloud data to perform element modeling on the camera monitoring area to obtain a camera monitoring area model includes: dividing the point cloud data into three-dimensional voxel grids, identifying the center of gravity of the three-dimensional voxel grids, based on the center of gravity, reducing the density of the three-dimensional voxel grids to obtain a filtered point cloud, performing point cloud stitching on the filtered point cloud to obtain a stitched point cloud, and using the stitched point cloud to construct an environmental model of the camera monitoring area to obtain a camera monitoring area model.

[0089] Optionally, the point cloud data is divided into three-dimensional voxel grids by a voxel grid method, and the process of identifying the centroid point of the three-dimensional voxel grid is as follows: traversing the number of voxel points in the three-dimensional voxel grid and calculating the center of the voxel points; based on the centroid point, the three-dimensional voxel grid is densified to obtain a filtered point cloud, which is achieved by a voxel filtering function.

[0090] Furthermore, in another optional embodiment of the present invention, the step of performing point cloud stitching on the filtered point cloud to obtain a stitched point cloud includes: extracting adjacent point clouds from the filtered point cloud, and using the adjacent point clouds to create a rotation matrix of the filtered point cloud, wherein the rotation matrix can be expressed in the following form:

[0091]

[0092] Among them, ω represents the rotation matrix, R represents the cross vector of adjacent point clouds, and Zr x , Z y , Z z Represents the three-dimensional space coordinates of the intersection vector of adjacent point clouds, θ represents the intersection angle of adjacent point clouds, and E represents the unit vector of adjacent point clouds;

[0093] After the ground normal vector angles of the filtered point clouds are rotated to be consistent using the rotation matrix, the filtered point clouds are spliced ​​to obtain a spliced ​​point cloud.

[0094] Furthermore, the embodiment of the present invention can reduce the number of polygonal faces of the camera monitoring area model by performing quadratic error measurement on the camera monitoring area model to obtain the target environment model, thereby reducing the complexity of the model and improving the operating efficiency of the model.

[0095] As an embodiment of the present invention, the quadratic error measurement of the camera monitoring area model to obtain the target environment model includes: taking the monitoring coverage, image clarity, target recognition accuracy and object motion trajectory as the key features of the camera monitoring area model, calculating the feature errors of the key features, and based on the feature errors, calculating the comprehensive quadratic error of the camera monitoring area model using the following formula:

[0096] E 总 =W1E1+W2E2+W3E3+W4E4

[0097] Among them, E 总 represents the comprehensive quadratic error, E1 represents the monitoring coverage error in the feature error, E2 represents the image clarity error in the feature error, E3 represents the target recognition accuracy error in the feature error, E4 represents the object motion trajectory error in the feature error, W1 represents the weight corresponding to the monitoring coverage error in the feature error, W2 represents the weight corresponding to the image clarity error in the feature error, W3 represents the weight corresponding to the target recognition accuracy error in the feature error, and W4 represents the weight corresponding to the object motion trajectory error in the feature error;

[0098] Based on the comprehensive quadratic error, the camera monitoring area model is adjusted to obtain a target environment model.

[0099] Optionally, the process of adjusting the camera surveillance area model based on the comprehensive quadratic error to obtain the target environment model is as follows: analyzing the error distribution of different monitoring positions and different times of the camera surveillance area based on the comprehensive quadratic error, and determining the areas and time periods with larger errors to adjust the camera surveillance area model. If the monitoring images at specific angles or distances have larger errors, it may be that the model's simulation of the viewing angle and distance is inaccurate; if the error is prominent in a specific time period, it may be that the modeling of environmental factors (such as changes in lighting) is incomplete, and the parameters of the pre-configured neural network in the model can be adjusted.

[0100] Further, as an optional embodiment of the present invention, the calculating the feature error of the key feature includes: calculating the monitoring coverage error in the key feature using the following formula:

[0101] E1=(A actual -A model) 2

[0102] Among them, E1 represents the monitoring coverage error, A actual Indicates the area of ​​the actual monitoring area, A model represents the area predicted by the model;

[0103] The image clarity error in the key feature is calculated using the following formula:

[0104] E2=∑ x ∑ y (P actual (x,y)-P model (x,y) 2

[0105] Among them, E2 represents the image clarity error, P actual (x, y) represents the pixel of the key feature corresponding to the coordinate point (x, y) in the original image, P model (x,y) represents the pixel at point (x,y) in the model;

[0106] The target recognition accuracy error in the key feature is calculated using the following formula:

[0107] E3=(N actual -N model ) 2

[0108] Among them, E3 represents the target recognition accuracy error, N actual Indicates the number of correctly identified targets, N model Indicates the correct number of targets predicted by the model;

[0109] The object motion trajectory error in the key feature is calculated using the following formula:

[0110]

[0111] Among them, E4 represents the object motion trajectory error, O represents the object motion coordinate sequence, A coordinate sequence representing the motion trajectory of an object in the monitoring area, A sequence of coordinates representing the trajectory of an object predicted by the model.

[0112] S3. Capturing the image information of the camera monitoring area in real time to obtain the analysis target, collecting the infrared image and the visible light image of the analysis target, and analyzing the target dynamic trend of the analysis target by using the infrared image and the visible light image.

[0113] The embodiment of the present invention can obtain the analysis target by capturing the picture information of the camera monitoring area in real time, and collect the infrared image and visible light image of the analysis target. It can continuously obtain the image data of the monitoring area, and determine the specific object or target that needs to be deeply analyzed. The infrared image can highlight the heat distribution of the target, such as the heating area of ​​the vehicle engine, and the visible light image can clearly show the appearance characteristics of the target, such as the shape, color and license plate of the vehicle, which can be collected by the image acquisition device in the Internet of Things device.

[0114] Furthermore, the embodiment of the present invention can understand the past movement patterns and trends of the target by analyzing the target dynamic trends of the analysis target using the infrared image and the visible light image, and can predict its future position and behavior.

[0115] As an embodiment of the present invention, the use of the infrared image and the visible light image to analyze the target dynamic trend of the analysis target includes: extracting image features of the infrared image and the visible light image to obtain a first feature and a second feature, performing feature fusion on the first feature and the second feature to obtain a fusion feature, tracking the analysis target based on the fusion feature to obtain a target movement trajectory, and analyzing the target dynamic trend of the analysis target based on the target movement trajectory.

[0116] Optionally, the image features of the infrared image and the visible light image can be extracted through a dual-stream network, the feature fusion of the first feature and the second feature to obtain the fused feature can be fused through a fully convolutional network, and the target tracking of the analysis target based on the fused feature can be obtained by marking the analysis target using the fused feature and continuously collecting associated images of the analysis target.

[0117] Optionally, as another optional embodiment of the present invention, the target dynamic trend of the analysis target based on the target motion trajectory includes: segmenting the target motion trajectory to obtain segmented motion trajectories, querying motion parameters of the segmented motion trajectory, identifying trajectory features of the target motion trajectory based on the action parameters, performing curve simulation on the target motion trajectory based on the trajectory features to obtain a simulation curve, and analyzing the target dynamic trend of the target based on the simulation curve.

[0118] The action parameters refer to the changes in the average speed, acceleration, turning frequency and movement direction of the analysis target, as well as the position coordinates, speed, direction and other parameters of the target at different time points.

[0119] Optionally, the target action trajectory is segmented to obtain a segmented action trajectory, which means dividing the target action trajectory into different segments, each segment can represent a relatively stable motion mode, and the trajectory characteristics of the target action trajectory are identified based on the action parameters, wherein the trajectory characteristics are motion characteristics such as linear motion, curved motion, pause, acceleration and deceleration, etc., specifically combined with motion speed analysis, and the target action trajectory is simulated based on the trajectory characteristics to obtain a simulation curve which can be constructed through a machine learning algorithm, and the target dynamic trend analyzed based on the simulation curve can be described by constructing a state transition diagram or a Markov model of the analyzed target through the simulation curve to describe the transition probability between different motion modes, thereby inferring the possible dynamic trend of the target in the future.

[0120] S4. Based on the dynamic trend of the target, collect the dynamic images of the analysis target, use the target dynamic images to perform multivariate composite mapping on the target environment model to obtain a real model, use the real model to perform a joint analysis on the analysis target to obtain a joint analysis result, and based on the joint analysis result, formulate an intelligent control strategy for the camera device in the camera monitoring area to use the intelligent control strategy to perform real-time control of the camera image in the camera monitoring area.

[0121] The embodiment of the present invention can deeply understand the behavior pattern of the target by collecting the dynamic image of the target based on the dynamic trend of the target, such as by collecting the dynamic trend image of the vehicle, the traffic flow distribution in different time periods and sections can be understood. For example, if it is found that a certain intersection is always severely congested during peak hours in the morning and evening, the traffic management department can adjust the duration of the signal light, add a special turning lane, or implement traffic restriction measures based on the image information to optimize the traffic flow.

[0122] Furthermore, the embodiment of the present invention uses the target dynamic picture to perform multivariate composite mapping on the target environment model to obtain a real model, which can be used to finely construct the model to improve the accuracy of the model.

[0123] As an embodiment of the present invention, the target environment model is subjected to multivariate composite mapping using the target dynamic picture to obtain a real model, including: extracting key features from the target dynamic picture, determining the mappable features of the target environment model, matching the key features with the mappable features to obtain matching features, and performing multivariate composite mapping on the target environment model based on the matching features to obtain a real model. Wherein, the key features may be the size and appearance of the target. For example, in a traffic scene, the key features may be the position, speed, acceleration, driving direction and vehicle spacing features of the vehicle. The determination of the mappable features of the target environment model refers to features that can be mapped in the target environment model, which can be identified by querying the basic code generated by the target environment model. The feature matching of the key features with the mappable features to obtain matching features refers to matching the key features with the mappable features. For example, if the mappable features in the traffic scene include the position, speed and appearance of the vehicle, the corresponding features are selected from the key features. The target environment model is subjected to multivariate composite mapping based on the matching features to obtain a real model by converting the matching features into feature values ​​and then fusing them using the feature values.

[0124] Furthermore, the embodiment of the present invention uses the real model to perform a joint analysis on the analysis target, and the joint analysis result can accurately predict the target's direction of movement, and then perform subsequent camera deployment. For example, in a shopping mall, if a suspicious person is found always wandering around the valuable goods area, the monitoring and patrol of the area can be strengthened.

[0125] As an embodiment of the present invention, the use of the real model to perform a joint analysis on the analysis target to obtain a joint analysis result includes: extracting feature data in the dynamic picture of the analysis target, integrating the feature data to obtain integrated data, initializing the real model using the integrated data to obtain an initialized model, performing motion simulation on the analysis target using the initialized model to obtain a motion simulation result, analyzing the simulation deviation of the motion simulation result, and when the simulation deviation meets the preset deviation, using the motion simulation result as the joint analysis result.

[0126] Among them, the characteristic data refers to the action and distribution characteristics of the target, such as the vehicle speed, position and flow in the traffic scene, etc., the data integration of the characteristic data to obtain the integrated data refers to integrating the characteristic data with the scene of the real model, such as the topological structure of the traffic road, the setting of the signal light and the traffic rules, etc., the initialization setting of the real model using the integrated data to obtain the initialized model refers to setting the corresponding initial conditions in the real model with the current environmental state as the starting point, and the simulation deviation of the motion simulation result can be analyzed by comparing the results to find the deviation between the simulation result and the actual observation result. For example, if the congestion level of a certain road section predicted by the real model is lower than the actual observation, the possible reasons are analyzed, such as the inaccurate simulation of the vehicle merging behavior in the model, or the underestimation of the traffic demand, and when the simulation deviation meets the preset deviation, it means that the error between the simulation result of the real model and the actual observation result is within an acceptable range, such as the average queue length of the east-west straight vehicles predicted by the real model on the traffic road is 50 meters, and the actual observation result is 55 meters, indicating that the error is small and within an acceptable range, which needs to be set in combination with the actual application scenario.

[0127] Furthermore, the embodiment of the present invention formulates an intelligent control strategy for the camera device in the camera monitoring area based on the joint analysis result, so as to use the intelligent control strategy to control the camera image in the camera monitoring area in real time. According to the on-site situation of the analysis target, the focal length, angle and shooting range of the camera can be intelligently adjusted to ensure that key areas are monitored more clearly and comprehensively, and to avoid the waste of monitoring resources. For example, on the peripheral roads of a large commercial center, the joint analysis results show that the traffic flow is large and there are many pedestrians in the afternoon on weekends. The intelligent control strategy can adjust the nearby cameras to wide-angle shooting during this time period to cover a larger area; at the same time, the shooting resolution can be improved to capture the details of vehicles and pedestrians more clearly. In this way, traffic conditions can be better monitored, and possible problems such as vehicle scratches or pedestrian accidents can be discovered and handled in a timely manner.

[0128] like Figure 2 The figure shows a functional module diagram of an intelligent control system for realizing camera images based on the Internet of Things according to the present invention.

[0129] The intelligent control system 200 for realizing camera images based on the Internet of Things described in the present invention can be installed in an electronic device. According to the functions implemented, the intelligent control system for realizing camera images based on the Internet of Things can include a device adjustment module 201, an environment model construction module 202, a target dynamic analysis module 203, and a camera image intelligent control module 204. The module described in the present invention can also be called a unit, which refers to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, which are stored in the memory of the electronic device.

[0130] In the embodiment of the present invention, the functions of each module / unit are as follows:

[0131] The device adjustment module 201 is used to deploy an Internet of Things device in a camera monitoring area, establish a communication connection between the Internet of Things device and the camera monitoring area, use the Internet of Things device to collect environmental parameters of the camera monitoring area, and adjust the camera system equipment parameters of the camera monitoring area based on the environmental parameters to obtain an adjusted camera device;

[0132] The environment model building module 202 is used to collect point cloud data of the camera monitoring area, use the point cloud data to perform element modeling on the camera monitoring area to obtain a camera monitoring area model, and perform quadratic error measurement on the camera monitoring area model to obtain a target environment model;

[0133] The target dynamic analysis module 203 is used to capture the image information of the camera monitoring area in real time to obtain the analysis target, collect the infrared image and the visible light image of the analysis target, and analyze the target dynamic trend of the analysis target by using the infrared image and the visible light image;

[0134] The camera image intelligent control module 204 is used to collect the dynamic image of the analysis target based on the dynamic trend of the target, use the target dynamic image to perform multivariate composite mapping on the target environment model to obtain a real model, use the real model to perform a joint analysis on the analysis target to obtain a joint analysis result, and based on the joint analysis result, formulate an intelligent control strategy for the camera device in the camera monitoring area to use the intelligent control strategy to perform real-time control of the camera image of the camera monitoring area.

[0135] In detail, the modules in the intelligent control system 200 for realizing camera images based on the Internet of Things described in the embodiment of the present invention are used in the same manner as described above. Figure 1 The technical means described in the article are the same as the intelligent control method for camera images based on the Internet of Things, and can produce the same technical effects, so they will not be repeated here.

[0136] In the several embodiments provided by the present invention, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are only illustrative, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

[0137] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0138] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0139] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0140] Therefore, no matter from which point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present invention is limited by the appended claims rather than the above description, so it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any attached figure mark in the claims should not be regarded as limiting the claims involved.

[0141] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0142] The above description is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. An intelligent control method for camera images based on the Internet of Things, characterized in that: The method comprises: An Internet of Things device is deployed in a video surveillance area, and after the Internet of Things device is connected to the video surveillance area through a communication connection, the environmental parameters of the video surveillance area are collected by using the Internet of Things device, and the parameters of the camera system equipment in the video surveillance area are adjusted based on the environmental parameters to obtain an adjusted camera device; Collecting point cloud data of the camera monitoring area, using the point cloud data to perform element modeling on the camera monitoring area to obtain a camera monitoring area model, and performing secondary error measurement on the camera monitoring area model to obtain a target environment model; Capturing the image information of the camera monitoring area in real time to obtain the analysis target, collecting the infrared image and the visible light image of the analysis target, and analyzing the target dynamic trend of the analysis target by using the infrared image and the visible light image; Based on the dynamic trend of the target, the dynamic picture of the analysis target is collected, and the target environment model is mapped with a multivariate composite map using the target dynamic picture to obtain a real model. The analysis target is jointly analyzed using the real model to obtain a joint analysis result. Based on the joint analysis result, an intelligent control strategy for the camera device in the camera monitoring area is formulated to use the intelligent control strategy to perform real-time control of the camera picture in the camera monitoring area.

2. The method according to claim 1, characterized in that The step of adjusting the parameters of the camera system equipment in the camera monitoring area based on the environmental parameters to obtain an adjusted camera device includes: Determine the light intensity, ambient decibel and ambient humidity of the camera monitoring area according to the environmental parameters; Based on the light intensity, the ambient decibel level and the ambient humidity, determining the ambient condition of the camera monitoring area; When the environmental condition does not meet the preset environmental condition, the camera system equipment parameters of the camera monitoring area are adjusted to obtain an adjusted camera device.

3. The method according to claim 1, characterized in that The step of performing element modeling on the camera monitoring area by using the point cloud data to obtain a camera monitoring area model includes: Dividing the point cloud data into three-dimensional voxel grids; Identifying a centroid point of the three-dimensional voxel grid; Based on the centroid point, the three-dimensional voxel grid is de-densified to obtain a filtered point cloud; Performing point cloud stitching on the filtered point cloud to obtain a stitched point cloud; The environment model of the camera monitoring area is constructed using the spliced ​​point cloud to obtain a camera monitoring area model.

4. The method according to claim 3, characterized in that The step of performing point cloud stitching on the filtered point cloud to obtain a stitched point cloud comprises: Extracting adjacent point clouds from the filtered point cloud; Using the adjacent point clouds, a rotation matrix of the filtered point cloud is created, wherein the rotation matrix can be expressed in the following form: Among them, ω represents the rotation matrix, R represents the cross vector of adjacent point clouds, and Zr x , Z y , Z z Represents the three-dimensional space coordinates of the intersection vector of adjacent point clouds, θ represents the intersection angle of adjacent point clouds, and E represents the unit vector of adjacent point clouds; After the ground normal vector angles of the filtered point clouds are rotated to be consistent using the rotation matrix, the filtered point clouds are spliced ​​to obtain a spliced ​​point cloud.

5. The method according to claim 1, characterized in that The performing of a secondary error measurement on the camera monitoring area model to obtain a target environment model includes: The monitoring coverage, image clarity, target recognition accuracy and object motion trajectory are used as key features of the camera monitoring area model; Calculating a characteristic error of the key characteristic; Based on the characteristic error, the comprehensive quadratic error of the camera monitoring area model is calculated using the following formula: AND 总 =W1E1+W2E2+W3E3+W4E4 Among them, E 总 represents the comprehensive quadratic error, E1 represents the monitoring coverage error in the feature error, E2 represents the image clarity error in the feature error, E3 represents the target recognition accuracy error in the feature error, E4 represents the object motion trajectory error in the feature error, W1 represents the weight corresponding to the monitoring coverage error in the feature error, W2 represents the weight corresponding to the image clarity error in the feature error, W3 represents the weight corresponding to the target recognition accuracy error in the feature error, and W4 represents the weight corresponding to the object motion trajectory error in the feature error; Based on the comprehensive quadratic error, the camera monitoring area model is adjusted to obtain a target environment model.

6. The method according to claim 5, characterized in that The calculating the characteristic error of the key characteristic comprises: The monitoring coverage error in the key feature is calculated using the following formula: E1=(A actual -A model ) 2 Among them, E1 represents the monitoring coverage error, A actual Indicates the area of ​​the actual monitoring area, A model represents the area predicted by the model; The image clarity error in the key feature is calculated using the following formula: E2=∑ x ∑ y (P actual (x,y)-P model (x,y)) 2 Among them, E2 represents the image clarity error, P actual (x, y) represents the pixel of the key feature corresponding to the coordinate point (x, y) in the original image, P model (x,y) represents the pixel at point (x,y) in the model; The target recognition accuracy error in the key feature is calculated using the following formula: E3=(N actual -N model ) 2 Among them, E3 represents the target recognition accuracy error, N actual Indicates the number of correctly identified targets, N model Indicates the correct number of targets predicted by the model; The object motion trajectory error in the key feature is calculated using the following formula: Among them, E4 represents the object motion trajectory error, O represents the object motion coordinate sequence, A coordinate sequence representing the motion trajectory of an object in the monitoring area, A sequence of coordinates representing the trajectory of an object predicted by the model.

7. The method according to claim 1, characterized in that The step of analyzing the target dynamic trend of the analysis target by using the infrared image and the visible light image includes: Extracting image features of the infrared image and the visible light image to obtain a first feature and a second feature; Performing feature fusion on the first feature and the second feature to obtain a fused feature; Tracking the analysis target based on the fusion feature to obtain the target action trajectory; The target dynamic trend of the analysis target is analyzed based on the target action trajectory.

8. The method according to claim 7, characterized in that Analyzing the target dynamic trend of the analysis target based on the target action trajectory includes: Segmenting the target motion trajectory to obtain a segmented motion trajectory; Querying the action parameters of the segmented action trajectory; Identifying trajectory features of the target action trajectory based on the action parameters; Performing curve simulation on the target action trajectory based on the trajectory characteristics to obtain a simulation curve; The target dynamic trend of the target is analyzed based on the simulation curve.

9. The method according to claim 1, characterized in that: Using the true model, a joint analysis is performed on the analysis target to obtain a joint analysis result, including: Extracting feature data from the dynamic picture of the analysis target; Performing data integration on the feature data to obtain integrated data; Initializing the real model using the integrated data to obtain an initialized model; Using the initialization model to perform motion simulation on the analysis target to obtain a motion simulation result; analyzing simulation deviations of the motion simulation results; When the simulation deviation meets the preset deviation, the motion simulation result is used as the joint analysis result.

10. An intelligent control system for camera images based on the Internet of Things, characterized in that: The system comprises: A device adjustment module is used to deploy an Internet of Things device in a camera monitoring area, establish a communication connection between the Internet of Things device and the camera monitoring area, collect environmental parameters of the camera monitoring area using the Internet of Things device, and adjust the parameters of the camera system equipment in the camera monitoring area based on the environmental parameters to obtain an adjusted camera device; An environment model building module is used to collect point cloud data of the camera monitoring area, use the point cloud data to perform element modeling on the camera monitoring area to obtain a camera monitoring area model, perform quadratic error measurement on the camera monitoring area model to obtain a target environment model; A target dynamic analysis module is used to capture the image information of the camera monitoring area in real time to obtain the analysis target, collect the infrared image and the visible light image of the analysis target, and analyze the target dynamic trend of the analysis target by using the infrared image and the visible light image; The camera image intelligent control module is used to collect the dynamic image of the analysis target based on the dynamic trend of the target, use the target dynamic image to perform multivariate composite mapping on the target environment model to obtain a real model, use the real model to perform a joint analysis on the analysis target to obtain a joint analysis result, and based on the joint analysis result, formulate an intelligent control strategy for the camera device in the camera monitoring area to use the intelligent control strategy to perform real-time control of the camera image in the camera monitoring area.

Images