A monitoring method, device, equipment and storage medium in a smart station

Abstract

The present invention relates to a monitoring method, device, equipment and storage medium in a smart station, and to the field of station management technology, including: collecting images in the smart station, identifying and determining monitoring targets, and continuously collecting and identifying continuous feature information of monitoring targets; matching and obtaining public equipment distribution information in the smart station according to the monitoring targets; performing target trend prediction according to the continuous feature information and public equipment distribution information, and obtaining target trend prediction results, wherein the target trend prediction includes movement prediction and behavior prediction; when the target trend prediction results meet the preset conditions, generating corresponding monitoring management strategies for management. The present invention solves the technical problem that the "camera + manual monitoring solution" in the prior art has a low intelligence level and cannot make comprehensive and scientific judgments on dangerous behaviors in smart stations.

Description

Technical Field

[0001] The present invention relates to the field of station management, and in particular to a monitoring method, device, equipment and storage medium in a smart station. Background Art

[0002] With the development of automobile technology and the improvement of living standards, people have more diverse requirements for stations, such as adding charging pile functions, adding locks that can automatically identify vehicles, adding directions for parked vehicles, adding reminders for empty parking spaces, etc. Therefore, smart stations came into being. By using advanced information technology, Internet of Things technology, intelligent control technology, etc., various facilities, equipment, and operation management of the stations are fully upgraded and integrated to achieve efficient, safe, convenient, and green operation and management goals.

[0003] At the same time, the on-site operation and maintenance of smart stations has gradually exposed some problems. People do not actually understand some of the new functions added to the stations, and often rely on their own understanding to do many things that will actually damage the station’s property (for example, unplugging a charging gun that is charging without authorization, damaging the charging gun and the charging port, and easily bringing the risk of electric shock and fire).

[0004] In this regard, the existing smart stations that only use the previous "camera + manual monitoring solution" can no longer meet the monitoring needs of smart stations with increasingly more functions, and require a more intelligent and comprehensive monitoring solution.

[0005] The existing technology of "camera + manual monitoring solution" has a low level of intelligence and is unable to make comprehensive and scientific judgments on dangerous behaviors in smart stations. Summary of the invention

[0006] The present invention aims to solve the technical problem that the "camera + manual monitoring solution" in the prior art has a low intelligence level and is unable to make comprehensive and scientific judgments on dangerous behaviors in smart stations, resulting in difficulty in accurately judging dangerous behaviors. The present invention provides a monitoring method, device, equipment and storage medium in a smart station.

[0007] In a first aspect, the present invention provides a monitoring method within a smart station, comprising: collecting images within the smart station, identifying and determining a monitoring target, and continuously collecting and identifying continuous feature information of the monitoring target; matching and acquiring public equipment distribution information within the smart station according to the monitoring target, wherein the public equipment distribution information includes first public equipment distribution information, second public equipment distribution information, and third public equipment distribution information; performing target trend prediction based on the continuous feature information and the public equipment distribution information to obtain a target trend prediction result, wherein the target trend prediction includes movement prediction and behavior prediction; when the target trend prediction result meets a preset condition, generating a corresponding monitoring management strategy for management.

[0008] In the second aspect, the present invention provides a monitoring device in a smart station, including: a feature information acquisition module, used to collect images in the smart station, identify and determine the monitoring target, and continuously collect and identify the continuous feature information of the monitoring target; a distribution information matching module, used to match and obtain the public equipment distribution information in the smart station according to the monitoring target, wherein the public equipment distribution information includes first public equipment distribution information, second public equipment distribution information and third public equipment distribution information; a target trend prediction module, used to perform target trend prediction based on the continuous feature information and the public equipment distribution information, and obtain a target trend prediction result, wherein the target trend prediction includes movement prediction and behavior prediction; a management strategy generation module, used to generate a corresponding monitoring management strategy for management when the target trend prediction result meets a preset condition.

[0009] In a third aspect, the present invention provides an electronic device, comprising: a memory for storing a computer software program; and a processor for reading and executing the computer software program, thereby implementing a monitoring method within a smart station as described in any one of the first aspects.

[0010] In a fourth aspect, the present invention provides a non-transitory computer-readable storage medium, in which a computer software program is stored. When the computer software program is executed by a processor, a monitoring method within a smart station as described in any one of the first aspects is implemented.

[0011] The beneficial effects of the present invention are: it achieves station monitoring by combining path prediction and behavior detection on the basis of image detection, and by dividing the public equipment in the station, finely setting corresponding protection mechanisms for the three types of public equipment, thereby ensuring that the misjudgment rate is reduced as much as possible, and providing a more intelligent and comprehensive monitoring solution for smart station management and monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1A flow chart of a monitoring method in a smart station provided by the present invention;

[0013] Figure 2 A schematic diagram of the structure of a monitoring device in a smart station provided by the present invention;

[0014] Figure 3 A functional block diagram of the electronic device provided by the present invention;

[0015] Figure 4 A functional block diagram of a computer-readable storage device provided by the present invention.

[0016] In the accompanying drawings, the components represented by the reference numerals are listed as follows:

[0017] Feature information collection module 100, distribution information matching module 200, target trend prediction module 300, management strategy generation module 400, electronic device 500, memory 510, processor 520, computer program 511, computer readable storage medium 600, computer program 611. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0019] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, "plurality" means two or more, unless otherwise clearly and specifically defined.

[0020] In the description of the present application, the term "for example" is used to mean "used as an example, illustration or description". Any embodiment described as "for example" in the present application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is given to enable any technician in the field to implement and use the present invention. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present invention can be implemented without using these specific details. In other examples, well-known structures and processes will not be elaborated in detail to avoid unnecessary details to obscure the description of the present invention. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in the present application.

[0021] Example 1

[0022] like Figure 1 As shown, an embodiment of the present invention provides a monitoring method in a smart station, including:

[0023] S10: Collect images in the smart station, identify and determine the monitoring target, and continuously collect and identify continuous feature information of the monitoring target;

[0024] Furthermore, the monitoring target is determined, and the continuous feature information for identifying the monitoring target is continuously collected, step S10 includes:

[0025] S11: According to the monitoring data records in the smart station, a sample station image set is collected, and the targets in the sample station image are identified and marked to obtain a sample monitoring target set;

[0026] S12: using the sample station image set and the sample monitoring target set to train a target identifier based on a convolutional neural network;

[0027] S13: collecting images of the smart station, inputting them into the target identifier for target identification, and obtaining the monitoring target and basic target information;

[0028] S14: Continuously collect site images including the monitored target, and identify continuous feature information of the monitored target.

[0029] Specifically, the method for monitoring the smart station is applied to the corresponding monitoring device, which includes a camera device and a central control center. The monitoring device mainly includes multiple cameras, which are installed in various corners of the smart station to monitor the situation in various places of the smart station and transmit the corresponding images back to the central control center. The central control center mainly communicates and interacts with various public equipment installed in the smart station. The public equipment includes but is not limited to smart car locks, smart fire extinguishing systems, parked vehicle route prompt devices, empty parking space prompt devices, charging piles and other common public equipment.

[0030] Image acquisition is performed through multiple cameras in the smart station. It is necessary to identify and determine the monitoring target from the acquired images and identify it through the target identifier. The target identifier is built based on a convolutional neural network for target recognition in the smart station.

[0031] The specific training process of the target identifier includes retrieving monitoring data records in the current smart station to obtain the sample station image set. The sample station image set includes image information of several targets in the station, such as pedestrians, vehicles, charging piles, prompt devices corresponding to charging piles, shops or express racks and other service areas, etc. Target recognition and identification are performed according to the characteristics of different targets. Exemplarily, it can be completed by manual monitoring to obtain the sample monitoring target set. The sample monitoring target set is image information with identification, including gender (male / female), age (adult / child / elderly), entry method of the station (walking entry / vehicle entry), role in the entry method of the station (classifying passengers and drivers in the case of vehicle entry), etc.

[0032] Furthermore, the sample station image set and the sample monitoring target set are divided into an 8:2 ratio, and 80% of the data is set as training data, and 20% of the data is set as verification data. When training the target identifier, any set of sample station image sets and sample monitoring target sets are input into the convolutional neural network as training input data and target output information. After forward propagation through the convolution layer, pooling layer, and fully connected layer, the output result, that is, the predicted image identification result, is obtained. When the output image identification result is consistent with the input identification information, the verification counter is +1, and when the output image identification result is inconsistent with the input identification information, the verification counter is set to 0; continue to select sample station image sets and sample monitoring target sets from the training data for training. When the number of the verification counter is greater than or equal to the preset number of times, use the verification data for verification. The verification method is exactly the same as the training method. When the count of the counter in the verification process is greater than or equal to the preset number of times, it is considered that the convolutional neural network has converged, otherwise the training data is returned to continue training the target identifier.

[0033] After the target identifier is trained, the images of the smart station are collected in real time and input into the target identifier. After receiving the input information, the target identifier extracts the features and converts the feature map into the final output of the network to obtain the monitoring target and target basic information. The monitoring target is the individual in the station, and the target basic information is the relevant information of the individual, including but not limited to: gender (male / female), age (adult / child / elderly), entry method (walking in / vehicle entry), and role in the entry method (classifying passengers and drivers in the case of vehicle entry).

[0034] The images of the station including the monitored target are continuously collected by multiple cameras, and the continuous feature information of the monitored target is identified by a deep learning algorithm, that is, the movement of the monitored target is identified. Through the training of the target identifier, the accurate identification and basic information collection of the monitored target in the station can be increased, thereby achieving the effect of providing accurate data information for accurately analyzing the target movement trend and judging whether to carry out harmful behavior.

[0035] S20: According to the monitoring target, matching and acquiring the public equipment distribution information in the smart station, wherein the public equipment distribution information includes first public equipment distribution information, second public equipment distribution information and third public equipment distribution information;

[0036] Furthermore, according to the monitoring target, matching and acquiring the public equipment distribution information in the smart station, step S20 includes:

[0037] S21: According to the monitoring target, matching is performed in the station equipment library of the smart station to obtain first public equipment distribution information, wherein the station equipment library includes multiple index tables recording monitoring targets and multiple index tables recording first public equipment;

[0038] S22: Filter and obtain second public equipment distribution information in the smart station according to the first public equipment distribution information;

[0039] S23: Obtaining the third public equipment distribution information of the smart station;

[0040] S24: Obtain public device distribution information by combining the first public device distribution information, the second public device distribution information and the third public device distribution information.

[0041] Specifically, the monitoring target is every individual entering the station. Based on the relevant information of each individual, all public equipment in the smart station is divided into three categories according to the danger level. The first public equipment is the equipment that the individual is very likely to touch, such as the charging pile and parking lock next to his own parking space. The second public equipment is the equipment that the individual basically will not touch under normal circumstances, such as the locks on other people’s parking spaces. The third public equipment is the equipment that the individual is almost unlikely to have the motivation to actively touch under normal circumstances, mainly other public equipment that is being used and public equipment that is only allowed to be used under specific circumstances, such as charging piles and fire-fighting equipment that others are using. These three categories of public equipment are relative to an individual, that is, for different individuals, the division results of these three categories of equipment will be different.

[0042] All public devices are stored in the site equipment library of the smart site. The site equipment library includes multiple index tables that record monitoring targets and multiple index tables that record first public devices. The index table facilitates matching and calling device information. The first public device is a device that the individual is very likely to come into contact with. According to the location of the monitoring target, the corresponding first public device is matched, and the corresponding first public device distribution information is found according to the index table.

[0043] Matching is performed through the monitoring target, wherein, when the first public device distribution information is matched, other devices of the same type except the first public device in the site device library of the smart site are screened to obtain the second public device distribution information. That is, after the index number of the first public device is known, other devices except the index number of the first public device are screened in devices such as parking spaces and locks to obtain the second public device and obtain the second public device distribution information.

[0044] The third public equipment is a pre-set public equipment that is only allowed to be used under specific circumstances and a public equipment that is currently in use. The central control center and various public equipment set up in the smart station exchange communication data, obtain the index number of the public equipment in use in the smart station, call the index number of the public equipment allowed to be used under specific circumstances, and obtain the distribution information of the third public equipment.

[0045] The public device distribution information is obtained by combining the first public device distribution information, the second public device distribution information and the third public device distribution information, and is used for subsequent target motion behavior prediction and analysis.

[0046] S30: performing target trend prediction according to the continuous feature information and the public device distribution information to obtain a target trend prediction result, wherein the target trend prediction includes movement prediction and behavior prediction;

[0047] Further, the target trend prediction is performed according to the continuous feature information and the public device distribution information to obtain the target trend prediction result, and step S30 includes:

[0048] S31: Predicting the movement trend of the monitored target according to the continuous feature information to obtain a movement trend prediction result;

[0049] S32: judging, according to the public device distribution information and the mobile trend prediction result, whether the moving target of the mobile trend prediction result is the first public device, the second public device or the third public device;

[0050] S33: If not, continue monitoring the monitoring target;

[0051] S34: If yes, when the moving target of the moving trend prediction result is the first public device, predict the behavior trend of the monitored target according to the continuous feature information to obtain a behavior trend prediction result as a target trend prediction result;

[0052] S35: when the moving target of the moving trend prediction result is a second public device, calculate and obtain a second distance to the second public device according to the moving trend prediction result, and when the second distance is less than a second distance threshold, perform a behavior trend prediction of the monitored target according to the continuous feature information to obtain a behavior trend prediction result as a target trend prediction result;

[0053] S36: When the moving target of the moving trend prediction result is a third public device, a third distance to the third public device is calculated according to the moving trend prediction result as the target trend prediction result.

[0054] Specifically, the movement trend prediction is path prediction. The continuous feature information characterizes the movement of the monitored target, including a number of continuous movement trajectory points. According to the accumulation of continuous feature information over a certain period of time, the predicted movement trajectory of the monitored target can be marked, that is, the movement trend prediction result. An exemplary long short-term memory (LSTM) classification model can be set. The input of the model is the running trajectory points at the current time N and the time NL before the current time; the output of the model is the probability of multiple end points; in order to reduce the number of classifications, multiple classifiers (models) in series can be used to implement it. For example, the classification result of the first model is: the probabilities of four large areas, including the end point is not any public equipment, the end point is the first public equipment, the end point is the second public equipment, and the end point is the third public equipment; the classification result of the second model is the probability of small areas within the large areas; the classification result of the third model is the probability of each end point in the small area; finally, the probability values ​​of the top three probabilities output by the third model are obtained (corresponding to the three end points), and based on other individual related information (gender (male / female), age (adult / child / elderly), entry method (walking entry / vehicle entry), role in the entry method (this is mainly to classify passengers and drivers when vehicles enter), and actions performed in the station), the weighted values ​​of the three end points are calculated respectively, the three probability values ​​are weighted, and finally the end point with the highest probability is obtained as the output result, and the movement trend prediction result is obtained.

[0055] The positioning information of the final position of the mobile trend prediction result is compared with the positioning information of the public device distribution information to determine whether it is within the distance threshold range of the first public device, the second public device or the third public device. If it is not within the threshold range, it is considered that the monitoring of the monitoring target needs to be continued. If it is within the threshold range, it is considered that the moving target of the mobile trend prediction result is the first public device, the second public device or the third public device.

[0056] When the moving target of the mobile trend prediction result is the first public device, it is considered to be relatively safe and there is no need to judge the distance, but a behavioral safety judgment is required, that is, it is necessary to predict the behavioral trend of the monitored target based on the continuous feature information to obtain a target trend prediction result.

[0057] When the moving target of the mobile trend prediction result is the second public device, it means that the security is low. Based on the above logic, distance judgment is added. The central control center continuously judges the distance between the individual and the public device, that is, the second distance, based on the image returned by the camera. When the distance is lower than the threshold, that is, the second distance threshold, the behavior trend is predicted based on the continuous feature information to obtain the target trend prediction result.

[0058] When the moving target of the moving trend prediction result is a third public device, a third distance to the third public device is calculated according to the moving trend prediction result as the target trend prediction result.

[0059] Behavior trend prediction can be carried out by training the behavior trend predictor, and the target trend prediction results are used to generate the logic of monitoring management strategies. Since the target movement prediction and behavior prediction are carried out, the behavior of the target can be analyzed more comprehensively. At the same time, the classification of public equipment into three categories is essentially to set up monitoring management methods according to the importance of different equipment, which is more targeted.

[0060] S40: When the target trend prediction result reaches a preset condition, a corresponding monitoring management strategy is generated for management.

[0061] Furthermore, when the target trend prediction result reaches a preset condition, a corresponding monitoring management strategy is generated for management, and step S40 includes:

[0062] S41: when the moving target of the moving trend prediction result is the first public device, if the behavior trend prediction result is a preset behavior type, generating a first monitoring management strategy;

[0063] S42: when the moving target of the moving trend prediction result is the second public device, if the behavior trend prediction result is a preset behavior type, generating a second monitoring management strategy;

[0064] S43: when the moving target of the moving trend prediction result is a third public device, if the distance is less than a third distance threshold, generating a third monitoring management strategy;

[0065] S44: adopting the first monitoring management strategy, the second monitoring management strategy or the third monitoring management strategy for management.

[0066] Specifically, corresponding management strategies are designed according to different target trend prediction results. The preset behavior type is misoperation or destructive behavior on public equipment. When the moving target of the mobile trend prediction result is the first public equipment, the behavior is identified based on the image judgment and recognition sent back by the central control center. The preset behavior type includes a series of dangerous behaviors, such as misoperation or destructive behavior on public equipment. When the behavior trend prediction result is identified as the preset behavior type, a first monitoring management strategy is generated. The first monitoring management strategy includes sending relevant prompt information to security personnel to stop the individual's behavior in time, thereby achieving protection of the first type of public equipment.

[0067] When the moving target of the mobile trend prediction result is the second public equipment, if the behavior trend prediction result is a preset behavior type, that is, based on the image judgment and recognition sent back by the central control center, when it is identified that the public equipment is being operated incorrectly or destroyed, a second monitoring management strategy is generated, and relevant prompt information is sent to the security personnel to stop the individual's behavior in time to achieve the protection of the second type of public equipment. Exemplarily, if the target is identified to walk towards the charging pile that others are using, and the distance is lower than the threshold, the charging pile will respond to the command of the central control center to actively cut off the power to avoid the risk of electric shock to the target, and if the behavior is subsequently identified that the target forcibly pulls out the charging gun, it will issue a command to let the charging pile actively unlock the interlocking mechanism of the charging gun and the charging interface so that it can be pulled out smoothly, and avoid further violent behavior caused by failure to pull out that causes damage to the charging pile and the charging interface. Since this is a unilateral unlocking of the interlocking mechanism of the charging pile, according to the current electric vehicle charging alarm mechanism, the electric vehicle will determine that the charging status is abnormal and will send the same reminder message to the owner's mobile phone, which is equivalent to triggering a prompt to the owner, and the central control center can also send an alarm message to the owner based on the information retained by the electric vehicle, so as to achieve the effect of double prompting to the owner.

[0068] When the moving target of the moving trend prediction result is the third public device, the public device at the end point is locked, and the distance between the individual and the public device is continuously determined. If the distance is less than the third distance threshold, a third monitoring management strategy is generated, that is, the central control center immediately issues an instruction to the public device to control it to stop working first, so as to avoid the risk of electric shock caused by the individual performing dangerous operations on the working public device, and at the same time, the action recognition of the individual is started according to the image to determine whether the individual continues to perform harmful actions on the public device. If it is determined that the individual is performing harmful actions, the instruction is continued to be issued to control the public device to restore the initial state, so as to avoid the individual forcibly changing the state of the public device and damaging the public device. For example, assuming that the individual is identified to use fire-fighting equipment, if there is no fire, prompt information is sent to the relevant security personnel in time to stop the individual's behavior in time. If there is a specific situation, prompt information of the specific situation is sent to the security personnel, such as prompting the existence of a fire.

[0069] By adopting the first monitoring and management strategy, the second monitoring and management strategy or the third monitoring and management strategy for management, the three types of public equipment are divided into three categories, and corresponding protection mechanisms are set up for the three categories of public equipment in a refined manner. In particular, the distance judgment of the third type of public equipment can ensure that the misjudgment rate is reduced as much as possible, providing a more intelligent and comprehensive monitoring solution for smart station management and monitoring. Technical effect.

[0070] Further, the continuously collecting the site image including the monitored target and identifying the continuous feature information of the monitored target, step S14 includes:

[0071] S141: continuously collecting images of the station including the monitored target to obtain a target time-series image sequence;

[0072] S142: According to the monitoring data records in the smart station, a sample time-series image sequence set is collected, and the target continuous features in the sample time-series image sequence are identified and marked to obtain a sample continuous feature information set;

[0073] S143: Using the sample time-series image sequence set and the sample continuous feature information set, based on a time series convolutional neural network, training a continuous feature identifier, identifying the target time-series image sequence, and obtaining the continuous feature information.

[0074] Specifically, the continuous feature information includes the actions performed by the individual in the station and the movement path in the station. The station images including the monitored target are continuously collected with the monitored target as the center, that is, the image collection is continuously performed in the area where people are active. The collected images are arranged in chronological order to form the target time-series image sequence.

[0075] The sample time-series image sequence set is based on the historical monitoring data records in the smart station, and the monitoring image data covers various age groups, various ways of entering the station, and various roles (passengers and drivers). The target continuous features are identified and marked according to the behavioral actions of the characters. Exemplarily, the actions performed are marked and the trajectories of the actions are marked to obtain the sample continuous feature information set, which includes the feature images and identification information of the samples.

[0076] Temporal Convolutional Network (TCN) is a deep learning model for processing time series data. It has been widely used in speech recognition, natural language processing, action recognition and other fields. It is based on the idea of ​​convolutional neural network, extracts and learns features in time series data through convolution operations, and has achieved success in a series of time series prediction and classification tasks. The core of TCN is the use of dilated convolution. This convolution structure can increase the receptive field of the network without increasing the number of parameters, thereby effectively capturing long-term dependencies. In addition, TCN can also increase the representation ability of the model by stacking multiple layers of convolution, making it perform well in various time series tasks.

[0077] The sample time series image sequence set and the sample continuous feature information set are divided into 8:2, 80% of the data is set as the training set, and 20% of the data is set as the validation set. In order to train the feature identifier, the training set will consist of equal-sized subsequence pairs (input sequence, target sequence) of the given time series, that is, the sample time series image sequence set and the sample continuous feature information set must have the same shape. Using the sliding window method, many overlapping input and target sequences can create a time series. In order to evaluate the continuous feature identifier, a fixed time threshold range is set, and its performance is tested at many different time points in the validation set. The similarity is compared based on the real sample continuous feature information and the predicted continuous feature information. The closer the similarity is to 1, the better the model prediction effect is. When the similarity analysis result meets the preset threshold, the feature identifier training is completed.

[0078] The target time-series image sequence is identified using a feature identifier, that is, the target time-series image is input into the feature identifier, and the continuous feature information is output. Based on image detection, path prediction is combined to realize station monitoring, providing data support for scientific prediction of individual travel routes and individual behavior.

[0079] Further, the behavior trend prediction of the monitored target is performed according to the continuous feature information, step S34 includes:

[0080] S341: Collecting sample continuous feature information set and sample behavior trend prediction result set;

[0081] S342: training a behavior trend predictor using the sample continuous feature information set and the sample behavior trend prediction result set;

[0082] S343: Using the behavior trend predictor to identify the continuous feature information, and obtain a behavior trend prediction result.

[0083] Specifically: the sample continuous feature information set and the sample behavior trend prediction result set are historical data information collected based on the monitoring data records in the station. The sample behavior trend prediction result set is the individual behavior in several types of smart stations. The sample continuous feature information set is used as input information, the sample continuous features at the current moment N and the NL moments before the current moment are converted into spatial trajectory points, and the sample behavior trend prediction result set is used as the target output information.

[0084] The behavior trend predictor can be exemplarily a long short-term memory network (Long Short-Term Memory, LSTM), which can effectively handle long-term dependencies. In behavior prediction, LSTM can better capture complex behavior patterns and dynamic changes, thereby improving prediction performance. The sample continuous feature information set and the sample behavior trend prediction result set are input into the behavior trend predictor for training. When the number of training times reaches the set number threshold, the training ends. The continuous feature information is input into the behavior trend predictor to obtain the trend prediction result, which is a certain type of behavior in the preset behavior type. The behavior trend predictor can identify the continuous feature information and predict the behavior trend of the individual, which can lay the foundation for judging the safety factor of individual behavior and matching the monitoring management strategy.

[0085] Example 2

[0086] like Figure 2 As shown, based on the same inventive concept as the monitoring method in a smart station provided in Example 1, an embodiment of the present invention provides a monitoring device in a smart station, including:

[0087] The feature information acquisition module 100 is used to collect images in the smart station, identify and determine the monitoring target, and continuously collect and identify the continuous feature information of the monitoring target;

[0088] The distribution information matching module 200 is used to match and obtain the public equipment distribution information in the smart station according to the monitoring target, wherein the public equipment distribution information includes the first public equipment distribution information, the second public equipment distribution information and the third public equipment distribution information;

[0089] The target trend prediction module 300 is used to perform target trend prediction based on the continuous feature information and the public device distribution information to obtain a target trend prediction result, wherein the target trend prediction includes movement prediction and behavior prediction;

[0090] The management strategy generation module 400 is used to generate a corresponding monitoring management strategy for management when the target trend prediction result reaches a preset condition.

[0091] Furthermore, the device also includes:

[0092] The sample image processing module is used to collect a sample station image set based on the monitoring data records in the smart station, and to identify and mark the targets in the sample station image to obtain a sample monitoring target set;

[0093] A first identifier training module is used to train a target identifier based on a convolutional neural network using the sample station image set and the sample monitoring target set;

[0094] A target recognition module is used to collect images of the smart station, input them into the target identifier for target recognition, and obtain the monitoring target and basic target information;

[0095] The feature recognition module is used to continuously collect site images including the monitored target and recognize continuous feature information of the monitored target.

[0096] Furthermore, the device also includes:

[0097] A time-series image acquisition module, used for continuously acquiring images of the site including the monitored target, and obtaining a target time-series image sequence;

[0098] The continuous feature acquisition module is used to collect a sample time-series image sequence set based on the monitoring data records in the smart station, and to identify and mark the target continuous features in the sample time-series image sequence to obtain a sample continuous feature information set;

[0099] The second identifier training module is used to use the sample time-series image sequence set and the sample continuous feature information set to train a continuous feature identifier based on a time series convolutional neural network to identify the target time-series image sequence and obtain the continuous feature information.

[0100] Furthermore, the device also includes:

[0101] A first device matching module is used to match the monitoring target in the site equipment library of the smart site to obtain the first public device distribution information, wherein the site equipment library includes a plurality of index tables recording the monitoring targets and a plurality of index tables recording the first public devices;

[0102] A second device matching module is used to filter and obtain second public device distribution information in the smart site according to the first public device distribution information;

[0103] A third device matching module, used to obtain the third public device distribution information of the smart station;

[0104] The information combination module is used to combine the first public device distribution information, the second public device distribution information and the third public device distribution information to obtain public device distribution information.

[0105] Furthermore, the device also includes:

[0106] A movement trend prediction module, used to predict the movement trend of the monitored target according to the continuous feature information to obtain a movement trend prediction result;

[0107] A first prediction result judgment module, used for judging whether the moving target of the moving trend prediction result is the first public device, the second public device or the third public device according to the public device distribution information and the moving trend prediction result;

[0108] A continue monitoring module is used to continue monitoring the monitoring target if no;

[0109] A first prediction result generating module is used for, if the moving target of the moving trend prediction result is the first public device, predicting the behavior trend of the monitored target according to the continuous feature information, and obtaining the behavior trend prediction result as the target trend prediction result;

[0110] A second prediction result generating module is used for, when the moving target of the moving trend prediction result is a second public device, calculating a second distance to the second public device according to the moving trend prediction result, and when the second distance is less than a second distance threshold, performing a behavior trend prediction of the monitored target according to the continuous feature information to obtain a behavior trend prediction result as a target trend prediction result;

[0111] The third prediction result generating module is used for, when the moving target of the moving trend prediction result is a third public device, calculating and obtaining a third distance to the third public device according to the moving trend prediction result as a target trend prediction result.

[0112] Furthermore, the device also includes:

[0113] The sample information collection module is used to collect the sample continuous feature information set and the sample behavior trend prediction result set;

[0114] A predictor training module, used to train a behavior trend predictor using the sample continuous feature information set and the sample behavior trend prediction result set;

[0115] The behavior prediction result generating module is used to adopt the behavior trend predictor to identify the continuous feature information and obtain the behavior trend prediction result.

[0116] Furthermore, the device also includes:

[0117] A first management strategy generating module, configured to generate a first monitoring management strategy when the moving target of the moving trend prediction result is the first public device and if the behavior trend prediction result is a preset behavior type;

[0118] A second management strategy generating module, configured to generate a second monitoring management strategy when the moving target of the moving trend prediction result is a second public device and if the behavior trend prediction result is a preset behavior type;

[0119] A third management strategy generating module, configured to generate a third monitoring management strategy when the moving target of the moving trend prediction result is a third public device and if the distance is less than a third distance threshold;

[0120] The monitoring management module is used to adopt the first monitoring management strategy, the second monitoring management strategy or the third monitoring management strategy for management.

[0121] See also Figure 3 , Figure 3 Schematic diagram of an electronic device provided by an embodiment of the present invention. Figure 3 As shown, an embodiment of the present invention further proposes an electronic device 500, including a memory 510, a processor 520, and a computer program 511 stored in the memory 510 and executable on the processor 520, wherein the processor 520 implements the following steps when executing the computer program 511: collecting images within the smart station, identifying and determining monitoring targets, and continuously collecting and identifying continuous feature information of the monitoring targets; matching and obtaining public equipment distribution information within the smart station according to the monitoring targets, wherein the public equipment distribution information includes first public equipment distribution information, second public equipment distribution information, and third public equipment distribution information; performing target trend prediction according to the continuous feature information and the public equipment distribution information, and obtaining a target trend prediction result, wherein the target trend prediction includes movement prediction and behavior prediction; when the target trend prediction result meets a preset condition, generating a corresponding monitoring management strategy for management.

[0122] See also Figure 4 , Figure 4 A schematic diagram of an embodiment of a computer-readable storage medium provided in an embodiment of the present invention. Figure 4 As shown, this embodiment provides a computer-readable storage medium 600, on which a computer program 611 is stored. When the computer program 611 is executed by a processor, the following steps are implemented: collecting images within the smart station, identifying and determining the monitoring target, and continuously collecting and identifying continuous feature information of the monitoring target; according to the monitoring target, matching and obtaining public equipment distribution information within the smart station, wherein the public equipment distribution information includes first public equipment distribution information, second public equipment distribution information and third public equipment distribution information; according to the continuous feature information and the public equipment distribution information, performing target trend prediction and obtaining a target trend prediction result, wherein the target trend prediction includes movement prediction and behavior prediction; when the target trend prediction result meets a preset condition, generating a corresponding monitoring management strategy for management.

[0123] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0124] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0125] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0126] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0127] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0128] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0129] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A monitoring method in a smart station, characterized in that: The method comprises: Collect images within the smart station, identify and determine the monitoring target, and continuously collect and identify the continuous feature information of the monitoring target; According to the monitoring target, matching and acquiring the public equipment distribution information in the smart station, wherein the public equipment distribution information includes first public equipment distribution information, second public equipment distribution information and third public equipment distribution information; According to the continuous feature information and the public device distribution information, target trend prediction is performed to obtain a target trend prediction result, wherein the target trend prediction includes movement prediction and behavior prediction; When the target trend prediction result reaches the preset condition, a corresponding monitoring management strategy is generated for management; Wherein, according to the monitoring target, matching and obtaining the public equipment distribution information in the smart station includes: According to the monitoring target, matching is performed in the station equipment library of the smart station to obtain the first public equipment distribution information, wherein the station equipment library includes multiple index tables recording monitoring targets and multiple index tables recording first public equipment; According to the first public equipment distribution information, second public equipment distribution information is obtained by screening in the smart site; Obtaining third public equipment distribution information of the smart site; Combining the first public device distribution information, the second public device distribution information and the third public device distribution information to obtain public device distribution information; Wherein, performing target trend prediction according to the continuous feature information and the public equipment distribution information to obtain the target trend prediction result includes: Predicting the movement trend of the monitored target according to the continuous feature information to obtain a movement trend prediction result; According to the public device distribution information and the mobile trend prediction result, determining whether the moving target of the mobile trend prediction result is the first public device, the second public device or the third public device; If not, continue monitoring the monitoring target; If so, when the moving target of the moving trend prediction result is the first public device, predicting the behavior trend of the monitored target according to the continuous feature information, and obtaining a behavior trend prediction result as a target trend prediction result; When the moving target of the moving trend prediction result is a second public device, a second distance to the second public device is calculated according to the moving trend prediction result, and when the second distance is less than a second distance threshold, a behavior trend prediction of the monitored target is performed according to the continuous feature information to obtain a behavior trend prediction result as a target trend prediction result; When the moving target of the moving trend prediction result is a third public device, a third distance to the third public device is calculated according to the moving trend prediction result as the target trend prediction result.

2. The method according to claim 1, characterized in that Determine the monitoring target and continuously collect and identify the continuous feature information of the monitoring target, including: According to the monitoring data records in the smart station, a sample station image set is collected, and the targets in the sample station image are identified and marked to obtain a sample monitoring target set; Using the sample station image set and the sample monitoring target set, training a target identifier based on a convolutional neural network; Collecting images of the smart station, inputting them into the target identifier for target identification, and obtaining the monitoring target and basic target information; Continuously collect site images including the monitored target, and identify continuous feature information of the monitored target.

3. The method according to claim 2, characterized in that Continuously collecting images of the site including the monitored target and identifying continuous feature information of the monitored target, including: Continuously collecting station images including the monitored target to obtain a target time-series image sequence; According to the monitoring data records in the smart station, a sample time-series image sequence set is collected, and the target continuous features in the sample time-series image sequence are identified and marked to obtain a sample continuous feature information set; The sample time-series image sequence set and the sample continuous feature information set are used to train a continuous feature identifier based on a time series convolutional neural network to identify the target time-series image sequence and obtain the continuous feature information.

4. The method according to claim 1, characterized in that: Predicting the behavior trend of the monitored target according to the continuous feature information includes: Collect sample continuous feature information set and sample behavior trend prediction result set; Using the sample continuous feature information set and the sample behavior trend prediction result set to train a behavior trend predictor; The behavior trend predictor is used to identify the continuous feature information to obtain a behavior trend prediction result.

5. The method according to claim 1, characterized in that When the target trend prediction result reaches the preset condition, a corresponding monitoring management strategy is generated for management, including: When the moving target of the moving trend prediction result is the first public device, if the behavior trend prediction result is a preset behavior type, generating a first monitoring management strategy; When the moving target of the moving trend prediction result is the second public device, if the behavior trend prediction result is a preset behavior type, generating a second monitoring management strategy; When the moving target of the moving trend prediction result is a third public device, if the distance is less than a third distance threshold, generating a third monitoring management strategy; The first monitoring management strategy, the second monitoring management strategy or the third monitoring management strategy is adopted for management.

6. A monitoring device in a smart station, characterized in that: include: The feature information acquisition module is used to collect images in the smart station, identify and determine the monitoring target, and continuously collect and identify the continuous feature information of the monitoring target; A distribution information matching module, used to match and obtain the public equipment distribution information in the smart station according to the monitoring target, wherein the public equipment distribution information includes first public equipment distribution information, second public equipment distribution information and third public equipment distribution information; A target trend prediction module, used to perform target trend prediction according to the continuous feature information and the public device distribution information, and obtain a target trend prediction result, wherein the target trend prediction includes movement prediction and behavior prediction; A management strategy generation module is used to generate a corresponding monitoring management strategy for management when the target trend prediction result reaches a preset condition; Wherein, according to the monitoring target, matching and obtaining the public equipment distribution information in the smart station includes: According to the monitoring target, matching is performed in the station equipment library of the smart station to obtain the first public equipment distribution information, wherein the station equipment library includes multiple index tables recording monitoring targets and multiple index tables recording first public equipment; According to the first public equipment distribution information, second public equipment distribution information is obtained by screening in the smart site; Obtaining third public equipment distribution information of the smart site; Combining the first public device distribution information, the second public device distribution information and the third public device distribution information to obtain public device distribution information; Wherein, performing target trend prediction according to the continuous feature information and the public equipment distribution information to obtain the target trend prediction result includes: Predicting the movement trend of the monitored target according to the continuous feature information to obtain a movement trend prediction result; According to the public device distribution information and the mobile trend prediction result, determining whether the moving target of the mobile trend prediction result is the first public device, the second public device or the third public device; If not, continue monitoring the monitoring target; If so, when the moving target of the moving trend prediction result is the first public device, predicting the behavior trend of the monitored target according to the continuous feature information, and obtaining a behavior trend prediction result as a target trend prediction result; When the moving target of the moving trend prediction result is a second public device, a second distance to the second public device is calculated according to the moving trend prediction result, and when the second distance is less than a second distance threshold, a behavior trend prediction of the monitored target is performed according to the continuous feature information to obtain a behavior trend prediction result as a target trend prediction result; When the moving target of the moving trend prediction result is a third public device, a third distance to the third public device is calculated according to the moving trend prediction result as the target trend prediction result.

7. An electronic device, characterized in that: include: Memory for storing computer software programs; A processor is used to read and execute the computer software program, thereby implementing a monitoring method within a smart station as described in any one of claims 1-5.

8. A non-transitory computer-readable storage medium, characterized in that: The storage medium stores a computer software program, which, when executed by a processor, implements a monitoring method within a smart station as described in any one of claims 1 to 5.

Images