Target behavior prediction method, device, electronic device and storage medium based on video surveillance

Abstract

The present application belongs to the field of video surveillance technology, and in particular, relates to a method, device, electronic device and storage medium for predicting target behavior based on video surveillance. The present application obtains surveillance video information; obtains the coordinate information of the target based on the surveillance video information; when the coordinate information of the target is within the warning range, obtains the image information of the target based on the surveillance video information; and determines the target intention based on the image information of the target and a well-established prediction model. The scheme described in the present application, when the coordinate information of the target is within the warning range, determines the target intention through the prediction model, thereby reducing the possibility of misjudgment; in the process of predicting the target intention, prediction is performed through multiple continuous behavior images of the target, thereby improving the accuracy of the prediction result.

Description

Technical Field

[0001] The present application belongs to the field of video surveillance technology, and in particular relates to a method, device, electronic device and storage medium for predicting target behavior based on video surveillance. Background Art

[0002] With the development of society, video surveillance cameras are deployed in various locations in public places. The purpose is to be able to conduct 24-hour uninterrupted real-time monitoring of public places, detect abnormal situations in time, and after an emergency occurs, the monitoring system can provide first-hand information to help relevant departments handle the incident.

[0003] However, video surveillance always has some lag in handling emergency dangerous situations. That is, after the danger occurs, the entire behavior process of the target person in the video can be obtained based on video surveillance in public places. For example, when the target person commits suicide by jumping off a bridge or a building, it is difficult to predict the target behavior during the process of the target person's suicide behavior and promptly notify relevant personnel to stop the target behavior, thereby improving public safety.

[0004] Therefore, how to accurately predict the target behavior in video surveillance in real time to improve public social security has become an urgent problem to be solved. Summary of the invention

[0005] The present application provides a method, device, electronic device and storage medium for predicting target behavior based on video surveillance, which can accurately predict the target behavior in video surveillance in real time and effectively, thereby improving public social security.

[0006] In a first aspect, an embodiment of the present application provides a target behavior prediction method based on video surveillance, the method comprising: obtaining surveillance video information, the surveillance video information comprising the relative distance and relative angle between the target and the video shooting point; obtaining the coordinate information of the target based on the surveillance video information; when the coordinate information of the target is within the warning range, obtaining the image information of the target based on the surveillance video information; determining the target intention based on the image information of the target and a constructed prediction model, the prediction model being created by a convolutional neural network.

[0007] Preferably, obtaining the coordinate information of the target based on the surveillance video information includes: determining the first coordinate position information of the target based on the relative distance and relative angle, a first preset coordinate conversion matrix and an error matrix; determining the coordinate information of the target based on the first coordinate position information and a second coordinate conversion matrix.

[0008] Preferably, when the coordinate information of the target is within the warning range, the target image information is obtained based on the monitoring video information, including: when the time that the coordinate information of the target is within the warning range reaches a preset threshold, multiple continuous behavior image information of the target within a preset time interval is obtained based on the monitoring video information; accordingly, the target intention is determined based on the image information of the target and the prediction model that has been constructed, including: determining the target intention based on the multiple continuous behavior image information and the prediction model that has been constructed.

[0009] Preferably, constructing the prediction model includes: constructing the prediction model includes: obtaining target image information in historical monitoring video information, the target image information including multiple continuous behavior images of the target within the warning range; based on the multiple continuous behavior images, determining an image group, the image group including at least two continuous behavior images; based on the image group, determining a preprocessed image group through preprocessing; based on the preprocessed image group and a convolutional neural network, determining a prediction model, the convolutional neural network including a convolutional layer, a weight layer, a downsampling layer and a fully connected layer.

[0010] Preferably, the prediction model is determined based on the preprocessed image group and the convolutional neural network, including: based on the continuous behavior images in the preprocessed image group, determining the feature map of the continuous behavior image through the convolution layer, the feature map of the continuous behavior image includes a first behavior feature map and a second behavior feature map; based on the feature map of the continuous behavior image, determining the combined weight map of the image group through the weight layer; based on the combined weight map of the image group, determining a simplified feature map through the downsampling layer, the size of the simplified feature map is smaller than the size of the combined weight map; based on the simplified feature map, determining the prediction model through the fully connected layer.

[0011] Preferably, the feature map based on the continuous behavior image determines the combined weight map of the image group through the weight layer, including: determining channel information through pooling processing based on the feature map of the continuous image; determining the one-dimensional convolution kernel size of the channel information based on the channel information and a preset formula; applying the one-dimensional convolution kernel size to the one-dimensional convolution to determine the first weight of the channel information; determining the second weight of the channel information based on the first weight and the swish activation function; determining the weighted feature map of the feature map based on the second weight and the feature map, the weighted feature map including a first behavior weighted feature map and a second behavior weighted feature map; determining the combined weight map of the image group based on the weighted feature map and the weight function.

[0012] Preferably, the preset formula is:

[0013]

[0014] Where k represents the size of the one-dimensional convolution kernel, C is the number of channels, and γ and b are preset parameters.

[0015] In the second aspect, an embodiment of the present application provides a target behavior prediction device based on video surveillance, the device comprising: an acquisition module for acquiring surveillance video information; a processing module for acquiring coordinate information of the target based on the surveillance video information; the processing module is also used to acquire image information of the target based on the surveillance video information when the coordinate information of the target is within the warning range; a determination module for determining the target intention based on the image information of the target and a constructed prediction model, the prediction model being created by a convolutional neural network.

[0016] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method described in the first aspect or any one of the implementation methods thereof is implemented.

[0017] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect or any one of the implementation methods thereof is implemented.

[0018] In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product runs on an electronic device, the electronic device executes the method described in the first aspect or any one of the implementation methods thereof.

[0019] The present application obtains monitoring video information; obtains the coordinate information of the target based on the monitoring video information; when the coordinate information of the target is within the warning range, obtains the image information of the target based on the monitoring video information; and determines the target intention based on the image information of the target and the prediction model that has been constructed. Compared with the prior art, video monitoring only records the behavior process of the target, and it is difficult to predict the target behavior, and it is not possible to notify relevant personnel to stop the target's dangerous behavior in time based on the prediction results. The present application obtains the actual coordinate information of the target through video monitoring information, and when the coordinate information of the target is within the warning range, predicts the target behavior, and notifies relevant personnel to stop the target's dangerous behavior in time based on the prediction results, thereby improving public social safety. Based on multiple continuous behavior images and prediction models of the target within the warning range, and in the process of model training, a weight layer is introduced to enhance the model's recognition of features and improve the accuracy of the prediction results. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0021] Figure 1 A flowchart of a method for predicting target behavior based on video surveillance provided in one embodiment of the present application;

[0022] Figure 2 A schematic diagram of coordinate information conversion provided in an embodiment of the present application;

[0023] Figure 3 A flowchart of a prediction model creation method provided in an embodiment of the present application;

[0024] Figure 4 Provide a graph of the change in F1 score of the prediction model at different probability thresholds for this application;

[0025] Figure 5 A structural block diagram of a target behavior prediction device based on video surveillance provided in one embodiment of the present application;

[0026] Figure 6 A schematic diagram of the structure of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0027] The following description is for a thorough understanding of the embodiments of the present application; therefore, the present application provides a target behavior prediction method, device, electronic device and storage medium based on video surveillance to solve the above problems.

[0028] Figure 1 A flowchart of a method for predicting target behavior based on video surveillance is provided in accordance with an embodiment of the present application. The method includes:

[0029] S110, acquiring monitoring video information, where the monitoring video information includes a relative distance and a relative angle between a target and a video shooting point.

[0030] Exemplarily, according to the image content of the surveillance video, surveillance video information is obtained, and the video surveillance information includes the motion trajectory of the target in the video, the action of the target and the direction of the target.

[0031] Furthermore, the camera that shoots surveillance video can be a binocular camera. The binocular camera observes the same target simultaneously through two cameras, and obtains accurate distance information, that is, the relative distance between the video shooting point and the target by analyzing the difference between the two images. Specifically, the distance between the target and the camera can be estimated by calculating the ratio of the pixel size of the target in the video to its actual size and combining the camera parameters. Then, the relative angle between the video shooting point and the target can be determined by analyzing the contour changes of the target in the image and the parallax information of the binocular camera.

[0032] S120, obtaining coordinate information of the target based on the monitoring video information.

[0033] In a possible implementation method, the coordinate information of the target is obtained based on the monitoring video information, including: determining the first coordinate position information of the target based on the relative distance and relative angle, the first preset coordinate conversion matrix and the error matrix; determining the coordinate information of the target based on the first coordinate position information and the second coordinate conversion matrix.

[0034] Exemplarily, the position information of the target in the first coordinate system, that is, the position information in the video coordinate system, is determined based on the relative distance and relative angle of the acquired video shooting points, the first preset coordinate transformation matrix and the error matrix. The calculation formula for the position information in the first coordinate system is: in, is the position information of the target in the first coordinate system, is the relative distance and relative angle of the target, is the first coordinate transformation matrix, R δ is the preset error matrix.

[0035] Further, based on the first coordinate position information and the second coordinate transformation matrix, the coordinate information of the target is determined. Based on the position information of the first coordinate system and the second coordinate transformation matrix, the coordinate information of the target is determined by the second calculation formula. The coordinate information of the target is the actual position information of the target in the monitoring area. The second calculation formula is in, is the target coordinate information, is the position information of the target in the first coordinate system, is the second coordinate transformation matrix. Converting the relative distance and relative angle between the target and the video shooting point into the coordinate information of the target actually converts the position information of the target in the video coordinate system into the specific position information of the life coordinate system. The life coordinate system is the actual position information in the video monitoring area.

[0036] Specifically, Figure 2 A schematic diagram of coordinate information conversion provided by an embodiment of the present application, according to the relative distance and relative angle of the target, Figure 2The relative distance and relative angle of the target shown in (a) are 15 meters and 30°. Through coordinate transformation, we can determine Figure 2 The actual position information of the target in (b) is (13,7.5).

[0037] S130, when the coordinate information of the target is within the warning range, acquiring image information of the target based on the monitoring video information.

[0038] For example, Figure 2 As shown in the figure, the living coordinate system in the monitoring area includes three ranges, namely safety range, warning range and alarm range.

[0039] In a possible implementation method, when the coordinate information of the target is within the warning range, the target image information is obtained based on the monitoring video information, including: when the time that the coordinate information of the target is within the warning range reaches a preset threshold, multiple continuous image information of the target within a preset time interval is obtained based on the monitoring video information; accordingly, based on the image information of the target and a well-constructed prediction model, the target intention is determined, including: the target intention is determined based on multiple continuous image information and the well-constructed prediction model.

[0040] Exemplarily, in order to prevent erroneous predictions, the present application sets a preset threshold. When the target is within the warning range and reaches the preset threshold, the prediction mode is started. The threshold set in the present application is 1 minute, that is, when the target stays within the warning range for 1 minute, the camera starts the prediction mode.

[0041] Furthermore, multiple continuous image information of the target within a preset interval is obtained based on the monitoring video information. The preset interval in the present application is set to 5s, and the image information of the target is obtained every 5s, wherein the image information is the specific behavior information of the target.

[0042] S140, determining the target intention based on the image information of the target and the constructed prediction model, the prediction model is created by a convolutional neural network.

[0043] Exemplarily, based on the acquired information of multiple consecutive images, each n consecutive images are grouped as a group (where n is a positive integer), and the constructed prediction model is inputted in groups to determine the target intent, that is, to determine the prediction result.

[0044] Preferably, when the prediction result is "dangerous behavior", the system directly notifies the management personnel to handle it. When the prediction result is "safe behavior", if the target is still within the warning range, the target's behavior continues to be predicted.

[0045] In another possible implementation, when the target is predicted to be an "emergency behavior", the target is in the "alarm range", that is, the target has exceeded the warning range and has most likely taken dangerous actions. The system immediately notifies the management personnel and performs an alarm process.

[0046] Before determining the target intention based on the target image information and the already constructed prediction model, a prediction model needs to be constructed. Figure 3 A flowchart of a prediction model creation method provided in an embodiment of the present application includes:

[0047] S310, obtaining target image information in historical monitoring video information, where the target image information includes a plurality of continuous behavior images of the target within the warning range.

[0048] Exemplarily, in order to ensure the accuracy of model training, target image information within the warning range in historical surveillance videos is obtained, and the target image information includes multiple continuous behavior images.

[0049] S320: Determine an image group based on the plurality of continuous behavior images, where the image group includes at least two continuous behavior images.

[0050] According to the multiple continuous behaviors in the target image information, the multiple continuous behavior images are grouped into groups of two each to determine the image group, which can be divided into multiple groups in total.

[0051] S330, determining a pre-processed image group based on the image group through pre-processing.

[0052] Exemplarily, each image group is preprocessed to determine the preprocessed image group. First, multiple continuous behavior images in each image group are Gaussian filtered to reduce image noise and make the curve smoother; then the Sobel operator is used to detect the contour edge of the target behavior in the image, so as to better capture the shape of the target behavior; the behavior images in each image group are normalized to facilitate gradient descent and model convergence. After the above processing, the preprocessed image group can be determined.

[0053] S340, based on the continuous behavior images in the preprocessed image group, determine a feature map of the continuous behavior images through a convolution layer, where the feature map of the continuous behavior images includes a first behavior feature map and a second behavior feature map.

[0054] The continuous images in the preprocessed image group are first converted into feature maps through a convolution layer with a convolution kernel size of 4×4 and a stride of 1 to extract preliminary features and keep the spatial dimension of the input data unchanged, only increasing the number of channels.

[0055] S350, based on the feature map of the continuous behavior image, determine the combined weight map of the image group through the weight layer.

[0056] Preferably, based on the feature map of the continuous behavior image, a combined weight map of the image group is determined through a weight layer, including: based on the feature map of the continuous behavior image, channel information is determined through pooling processing; based on the channel information and a preset formula, a one-dimensional convolution kernel size of the channel information is determined; the one-dimensional convolution kernel size is applied to the one-dimensional convolution to determine the first weight of the channel information; based on the first weight and the swish activation function, the second weight of the channel information is determined; based on the second weight and the feature map, a weighted feature map of the feature map is determined, the weighted feature map includes a first behavior weighted feature map and a second behavior weighted feature map; based on the weighted feature map and the weight function, a combined weight map of the image group is determined.

[0057] Specifically, the input feature map is subjected to global average pooling, and the feature map is transformed from a matrix of (h, w, c) to a vector of (1, 1, c), retaining only the channel information.

[0058] Specifically, based on the channel information and the preset formula, the size of the one-dimensional convolution kernel of the channel information is determined. Since the size of the one-dimensional convolution kernel is proportional to the channel dimension C, the size of the adaptive one-dimensional convolution kernel can be calculated by the following formula. The preset formula is:

[0059]

[0060] Where k represents the size of the one-dimensional convolution kernel, C is the number of channels, γ and b are preset parameters, which can be set to 2 and 1 respectively in this application.

[0061] Furthermore, the one-dimensional convolution kernel size is applied to the one-dimensional convolution to obtain the weight of each channel of the feature map, that is, the first weight, and the swish activation function is introduced for further processing to determine the second weight. The processed weight is multiplied by the original input feature map channel by channel to achieve weighting of the feature map and obtain a weighted feature map, that is, the first line and the second line of the image group are weighted feature maps.

[0062] After obtaining the weighted feature map of each continuous behavior image, in order to obtain the combined weight map of the two photos in each group, it is necessary to analyze the various features of the two continuous behavior photos in each group. Specifically, it includes: according to the weighted feature map of the first behavior of the target, the first behavior weighted feature map is the weighted feature map corresponding to the first photo in this group, the second behavior weighted feature map, the second behavior weighted feature map is the weighted feature map corresponding to the second photo in this group, and the weighted features and weight functions corresponding to the first behavior weighted feature map and the second behavior weighted feature map, determine the combined weight map.

[0063] When determining the combined weight map, the smaller the feature changes of the two weighted feature maps in the group, the lower the impact on the combined weight; and a sudden feature change cannot have a large impact on the weighted value. Therefore, using an exponential decay weight function can meet the requirements well.

[0064] As an example, the steps to solve the weight function are as follows:

[0065] In order to avoid data mutations that may have a significant impact on the weighted value, in this embodiment, the weight of the feature that changes suddenly accounts for no more than 1 / 3 of the total weight, that is:

[0066]

[0067] in, It represents the weight of the nth feature, and α is the weight parameter.

[0068] S360, based on the combined weight map of the image group, determine a simplified feature map through a downsampling layer, wherein the size of the simplified feature map is smaller than the size of the combined weight map.

[0069] Specifically, the combined weight map can reduce the size of each feature map by downsampling, thereby reducing the number of parameters and calculations in subsequent layers, thereby speeding up model training. Downsampling helps the model capture a wider range of feature information and helps extract a more global feature representation.

[0070] S370, based on the simplified feature graph, a prediction model is determined through a fully connected layer.

[0071] Preferably, after 4 cycles through the downsampling layer and the weight layer, it is flattened and sent to two fully connected layers. The number of neurons in the last layer is 2, corresponding to two output categories (dangerous behavior, safe behavior), and the softmax activation function is used to output the predicted probability of each category, thereby determining the model.

[0072] Specifically, after a series of downsampling processes, the data is flattened into a one-dimensional feature vector. It is then processed through two layers of fully connected layers to strengthen the connection between the features to achieve the final classification task. The swish activation function is used after the first fully connected layer to increase the nonlinearity of the features and help the model better capture complex feature relationships. The output of the second fully connected layer is mapped to two neurons, each corresponding to a category, namely: dangerous behavior and safe behavior. In order to ensure that the sum of the probabilities output by the model is 1, thereby forming a valid probability distribution, the last fully connected layer uses a softmax activation function. In this way, the model is able to provide a probability estimate for each category, thereby accurately classifying the target image information.

[0073] Preferably, a loss function can be further set for the model, using a cross entropy loss function CrossEntropyLoss, which is expressed as follows:

[0074]

[0075] In the formula, k represents the number of categories, m represents the number of samples, and y ij represents the sample label value, Represents the sample prediction value.

[0076] The cross entropy loss function works together with softmax in multi-classification problems. The softmax function is responsible for converting the output of the model into a probability distribution, while the cross entropy loss function measures the consistency of this probability distribution with the true label. The cross entropy loss function calculates the difference between the output probability and the target probability (i.e., the true label). During the model creation process, when the model's predicted value differs greatly from the actual value, the loss function will produce a larger value, which will cause the network parameters to be updated more quickly during the gradient back propagation process; on the contrary, when the predicted value is close to the actual value, the value of the loss function is smaller, and the speed of parameter update will also slow down accordingly. This helps the model converge to a more accurate prediction.

[0077] Furthermore, in order to ensure the accuracy of the model, this application uses four key indicators, namely accuracy, recall rate, precision rate, and F1 score, to evaluate the training effect, so as to further determine the threshold of model judgment.

[0078] Its expression is as follows:

[0079]

[0080] In the formula, Accuracy represents accuracy, Recall represents recall rate, Precision represents precision, F1-score represents F1 score, TP represents true positive, FP represents false positive, FN represents false negative, and TN represents true negative.

[0081] When the model probability threshold is set high, the model will predict an event only when it is very confident, which increases the possibility of detecting real events, thereby reducing the false positive rate and improving the precision rate. However, this may also cause some real events to be missed, that is, the false negative rate increases, resulting in a decrease in the recall rate. Conversely, when the probability threshold is set low, the model is more likely to predict events as positive, which reduces the false negative rate and improves the recall rate, but may also lead to more false positives and reduce the precision rate.

[0082] Therefore, in order to reconcile the precision and recall rates and comprehensively improve the accuracy of the model, the threshold is determined by the F1 score, because the F1 score is the harmonic average of the recall rate and the precision rate, taking both indicators into consideration at the same time.

[0083] Figure 4 Provide a graph of the change in F1 score of the prediction model at different probability thresholds for this application. Figure 4 (a) is a dangerous act, Figure 4 (b) is a safe behavior. It can be seen that when the probability threshold is 0.5, the F1 score reaches the highest, indicating that the model has achieved the best balance between recall rate and precision rate. The prediction result is considered valid only when the probability predicted by the model is higher than this threshold. This method can effectively reduce false positives and improve the reliability and accuracy of the model in practical applications. Thus, the best comprehensive detection effect can be obtained. Therefore, in this application, the threshold for the prediction model can be set to 0.5.

[0084] The technical solution provided by the present application converts the relative distance and relative angle between the target and the video shooting point into the actual coordinate information of the target based on the video surveillance information, and judges whether the target in the video is in the warning area based on the coordinate information. When the video reaches a preset time in the warning area, the target image information is obtained, and the prediction model is started. According to the image information and the prediction model, the target intention is determined. Compared with the monitoring video in the prior art that only records the process of the target's behavior, the present application can judge the target's intention based on the target's behavior, and when the judgment result is a dangerous behavior, the relevant personnel are notified in time to deal with it. At the same time, in the process of model creation, the present application combines multiple continuous photos of user behavior into a group for training, and then obtains a weighted feature map based on the feature map through two weighted processes, and then determines the prediction model based on the weighted feature map. Compared with the traditional neural network model, the accuracy of the prediction model is improved.

[0085] Figure 5 This is a structural block diagram of a target behavior prediction device based on video surveillance provided by an embodiment of the present application. For the sake of convenience, only the part related to the embodiment of the present application is shown. Figure 5 The target behavior prediction device 500 based on video surveillance includes an acquisition module 501, a processing module 502, and a determination module 503.

[0086] In one implementation, the apparatus 500 may be used to implement the above Figure 1 For example, the acquisition module 501 is used to implement S110, the processing module 502 is used to implement S120 and S130, and the determination module 503 is used to implement S140.

[0087] In another implementation, the apparatus 500 may be used to implement the above Figure 3 In the method shown, for example, the acquisition module 501 is used to implement S310, and the determination module 503 is used to implement S320 to S370.

[0088] The technical solution provided by the present application converts the relative distance and relative angle between the target and the video shooting point into the actual coordinate information of the target based on the video surveillance information, and judges whether the target in the video is in the warning area based on the coordinate information. When the video reaches a preset time in the warning area, the target image information is obtained, and the prediction model is started. According to the image information and the prediction model, the target intention is determined. Compared with the monitoring video in the prior art that only records the process of the target's behavior, the present application can judge the target's intention based on the target's behavior, and when the judgment result is a dangerous behavior, the relevant personnel are notified in time to deal with it. At the same time, in the process of model creation, the present application combines multiple continuous photos of user behavior into a group for training, and then obtains a weighted feature map based on the feature map through two weighted processes, and then determines the prediction model based on the weighted feature map. Compared with the traditional neural network model, the accuracy of the prediction model is improved.

[0089] Figure 6 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Figure 6 As shown, the electronic device 6 of this embodiment includes: at least one processor 60 ( Figure 6 Only one is shown in the figure) a processor, a memory 61, and a computer program 62 stored in the memory 61 and executable on the at least one processor 60, and when the processor 60 executes the computer program 62, the steps in any of the above method embodiments are implemented.

[0090] The electronic device 6 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The electronic device may include, but is not limited to, a processor 60 and a memory 61. Those skilled in the art will appreciate that Figure 6 It is only an example of the electronic device 6 and does not constitute a limitation on the electronic device 6. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include input and output devices, network access devices, etc.

[0091] The processor 60 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0092] In some embodiments, the memory 61 may be an internal storage unit of the electronic device 6, such as a hard disk or memory of the electronic device 6. In other embodiments, the memory 61 may also be an external storage device of the electronic device 6, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device 6. Further, the memory 61 may also include both an internal storage unit of the electronic device 6 and an external storage device. The memory 61 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program, etc. The memory 61 may also be used to temporarily store data that has been output or is to be output.

[0093] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0094] An embodiment of the present application also provides an electronic device, which includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the processor implements the steps of any of the above-mentioned method embodiments when executing the computer program.

[0095] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0096] An embodiment of the present application provides a computer program product. When the computer program product runs on a mobile terminal, the mobile terminal can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0097] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the camera / electronic device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a disk or an optical disk.

[0098] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0099] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0100] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A target behavior prediction method based on video surveillance, characterized in that: The method comprises: Acquire monitoring video information, wherein the monitoring video information includes a relative distance and a relative angle between a target and a video shooting point; Acquiring coordinate information of a target based on the monitoring video information; When the coordinate information of the target is within the warning range, acquiring image information of the target based on the monitoring video information; Determine the target intention based on the image information of the target and the constructed prediction model, wherein the prediction model is created by a convolutional neural network; wherein constructing the prediction model comprises: obtaining the image information of the target in the historical monitoring video information, wherein the image information of the target comprises a plurality of continuous behavior images of the target within the warning range; Based on the plurality of continuous behavior images, determining an image group, wherein the image group includes at least two continuous behavior images; Determining a preprocessed image group based on the image group being preprocessed; Based on the preprocessed image group and the convolutional neural network, a prediction model is determined, wherein the convolutional neural network includes a convolutional layer, a weight layer, a downsampling layer, and a fully connected layer; the prediction model is determined based on the preprocessed image group and the convolutional neural network, including: Based on the continuous behavior images in the preprocessed image group, determining a feature map of the continuous behavior images through the convolution layer, wherein the feature map of the continuous behavior images includes a first behavior feature map and a second behavior feature map; Based on the feature map of the continuous behavior image, determining the combined weight map of the image group through the weight layer, including: based on the feature map of the continuous behavior image, determining channel information through pooling processing; Based on the channel information and a preset formula, determining a one-dimensional convolution kernel size of the channel information; Applying the one-dimensional convolution kernel size to the one-dimensional convolution to determine a first weight of the channel information; Determining a second weight of the channel information based on the first weight and the swish activation function; Based on the second weight and the feature map, determine a weighted feature map of the feature map, the weighted feature map comprising a first behavior weighted feature map and a second behavior weighted feature map; Determining a combined weight map of the image group based on the weighted feature map and the weight function; Based on the combined weight map of the image group, determining a simplified feature map through the downsampling layer, wherein the size of the simplified feature map is smaller than the size of the combined weight map; Based on the simplified feature graph, the prediction model is determined through the fully connected layer.

2. The method according to claim 1, characterized in that The acquiring the coordinate information of the target based on the monitoring video information includes: Determine first coordinate position information of the target based on the relative distance and relative angle, a first preset coordinate conversion matrix and an error matrix; Based on the first coordinate position information and the second coordinate conversion matrix, coordinate information of the target is determined.

3. The method according to claim 1, characterized in that When the coordinate information of the target is within the warning range, acquiring target image information based on the monitoring video information includes: When the time when the coordinate information of the target is within the warning range reaches a preset threshold, a plurality of continuous image information of the target within a preset time interval is acquired based on the monitoring video information; Accordingly, determining the target intention based on the image information of the target and the constructed prediction model includes: The target intention is determined based on the multiple continuous image information and the constructed prediction model.

4. A target behavior prediction device based on video surveillance, used to execute the method according to any one of claims 1 to 3, characterized in that: include: An acquisition module is used to acquire surveillance video information; A processing module, used for acquiring coordinate information of a target based on the monitoring video information; The processing module is further configured to obtain image information of the target based on the monitoring video information when the coordinate information of the target is within the warning range; A determination module is used to determine the target intention based on the image information of the target and a constructed prediction model, wherein the prediction model is created by a convolutional neural network.

5. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 3 is implemented.

6. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 3 is implemented.

Images