Property safety prevention and control method and system based on artificial intelligence

Abstract

The present invention discloses a property safety prevention and control method and system based on artificial intelligence, comprising: first obtaining the current monitoring video, and using a pre-trained target video frame analysis model to extract the image information of the person in the video. Subsequently, based on this information, a local monitoring image containing the person is extracted, and the behavior type is identified through a target park behavior classification model. Once abnormal behavior is identified, the system will automatically send the current monitoring video to a security server and generate an alarm message, thereby realizing intelligent monitoring and early warning of property safety. Such a design improves the monitoring efficiency and response speed of property safety, reduces labor costs, and enhances security.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an artificial intelligence-based property safety control method and system. Background Art

[0002] With the advancement of technology, intelligent technology has gradually been introduced into property security management. Traditional property security prevention and control mainly relies on human monitoring, but this method is inefficient and prone to errors. Summary of the invention

[0003] The purpose of the present invention is to provide a property safety control method and system based on artificial intelligence.

[0004] In a first aspect, an embodiment of the present invention provides a property safety control method based on artificial intelligence, comprising:

[0005] Get the current surveillance video;

[0006] Input the current surveillance video into a pre-trained target video frame analysis model to obtain image information of people in the current surveillance video;

[0007] Extracting a partial surveillance image including the person from the current surveillance video according to the person image information;

[0008] Inputting the local monitoring image into a pre-trained target park behavior classification model to obtain a park behavior type recognition result;

[0009] When the park behavior type identification result is characterized as abnormal behavior, the current monitoring video is sent to a security server and corresponding alarm information is generated.

[0010] In a second aspect, an embodiment of the present invention provides a server system, including a server, wherein the server is used to execute the method described in the first aspect.

[0011] Compared with the prior art, the beneficial effects provided by the present invention include: adopting a property safety prevention and control method and system based on artificial intelligence disclosed by the present invention, by acquiring the current monitoring video and using a pre-trained target video frame analysis model, the image information of the characters in the video is extracted. Subsequently, the local monitoring image containing the characters is extracted based on this information, and the behavior type is identified through the target park behavior classification model. Once abnormal behavior is identified, the system will automatically send the current monitoring video to the security server and generate an alarm message, thereby realizing intelligent monitoring and early warning of property safety. Such a design improves the monitoring efficiency and response speed of property safety, reduces labor costs, and enhances security. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only illustrate certain embodiments of the present invention and should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can also be obtained based on these drawings without creative work.

[0013] Figure 1 A schematic diagram of the steps of the property safety control method based on artificial intelligence provided by an embodiment of the present invention;

[0014] Figure 2 A schematic block diagram of the structure of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0015] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0016] The specific implementation modes of the present invention are described in detail below in conjunction with the accompanying drawings.

[0017] In order to solve the technical problems in the aforementioned background technology, Figure 1 A flow chart of an artificial intelligence-based property safety control method provided in an embodiment of the present disclosure is provided below, and the artificial intelligence-based property safety control method is introduced in detail.

[0018] Step S201, obtaining the current surveillance video;

[0019] Step S202, inputting the current surveillance video into a pre-trained target video frame analysis model to obtain image information of people in the current surveillance video;

[0020] Step S203, extracting a partial surveillance image including the person from the current surveillance video according to the person image information;

[0021] Step S204, inputting the local monitoring image into a pre-trained target park behavior classification model to obtain a park behavior type recognition result;

[0022] Step S205: When the park behavior type identification result is characterized as abnormal behavior, the current monitoring video is sent to a security server and corresponding alarm information is generated.

[0023] In an embodiment of the present invention, illustratively, the server obtains real-time surveillance video streams from various surveillance cameras in the property community. For example, cameras are installed at key locations such as the community entrance, corridors, and public activity areas, and these cameras continuously transmit video data to the server. The server inputs the received surveillance video stream into a pre-trained target video frame analysis model. This model can identify dynamic targets in the video, especially human targets, and extract human image information. For example, the model can identify people walking, people riding bicycles, etc. in the video, and extract the images of these human targets. After extracting the human image information, the server will further process this information and extract local surveillance images containing these people from the entire surveillance video. For example, if a person wanders at the entrance of the community, the server will extract the local image of the person wandering from the entire surveillance video. Next, the server inputs these local surveillance images containing people into another pre-trained target park behavior classification model. This model can identify and analyze the human behavior in these images and classify them as normal behavior or abnormal behavior. For example, the model can identify abnormal behaviors such as whether a person is destroying public facilities or breaking into prohibited areas. Once the target park behavior classification model identifies abnormal behavior, the server will immediately send the entire current surveillance video to the security server and generate an alarm message at the same time. For example, if someone breaks into a prohibited area, the server will immediately send the surveillance video of this behavior to the security server and generate an alarm message: "Attention! Someone broke into the XX area, please check immediately and take appropriate measures." After receiving the alarm message, the security personnel can quickly check the relevant video and take appropriate security measures.

[0024] In an embodiment of the present invention, the target video frame analysis model is obtained in the following manner.

[0025] Obtain sample surveillance video;

[0026] Extracting a video frame feature set of the sample surveillance video;

[0027] Processing the video frame feature set of the sample surveillance video according to the initial video frame analysis model to obtain a predicted character information set of the sample surveillance video, wherein the character information set includes coordinate information of each character range in a plurality of character ranges and character image information corresponding to each character range, wherein the coordinate information of each character range is used to characterize the position of the corresponding character range in the sample surveillance video;

[0028] The initial video frame analysis model is trained according to the character information set and the target value of the character information set to obtain a target video frame analysis model, and the target video frame analysis model is used to process the current monitoring video to obtain the character image information in the current monitoring video.

[0029] In an embodiment of the present invention, exemplarily, in order to train the target video frame analysis model, the server first collects a large number of sample surveillance videos from the historical surveillance records of the property community. These videos contain various scenes, such as surveillance videos under different conditions such as day and night, sunny and rainy days, to ensure the generalization ability of the model. The server uses a specific algorithm to process the collected sample surveillance videos and extracts a feature set of each video frame. These features may include information such as color, texture, shape, motion, etc., which are essential for subsequent character detection and recognition. The server inputs the extracted video frame feature set into an initial video frame analysis model. This model will further process and analyze these features to predict a character information set in the sample surveillance video. This character information set includes the coordinate information of each character range in multiple character ranges and the corresponding character image information. For example, the model will identify multiple characters in the video and generate a bounding box for each character, and the coordinate information of this bounding box is used to characterize the specific position of the character in the video. After obtaining the predicted character information set, the server will compare this set with the real character information (i.e., the target value). By calculating the difference between the predicted value and the true value, the server can iterate and optimize the initial video frame analysis model to enable it to more accurately identify and locate the people in the video. This process is the training process of the model. Through multiple iterations and optimizations, the target video frame analysis model obtained in the end will be able to more accurately process and analyze the current surveillance video, thereby extracting more accurate image information of the person. For example, during the training process, the model will encounter some difficult-to-identify people or complex background situations. By continuously adjusting the parameters and structure of the model and introducing more training data, the server can gradually improve the recognition accuracy and robustness of the model. The target video frame analysis model obtained in the end will be able to provide strong support for property safety prevention and control in practical applications.

[0030] In the embodiment of the present invention, the video frame feature set of the sample surveillance video is processed according to the initial video frame analysis model to obtain a predicted character information set of the sample surveillance video, which can be implemented through the following examples.

[0031] Inputting a video frame feature set of the sample surveillance video, time sequence information of a plurality of character ranges, and a target value of the character information set into the initial video frame analysis model, wherein the target value of the character information set includes a target value of coordinate information of each character range and a target value of character image information corresponding to each character range;

[0032] The following steps are performed by the initial video frame analysis model to obtain a predicted person information set of the sample surveillance video:

[0033] For a target person range, the coordinate information of the target person range is determined according to the target values ​​of the coordinate information of all person ranges arranged before the target person range according to the time sequence information and the target values ​​of the person image information of all the person ranges;

[0034] The current person image information corresponding to the target person range is determined according to the coordinate information target value of the target person range, the coordinate information target values ​​of all person ranges arranged before the target person range and the person image information target values ​​of all person ranges.

[0035] In an embodiment of the present invention, illustratively, the server first organizes the video frame feature sets of the sample surveillance video, which capture key visual information in the video, such as edges, textures, colors, and motions. At the same time, the server also records the timing information of multiple character ranges appearing in the video, that is, the position and activity status of each character at different time points. In addition, the server also prepares a target value of a character information set, which includes the precise coordinate information target value of each character range and the corresponding character image information target value. These target values ​​are obtained by manual annotation or other reliable methods and are used as reference standards when training the model. Next, the server inputs these data (video frame feature set, timing information, and target value) into the initial video frame analysis model. This model is a deep learning network designed to understand the video content and predict the character information therein. For the target character range, the server will find all the character ranges that appear before the target character range based on the timing information. These previously appearing character ranges provide important contextual information, which helps the model to more accurately predict the location of the target character range. For example, if the target character range is a walking person, then the character range that appears before him may be his previous position. By analyzing these previous positions, the model can learn the trajectory and speed of the person, so as to more accurately predict his current position. The server uses the coordinate information target values ​​and the person image information target values ​​of these previous person ranges, combined with the complex calculations of the initial video frame analysis model, to determine the coordinate information of the target person range. This is an iterative and optimization process, and the model will continuously adjust its internal parameters to minimize the difference between the predicted coordinate information and the actual coordinate information target value. Once the coordinate information of the target person range is determined, the server will further use this information, as well as the coordinate information and person image information target values ​​of all person ranges arranged before the target person range, to determine the current person image information corresponding to the target person range. This step involves a detailed analysis of specific areas in the video frame to extract a clear image of the target person. For example, if the target person range is determined to be a specific area in the surveillance video, the server will focus on this area and apply advanced image processing technology to enhance and extract the image of the person in the area. This can include steps such as color correction, contrast enhancement, and denoising to ensure that the extracted person image is as clear and accurate as possible. Through the above steps, the server can use the initial video frame analysis model to deeply process the sample surveillance video and accurately predict the location and image information of each person in the sample surveillance video. This is crucial for subsequent property safety prevention and control, as it provides key person detection and recognition capabilities for the real-time monitoring system.

[0036] In an embodiment of the present invention, the initial video frame analysis model includes multiple state space models; the current character image information corresponding to the target character range is determined based on the coordinate information target value of the target character range, the coordinate information target values ​​of all character ranges arranged before the target character range, and the character image information target values ​​of all character ranges, which can be implemented through the following examples.

[0037] For a target person range, the coordinate information of the target person range, the target values ​​of the coordinate information of all person ranges arranged before the target person range, and the target values ​​of the person image information of all person ranges are processed according to the multimodal fusion unit in the last state space model to obtain a fusion feature;

[0038] Performing feature standardization on the fused features according to the feature standardization unit in the state space model of the last position to obtain a standardized feature;

[0039] The standardized features are processed according to the feature parsing layer in the final state-space model to obtain the recognition confidence of the target person range, and the recognition confidence of the target person range is used to characterize the current person image information corresponding to the target person range.

[0040] In an embodiment of the present invention, exemplarily, when processing surveillance video, the server will pay special attention to a target person range. For this range, the server will use an important component of the initial video frame analysis model, the multimodal fusion unit in the last state space model. This unit is responsible for fusing the coordinate information of the target person range, the target values ​​of the coordinate information of all person ranges arranged before the target person range, and the target values ​​of the person image information of all person ranges. For example, assuming that there is a walking person in the surveillance video, the server has determined the coordinates of this person (target person range) in the current frame, as well as the coordinate information and image information of the previous frames through the previous steps. The multimodal fusion unit will fuse this information together to form a more comprehensive and richer feature representation. This fused feature not only contains the spatial position information of the target person, but also contains his dynamic behavior and appearance characteristics. Next, the server will use the feature standardization unit in the last state space model to standardize the fused feature. The purpose of this step is to eliminate the dimensional differences and numerical range differences between features, so that different features can be compared and calculated on the same scale. The fused features of a walking person can include multiple dimensions such as position coordinates, speed, acceleration, and color histogram. These features may have large differences in numerical range and units. Through feature standardization, the server can unify these features into a standard numerical range, which facilitates subsequent feature analysis. Finally, the server will use the feature analysis layer in the terminal state space model to process the standardized features. The purpose of this step is to judge the recognition confidence of the target person range based on the standardized features. Recognition confidence is an important indicator that is used to characterize the accuracy and reliability of the current person image information corresponding to the target person range. In the above example, the feature analysis layer will deeply analyze the standardized features of the walking person, including his position, speed, acceleration, and appearance features. Through complex calculations and reasoning, the feature analysis layer will output a recognition confidence value. The higher this value is, the more confident the server is in recognizing the current person image information; conversely, it means that there may be a certain degree of uncertainty in the recognition result. This recognition confidence value has important reference value for subsequent property safety prevention and control decisions.

[0041] In the embodiment of the present invention, the aforementioned training of the initial video frame analysis model according to the character information set and the target value of the character information set to obtain the target video frame analysis model can be implemented through the following examples.

[0042] Calculating a deviation between the character information set and a target value of the character information set to obtain a cost parameter;

[0043] The initial video frame analysis model is trained according to the cost parameter to obtain a target video frame analysis model.

[0044] In an embodiment of the present invention, illustratively, during the training process of the server, it will first compare the difference between the character information set predicted by the model and the target value of the actual character information set. This difference is called deviation. Specifically, the server will compare its predicted coordinate information with the actual coordinate information target value for each character range, and will also compare the predicted character image information with the actual character image information target value. To illustrate with a specific example, assuming that in a surveillance video, the model predicts that a certain character is at the (100,200) position of the screen, but in fact the character appears at the (110,210) position, then the difference of these 10 pixels is the deviation of the coordinate information. Similarly, if the character image predicted by the model is significantly different from the actual image in color, shape or texture, these differences will also be counted in the deviation. The server will calculate the sum of these deviations through a specific algorithm, such as mean square error or cross entropy loss, to obtain a cost parameter. This cost parameter reflects the overall gap between the model prediction and the actual situation. Once the cost parameter is obtained, the server will use this parameter to adjust the parameters of the initial video frame analysis model. This is an optimization process, the purpose of which is to make the model's prediction results closer to the actual situation through continuous iteration and adjustment, that is, to reduce the cost parameter. Specifically, the server will use optimization algorithms such as gradient descent to fine-tune the weights and biases in the model according to the cost parameter. After each adjustment, the server will rerun the model, compare the deviation between the new prediction result and the true target value, and calculate the cost parameter again. This process will be repeated until the cost parameter is reduced to an acceptable range or the preset number of training rounds is reached. Finally, when the model's prediction performance reaches a satisfactory level, it is said that the model has been trained, and the model obtained at this time is the required target video frame analysis model. This model will be able to analyze surveillance video frames more accurately, extract key character information, and provide strong support for subsequent property safety prevention and control.

[0045] In the embodiment of the present invention, the step of extracting the video frame feature set of the sample surveillance video may be implemented through the following example.

[0046] Sampling the sample surveillance video to obtain a plurality of sample video frames;

[0047] Performing a feature extraction operation on each of the sample video frames according to a preset encoder to obtain a video frame feature of each of the sample video frames;

[0048] An integration operation is performed on the video frame features of the plurality of sample video frames to obtain a video frame feature set of the sample surveillance video.

[0049] In an embodiment of the present invention, the server first processes the sample surveillance video, and the first step of this process is sampling. Sampling refers to selecting representative moments or clips from a continuous video stream. This can reduce the amount of data processing and retain the key information of the video. For example, the server can sample one frame from the surveillance video every second to obtain a sequence of sample video frames containing multiple key moments. These sample video frames will serve as the basis for subsequent feature extraction. Next, the server will use a preset encoder to extract features from each sampled sample video frame. The encoder is a specially designed algorithm or model that can extract meaningful information from an image, such as edges, textures, color distribution, etc., which are called features. For example, the server can use a deep learning model as an encoder, which has been trained on a large amount of image data and can effectively extract rich features from the image. For each sample video frame, the encoder outputs a feature vector that contains the key visual information in the frame. Finally, the server integrates the feature vectors of all sample video frames to form a video frame feature set. This set is a compressed and abstract representation of the entire surveillance video content, which captures the key information and dynamic changes in the video. Specifically, the server can arrange the feature vectors of each sample video frame in the order of sampling time to form a two-dimensional array or matrix. This matrix is ​​the video frame feature set of the sample surveillance video, which will be used for subsequent video analysis and character information extraction tasks. In this way, the server can efficiently process a large amount of surveillance video data, extract useful feature information, and provide strong support for subsequent property safety prevention and control.

[0050] In the embodiment of the present invention, the feature extraction operation is performed on each of the sample video frames according to the preset encoder to obtain the video frame features of each of the sample video frames, which can be implemented through the following examples.

[0051] According to each local feature focuser in the preset encoder, a feature extraction operation is performed on each video segment region in a first sample video frame to obtain a video frame feature of each video segment region in the first sample video frame, where the first sample video frame is any one of the multiple sample video frames;

[0052] An integration operation is performed on the video frame features of each of the video clip regions in the first sample video frame according to the feature fusion unit in the preset encoder to obtain the video frame features of the first sample video frame.

[0053] In an embodiment of the present invention, exemplarily, when processing surveillance video, the server will use the local feature focuser in the preset encoder to perform detailed feature extraction on each sample video frame. Take the first sample video frame as an example, this frame is one randomly selected by the server from multiple sample video frames. The local feature focuser will focus on different video clip areas in the first sample video frame. For example, if the first sample video frame shows a person walking, the local feature focuser will focus on different video clip areas such as the person's head, body, limbs, etc. For each video clip area, the local feature focuser will perform feature extraction operations. This includes identifying visual information such as edges, textures, colors, etc., and encoding this information into feature vectors. For example, when focusing on the head area of ​​a person, the local feature focuser will extract features such as hair color, face shape, eye size and position. After the local feature extraction is completed, the server will use the feature fusion unit in the preset encoder to integrate these local features. The task of the feature fusion unit is to fuse the feature vectors of each video clip area together to form a feature vector that fully represents the first sample video frame. For example, the feature fusion unit will use weighted averaging, concatenation or other complex fusion strategies to combine feature vectors of areas such as the head, body and limbs. In this way, the final video frame features will be able to comprehensively reflect the information of each important area in the first sample video frame. In this way, the server can more accurately capture and describe the key content in the surveillance video, providing a rich feature basis for subsequent character information extraction and identification. This meticulous feature extraction and fusion method helps to improve the accuracy and reliability of video analysis.

[0054] In an embodiment of the present invention, the inputting of the local monitoring image into a pre-trained target park behavior classification model to obtain a park behavior type recognition result can be implemented through the following example.

[0055] Acquire a local monitoring image and a plurality of predefined motion behavior data, wherein one of the plurality of motion behavior data is used to describe a park behavior type in a preset park behavior type;

[0056] Input the local monitoring image and the multiple action behavior data into a pre-trained target park behavior classification model to obtain a local monitoring image feature representation and multiple action behavior feature representations, wherein the target park behavior classification model includes an action behavior conversion network, a monitoring image conversion network, action behavior prior knowledge and monitoring image prior knowledge, the action behavior prior knowledge and the monitoring image prior knowledge are generated by basic prior knowledge, the monitoring image prior knowledge is used to be input into the monitoring image conversion network in coordination with the local monitoring image to obtain the local monitoring image feature representation, and the action behavior prior knowledge is used to be input into the action behavior conversion network in coordination with the multiple action behavior data to obtain the multiple action behavior feature representations;

[0057] Determine the characteristic distance between the local monitoring image feature representation and each action behavior feature representation in the multiple action behavior feature representations through the target park behavior classification model, and use the target park behavior type indicated by the action behavior feature representation whose characteristic distance satisfies the preset characteristic distance limit as the park behavior type corresponding to the local monitoring image feature representation;

[0058] The park behavior type is used as the park behavior type recognition result of the local monitoring image.

[0059] In an embodiment of the present invention, illustratively, the server captures a 10-second local surveillance image from the property's surveillance system. This image clearly shows a person walking on a walkway in the park, suddenly stopping and bending down to pick up an object on the ground. The image is captured at a high definition and 30 frames per second, ensuring the consistency and detail clarity of the behavior. The server loads a set of predefined action behavior data, which are stored in the form of vectors, each of which represents a specific behavior pattern. For example, "walking" can be represented as a specific vector sequence, while "bending down to pick up an object" is another different vector sequence. These data are obtained through the previous learning and analysis of a large number of behavior samples, and are highly representative and accurate. The action behavior conversion network is responsible for processing the predefined action behavior data and converting it into feature vectors that are easier to compare and identify. The monitoring image conversion network processes the local monitoring image, extracts the key behavior features in the image, and converts them into corresponding feature vectors. Both networks are trained by deep learning algorithms and can efficiently and accurately extract and convert the features of input data. Prior knowledge is learned through a large amount of data, and they provide the model with an in-depth understanding of action behaviors and monitoring images. Prior knowledge of action behavior helps the model recognize and understand various predefined action behavior patterns. Prior knowledge of surveillance images helps the model extract key behavior features from complex surveillance images. The server uses the target park behavior classification model to calculate the feature distance between the local surveillance image feature representation and multiple action behavior feature representations. The feature distance is obtained by calculating the similarity between two feature vectors. The higher the similarity, the smaller the feature distance. The server compares all calculated feature distances and finds the action behavior feature representation that is closest to the local surveillance image feature representation (i.e., the smallest feature distance). In this example, the feature distance between the action behavior feature representation of "bending down to pick up something" and the feature representation of the local surveillance image is the smallest, so it is identified as the behavior type in the current surveillance image. The server outputs the identified behavior type "bending down to pick up something" as a result to the interface of the property security prevention and control system, and may trigger a corresponding early warning or recording mechanism. This result can not only help property management personnel to understand the dynamics in the park in a timely manner, but also serve as a basis for subsequent security policy adjustments. Through the above detailed step descriptions and technical feature explanations, we can see how the artificial intelligence-based property security prevention and control system automatically identifies and classifies various behavior types in the park through a deep learning model. This technology can not only improve the efficiency and safety of property management, but also provide more possibilities and innovations for future smart security systems.

[0060] In an embodiment of the present invention, the local monitoring image and the multiple action behavior data are input into a pre-trained target park behavior classification model to obtain a local monitoring image feature representation and multiple action behavior feature representations, which can be implemented through the following examples.

[0061] Performing attention weighting processing on the basic prior knowledge and the monitoring image attention weight matrix to determine the monitoring image prior knowledge, wherein the attention weighting processing is used to take the eigenvalue of the monitoring image prior knowledge corresponding to the first characteristic dimension in the matrix as the weighted sum of the eigenvalue on the first characteristic dimension in the basic prior knowledge and the eigenvalue on the first characteristic dimension in the monitoring image attention weight matrix;

[0062] Performing feature integration processing on the local monitoring image and the monitoring image prior knowledge to determine a monitoring image integration feature vector;

[0063] The integrated feature vector features of the monitoring image are converted to a target feature layer to determine the feature representation of the local monitoring image.

[0064] In an embodiment of the present invention, illustratively, the server first loads the basic prior knowledge and the monitoring image attention weight matrix. The basic prior knowledge is general knowledge learned from a large amount of data, while the monitoring image attention weight matrix is ​​generated according to the characteristics of the current local monitoring image, and is used to indicate which features are more important in the recognition process. For example, if a person is running in the monitoring image, the weights of the features related to the movement (such as leg movements, speed, etc.) will be relatively high. The server determines the specific prior knowledge for the current monitoring image, that is, the monitoring image prior knowledge, by performing attention weighting processing on the basic prior knowledge and the monitoring image attention weight matrix. Specifically, the server calculates the eigenvalue of the monitoring image prior knowledge corresponding to the first feature dimension in the matrix as the weighted sum of the eigenvalue on the corresponding feature dimension in the basic prior knowledge and the eigenvalue on the corresponding feature dimension in the monitoring image attention weight matrix. In this way, important features will receive more attention, thereby improving the accuracy of subsequent recognition. Next, the server performs feature integration processing on the local monitoring image and the monitoring image prior knowledge. The purpose of this step is to combine the original features of the image with the prior knowledge to form a more comprehensive and rich feature representation. For example, if a person is waving in a local surveillance image, and the surveillance image prior knowledge provides general features about the waving action (such as the swing amplitude and speed of the arm), then the feature integration process will fuse these two parts of information together to generate a surveillance image integrated feature vector that contains both image details and prior knowledge. Finally, the server will convert the surveillance image integrated feature vector to the target feature layer to determine the feature representation of the local surveillance image. This conversion process can be regarded as mapping the integrated feature vector to a feature space that is easier to compare and classify. In this feature space, similar behaviors will have similar feature representations, which facilitates subsequent behavior type recognition. For example, after feature conversion, two similar actions, "waving" and "hand shaking", will have similar positions in the feature space, while two actions with large differences, "running" and "walking", will have a longer distance in the feature space. Through the above steps, the server can use basic prior knowledge and attention mechanism to enhance the feature extraction capability of local surveillance images, thereby improving the accuracy of subsequent behavior type recognition.

[0065] In the embodiment of the present invention, the feature integration processing of the local monitoring image and the monitoring image prior knowledge to determine the monitoring image integrated feature vector can be implemented through the following examples.

[0066] Cutting the local surveillance image into a plurality of surveillance image segments;

[0067] Projecting the plurality of surveillance image segments onto a surveillance image feature layer to obtain a plurality of segment feature codes;

[0068] The feature codes of the multiple segments and the prior knowledge of the monitoring image are subjected to feature integration processing to determine the integrated feature vector of the monitoring image.

[0069] In an embodiment of the present invention, illustratively, the server first obtains a local surveillance image, which shows a series of actions of a person in the park, including walking, stopping, bending down to pick up something, etc. In order to analyze this image more carefully, the server cuts it into multiple shorter surveillance image segments, each of which contains a few seconds of action. For example, the server can cut a 30-second surveillance image into 6 5-second segments, each of which focuses on a specific action of the person, such as the first segment is walking, the second segment is stopping, and the third segment is bending down. Next, the server projects these cut surveillance image segments onto the surveillance image feature layer. This process is achieved through a deep learning model, which extracts the key features in each segment and encodes them into a feature vector, that is, the segment feature encoding. Taking the first segment (walking) as an example, the model extracts the features of the person's gait, speed, body posture, etc., and encodes these features into a high-dimensional vector. This vector represents the characteristics of the person's walking in the segment. After obtaining the feature encoding of all segments, the server will perform feature integration processing on these feature encodings and the previously determined surveillance image prior knowledge. This process aims to combine the specific features of each segment with general prior knowledge to form a more comprehensive and representative feature representation. Specifically, the server will weight the feature encoding of each segment, and the weights are provided by the surveillance image attention weight matrix. These weighted feature encodings are then spliced ​​or fused with the surveillance image prior knowledge to generate a surveillance image integrated feature vector. For example, in the segment of bending over to pick up something, the server will give higher weights to the features of the waist and hand movements because these features are crucial to identifying the behavior of "bending over to pick up something". The server will then integrate this weighted feature encoding with the surveillance image prior knowledge (such as general features about bending over to pick up something) to obtain an integrated feature vector that contains both the current segment-specific information and general prior knowledge. This integrated feature vector will serve as the basis for subsequent behavior classification and recognition, helping the server to more accurately determine the behavior type in the local surveillance image.

[0070] In an embodiment of the present invention, the plurality of action behavior data are input into a pre-trained target park behavior classification model to obtain a plurality of action behavior feature representations, which can be implemented through the following examples.

[0071] Performing attention weighting processing on the basic prior knowledge and the action behavior attention weight matrix to determine the action behavior prior knowledge, wherein the attention weighting processing is used to take the eigenvalue of the action behavior prior knowledge corresponding to the second characteristic dimension in the matrix as the weighted sum of the eigenvalue on the second characteristic dimension in the basic prior knowledge and the eigenvalue on the second characteristic dimension in the action behavior attention weight matrix;

[0072] Performing feature integration processing on the plurality of action behavior data and the action behavior prior knowledge respectively to determine a plurality of action behavior integrated feature vectors;

[0073] The plurality of action behavior integrated feature vectors are respectively feature-converted to a target feature layer to determine the plurality of action behavior feature representations.

[0074] In an embodiment of the present invention, illustratively, when processing multiple action behavior data, the server will first load basic prior knowledge and action behavior attention weight matrix. These action behavior data can include multiple actions such as "walking", "running", "bending down to pick up objects". Basic prior knowledge is a common feature learned from a large amount of action behavior data, while the action behavior attention weight matrix is ​​generated for specific action behavior data to emphasize the important features in different action behaviors. Taking the action behavior of "running" as an example, the server will determine the specific prior knowledge for "running", i.e., the action behavior prior knowledge, by performing attention weighting processing on the basic prior knowledge and the action behavior attention weight matrix. In this process, the features related to running (such as step frequency, body forward leaning degree, etc.) will obtain higher weights. Next, the server will perform feature integration processing on each action behavior data and the corresponding action behavior prior knowledge. The purpose of this step is to combine the original data of each action behavior with the prior knowledge to generate a more comprehensive feature representation. Taking the action of "bending down to pick up something" as an example, the server will extract the key features of the action (such as the degree of waist bending, hand movements, etc.) and integrate them with the prior knowledge of the action of "bending down to pick up something". In this way, the obtained action behavior integrated feature vector not only contains the specific information of the current action behavior, but also incorporates general prior knowledge. Finally, the server will convert each action behavior integrated feature vector to the target feature layer to determine the feature representation of multiple action behaviors. This conversion process is to make these feature representations easier to compare and classify. For example, after feature conversion, the feature representations of the two action behaviors "walking" and "running" will be clearly distinguished in the feature space because they have significant differences in speed, pace, etc. Similarly, the feature representations of the two action behaviors "bending down to pick up something" and "standing still" will also be clearly separated in the feature space. Through the above steps, the server can use basic prior knowledge and attention mechanism to enhance the feature extraction ability of multiple action behaviors, thereby improving the accuracy of subsequent behavior type recognition. This processing method helps the server to more accurately understand and classify various behaviors in the park, providing strong support for property safety prevention and control.

[0075] In the embodiment of the present invention, the feature integration processing is performed on the plurality of action behavior data and the action behavior prior knowledge respectively to determine a plurality of action behavior integrated feature vectors, which can be implemented through the following examples.

[0076] Based on the following steps, the plurality of action behavior data and the action behavior prior knowledge are respectively subjected to feature integration processing to determine a plurality of action behavior integrated feature vectors, wherein the action behavior data subjected to feature integration processing each time is determined as candidate action behavior data, and the action behavior integrated feature vector obtained by the feature integration processing each time is determined as a candidate action behavior integrated feature vector:

[0077] Performing sub-action data action splitting processing on the candidate action behavior data to obtain multiple sub-action data;

[0078] The plurality of sub-action data are projected onto the action behavior feature layer to obtain the candidate action behavior integrated feature vector.

[0079] In an embodiment of the present invention, illustratively, the server first obtains a video data containing multiple continuous action behaviors, which records the complete activity trajectory of a person in the park, such as from walking to stopping, then bending over to pick up items on the ground, and then continuing to walk. In order to analyze these actions more carefully, the server needs to split these continuous action behaviors. Taking this video data as candidate action behavior data, the server splits this continuous video data into multiple independent sub-action data through advanced action recognition algorithms. For example, the server can split the original video into four sub-action data of "walking", "stopping", "bending over to pick up objects" and "continue walking". After the splitting is completed, the server will then project each sub-action data onto a predefined action behavior feature layer. This process is achieved through a deep learning model, which can extract the key features in each sub-action data and encode it into a feature vector. Taking the sub-action data of "bending over to pick up objects" as an example, the server will input it into the deep learning model, and the model will extract the key features of the action, such as the degree of bending of the waist, the extension state of the hands, the position change of the object, etc., and encode these features into a high-dimensional feature vector. This feature vector is the action behavior integrated feature vector of the sub-action data of "bending down to pick up something", which comprehensively and compactly represents all the important features of this sub-action. The same process will be carried out for each sub-action data, and finally the server will obtain a series of action behavior integrated feature vectors, each of which corresponds to a specific sub-action data. These integrated feature vectors will serve as the basis for subsequent behavior classification and identification. Through the above steps, the server can more effectively process and analyze various action behavior data in the park, and provide more accurate and reliable support for property safety prevention and control. For example, when abnormal behavior is discovered, the server can quickly locate the specific sub-action data, and quickly and accurately identify and respond based on its feature vector.

[0080] In the embodiments of the present invention, the following implementation modes are also provided.

[0081] Acquire a monitoring image instance and a plurality of action behavior data instances, wherein the monitoring image instance is pre-assigned to correspond to a target action behavior data instance, and the plurality of action behavior data instances include the target action behavior data instance;

[0082] Inputting the monitoring image instance and the multiple action behavior data instances into the native park behavior classification model to be trained, obtaining a monitoring image instance feature representation and multiple action behavior feature representation instances, wherein the native park behavior classification model includes the action behavior conversion network, the monitoring image conversion network, native action behavior prior knowledge and native monitoring image prior knowledge, and the native action behavior prior knowledge and the native monitoring image prior knowledge are generated by native basic prior knowledge;

[0083] Determine, by means of the native park behavior classification model, a sample feature distance between the monitoring image instance feature representation and each action behavior feature representation instance in the plurality of action behavior feature representation instances;

[0084] The error parameter is calculated based on the sample feature distance, and the native basic prior knowledge and the native attention weight matrix are optimized using the error parameter until the preset training stop condition is met to obtain the target park behavior classification model.

[0085] In an embodiment of the present invention, illustratively, the server first obtains a set of surveillance image instances and multiple action behavior data instances from a database. These data are previously collected and labeled in an actual park environment. For example, a surveillance image instance may be a video of a person walking in a park, and the corresponding action behavior data instance is various parameters of the walking behavior, such as pace, step length, etc. Next, the server inputs these instance data into a native park behavior classification model to be trained. This model includes an action behavior conversion network, a surveillance image conversion network, native action behavior prior knowledge, and native surveillance image prior knowledge. These prior knowledge are generated by native basic prior knowledge, which provide the model with a preliminary understanding of park behavior. Taking walking video as an example, the surveillance image conversion network will extract features from the video, while the action behavior conversion network will process the corresponding action behavior data. Both networks will use native prior knowledge to assist feature extraction and data processing. Through the processing of the native park behavior classification model, the server obtains the feature representation of the surveillance image instance and multiple action behavior feature representation instances. Then, the server calculates the sample feature distance between the surveillance image instance feature representation and each action behavior feature representation instance. This distance can be regarded as a similarity measure between the two. Taking walking video as an example, the server will calculate the sample feature distance between the feature representation of the video and the feature representation of the walking action behavior data instance. If the distance between the two is small, it means that the model can associate the two well. Based on the calculated sample feature distance, the server will further calculate an error parameter. This error parameter reflects the accuracy of the model in matching the surveillance image with the action behavior data. Then, the server will use this error parameter to optimize the native basic prior knowledge and the native attention weight matrix. For example, if the error parameter is large, it means that the model's corresponding accuracy is not high. At this time, the server will adjust the parameters of the native basic prior knowledge and the native attention weight matrix so that the model can better learn and understand the behavior in the park. This process will continue until the preset training stop condition is met (such as reaching a certain number of training rounds, the error parameter is less than a certain threshold, etc.). Finally, the server will obtain an optimized target park behavior classification model, which can more accurately understand and classify various behaviors in the park.

[0086] In an embodiment of the present invention, the monitoring image instance and the multiple action behavior data instances are input into the native campus behavior classification model to be trained to obtain the monitoring image instance feature representation and multiple action behavior feature representation instances, which can be implemented through the following examples.

[0087] Acquire the native basic prior knowledge, the native monitoring image attention weight matrix, and the native action behavior attention weight matrix, wherein the native basic prior knowledge, the native monitoring image attention weight matrix, and the native action behavior attention weight matrix are tuning parameters for the training process of optimizing the native campus behavior classification model;

[0088] Performing attention weighting processing on the native basic prior knowledge and the native monitoring image attention weight matrix to determine the native monitoring image prior knowledge, wherein the attention weighting processing is used to take the eigenvalue of the native monitoring image prior knowledge corresponding to the third characteristic dimension in the matrix as the weighted sum of the eigenvalue on the third characteristic dimension in the native basic prior knowledge and the eigenvalue on the third characteristic dimension in the native monitoring image attention weight matrix;

[0089] Performing the attention weighting processing on the native basic prior knowledge and the native action behavior attention weight matrix to determine the native action behavior prior knowledge, wherein the attention weighting processing is used to take the eigenvalue of the native action behavior prior knowledge corresponding to the fourth characteristic dimension in the matrix as the weighted sum of the eigenvalue on the fourth characteristic dimension in the native basic prior knowledge and the eigenvalue on the fourth characteristic dimension in the native action behavior attention weight matrix;

[0090] Performing feature integration processing on the surveillance image instance and the native surveillance image prior knowledge to determine a surveillance image instance integrated feature vector, and performing feature integration processing on the multiple action behavior data instances and the native action behavior prior knowledge respectively to determine multiple sample action behavior integrated feature vectors;

[0091] The monitoring image instance integrated feature vector and the plurality of sample action behavior integrated feature vector features are converted to a native feature layer, and the monitoring image instance feature representation and the plurality of action behavior feature representation instances are determined.

[0092] In an embodiment of the present invention, exemplarily, the server first obtains native basic prior knowledge, native monitoring image attention weight matrix, and native action behavior attention weight matrix. These parameters are key tuning parameters for optimizing the training of the native park behavior classification model. For example, the native basic prior knowledge may include general features about park behavior learned from a large amount of data; the native monitoring image attention weight matrix and the native action behavior attention weight matrix respectively emphasize the important features in the monitoring image and action behavior data. Next, the server uses attention weighting processing to determine the native monitoring image prior knowledge and the native action behavior prior knowledge. Specifically, the server weights the native basic prior knowledge with the corresponding attention weight matrix to emphasize important features. For example, in terms of monitoring images, if a feature (such as the motion trajectory of a person) is particularly important in identifying behavior, then the feature will be given a higher weight in the native monitoring image prior knowledge. Then, the server performs feature integration processing on the monitoring image instance and the native monitoring image prior knowledge to determine the monitoring image instance integrated feature vector. At the same time, the server also performs feature integration processing on multiple action behavior data instances and native action behavior prior knowledge to determine multiple sample action behavior integrated feature vectors. This process aims to combine instance data with prior knowledge to generate a more comprehensive and representative feature representation. Finally, the server converts the integrated feature vector of the monitoring image instance and the integrated feature vector features of multiple sample action behaviors to the native feature layer to determine the monitoring image instance feature representation and multiple action behavior feature representation instances. This conversion process is to make these feature representations easier for the model to understand and classify. For example, through feature conversion, the server can map high-dimensional feature vectors to a lower-dimensional space while retaining key information for subsequent classification and recognition tasks. Through the above steps, the server can use native tuning parameters to perform in-depth processing and analysis of monitoring image instances and action behavior data instances to obtain more representative feature representations. These feature representations will serve as the basis for subsequent behavior classification and recognition, helping to improve the accuracy and generalization ability of the model.

[0093] In an embodiment of the present invention, the local monitoring image and the multiple action behavior data are input into a pre-trained target park behavior classification model to obtain a local monitoring image feature representation and multiple action behavior feature representations, which can be implemented through the following examples.

[0094] In the case where the target park behavior classification model includes a multi-level surveillance image classification unit, the corresponding basic prior knowledge and attention weight matrix are allocated to the surveillance image classification units at each level, wherein the output of the n-th level surveillance image classification unit in the multi-level surveillance image classification unit and the surveillance image prior knowledge corresponding to the n+1-th level surveillance image classification unit are coordinated as the input of the n+1-th level surveillance image classification unit, and n is a positive integer greater than 1;

[0095] In the case where the target park behavior classification model includes multi-level action behavior classification units, the corresponding basic prior knowledge and the attention weight matrix are allocated to the action behavior classification units at each level, wherein the output of the n-th level action behavior classification unit in the multi-level action behavior classification units and the action behavior prior knowledge corresponding to the n+1-th level action behavior classification unit are coordinated as the input of the n+1-th level action behavior classification unit.

[0096] In an embodiment of the present invention, exemplarily, the server first processes a target park behavior classification model including multiple levels of surveillance image classification units. In this model, each level of surveillance image classification unit has corresponding basic prior knowledge and attention weight matrix. These parameters are customized according to the classification requirements of different levels. Taking a three-level surveillance image classification model as an example, the first-level classification unit can be responsible for identifying basic elements in the surveillance image, such as people, vehicles, etc. When the server inputs the local surveillance image into the first-level classification unit, it will perform feature extraction and classification in combination with the corresponding basic prior knowledge and attention weight matrix. Then, the output of the first-level classification unit (such as the identified person and vehicle information) will be coordinated with the surveillance image prior knowledge corresponding to the second-level surveillance image classification unit (such as the behavior pattern of the person and the vehicle) as the input of the second-level classification unit. The second-level classification unit can further analyze the interactive relationship between these basic elements, such as the walking trajectory of the person, the driving direction of the vehicle, etc. Similarly, the server will use the corresponding basic prior knowledge and attention weight matrix for processing, and pass the result to the third-level classification unit. The third-level classification unit can be responsible for higher-level behavior recognition, such as judging whether the person is performing abnormal behavior, whether the vehicle is driving illegally, etc. The server will combine the outputs of the first two levels with the basic prior knowledge and attention weight matrix of this level to make the final classification judgment. Similar to the classification of surveillance images, the server will also process the target park behavior classification model containing multiple levels of action behavior classification units. Each level of action behavior classification unit also has corresponding basic prior knowledge and attention weight matrix. Taking a three-level action behavior classification model as an example, the first-level classification unit can be responsible for identifying basic action types, such as walking, running, jumping, etc. The server will combine these basic action data with the corresponding basic prior knowledge and attention weight matrix for processing. Then, the output of the first-level classification unit will be coordinated with the action behavior prior knowledge corresponding to the second-level action behavior classification unit (such as the conversion mode between various basic actions) as the input of the second-level classification unit. The second-level classification unit can further analyze the combination and sequence of these basic actions to identify more complex action sequences. Finally, the third-level classification unit can perform higher-level behavior recognition based on the outputs of the first two levels and the basic prior knowledge and attention weight matrix of this level, such as judging whether a person's behavior complies with the safety regulations of the park. Through this multi-level classification approach, the server can more accurately understand and classify various behaviors within the park, thereby providing more precise and reliable support for property security control.

[0097] In an embodiment of the present invention, the characteristic distance between the local monitoring image feature representation and each action behavior feature representation in the multiple action behavior feature representations is determined by the target park behavior classification model, and the target park behavior type indicated by the action behavior feature representation whose characteristic distance satisfies the preset characteristic distance limit is taken as the park behavior type corresponding to the local monitoring image feature representation. It can be implemented through the following examples.

[0098] Acquire a first Mahalanobis distance between the local monitoring image feature representation and a first motion behavior feature representation and a second Mahalanobis distance between the local monitoring image feature representation and a second motion behavior feature representation, wherein the multiple motion behavior feature representations include the first motion behavior feature representation and the second motion behavior feature representation;

[0099] When the first Mahalanobis distance is lower than the Mahalanobis distance threshold, determining that the first action behavior feature representation is a target action behavior feature representation, and the park behavior type of the local monitoring image is the target park behavior type;

[0100] When the second Mahalanobis distance is higher than the Mahalanobis distance threshold, it is determined that the second action behavior characteristic representation is not the target action behavior characteristic representation, and the park behavior type of the local monitoring image is not the target park behavior type.

[0101] In an embodiment of the present invention, illustratively, the server first calculates the Mahalanobis distance between the local surveillance image feature representation and multiple action behavior feature representations. Take two action behavior feature representations as an example, which are respectively referred to as the first action behavior feature representation and the second action behavior feature representation. The server obtains the first Mahalanobis distance between the local surveillance image feature representation and the first action behavior feature representation, and the second Mahalanobis distance between the local surveillance image feature representation and the second action behavior feature representation by calculation. For example, suppose that the local surveillance image shows a person walking in the park, the first action behavior feature representation is "walking", and the second action behavior feature representation is "running". The server will calculate the Mahalanobis distance between the surveillance image of this walking and the two action behavior feature representations of "walking" and "running". The server will then compare the calculated Mahalanobis distance with a preset Mahalanobis distance threshold. This threshold is set according to actual needs and is used to determine which action behavior feature representation the surveillance image is more matched with. If the first Mahalanobis distance calculated by the server (i.e., the distance between the surveillance image of walking and the characteristic representation of the action behavior of "walking") is lower than the Mahalanobis distance threshold, it means that this surveillance image is very consistent with the action behavior of "walking". Based on the above comparison results, the server will determine the park behavior type of the local surveillance image. If the first Mahalanobis distance is lower than the threshold, the server will determine that the first action behavior characteristic representation (in this case, "walking") is the target action behavior characteristic representation, and determine the park behavior type of the local surveillance image as "walking". On the contrary, if the second Mahalanobis distance (i.e., the distance between the surveillance image of walking and the action behavior characteristic representation of "running") is higher than the Mahalanobis distance threshold, the server will determine that the second action behavior characteristic representation is not the target action behavior characteristic representation, so the park behavior type of the local surveillance image is not "running". In this way, the server can accurately identify the park behavior type displayed in the local surveillance image, providing important information for subsequent property safety prevention and control.

[0102] In the embodiments of the present invention, the following implementation modes are also provided.

[0103] In the case where the target park behavior type belongs to a dangerous park behavior type, determining that the local monitoring image is a dangerous monitoring image, wherein the preset park behavior type includes the dangerous park behavior type;

[0104] If it is determined that the local monitoring image is the dangerous monitoring image, dangerous prior knowledge is generated, wherein the dangerous prior knowledge represents that the local monitoring image contains dangerous behavior.

[0105] In an embodiment of the present invention, exemplarily, after determining the park behavior type of a local surveillance image, the server will further determine whether this behavior type belongs to a dangerous park behavior type. The dangerous park behavior type is pre-set and may include but is not limited to illegal intrusion, vandalism, fighting, etc. For example, assume that the server determines through the previous steps that the park behavior type of a local surveillance image is "illegal intrusion", which is in the preset list of dangerous park behavior types. Since "illegal intrusion" belongs to the dangerous park behavior type, the server will immediately determine that this local surveillance image is a dangerous surveillance image. This means that the behavior recorded in the surveillance image may pose a threat to the safety of the park. Once it is determined that the local surveillance image is a dangerous surveillance image, the server will immediately generate danger prior knowledge. This process is automated and aims to quickly identify surveillance images containing dangerous behaviors. Danger prior knowledge is a special data structure that contains information such as descriptions of dangerous behaviors, timestamps, locations, and possible threat levels. This knowledge can be used for subsequent security analysis, early warning systems, or as training data to optimize security monitoring models. For example, in this "illegal intrusion" scenario, the risk prior knowledge can include the following information: "At XX:XX on XX / XX / XX, an illegal intrusion was detected in the northwest corner of the park, and the threat level is high." Such information is very valuable to security personnel because it can help them respond quickly and prevent potential security incidents. In this way, the server can identify and process dangerous surveillance images in real time, providing strong support for the security management of the park.

[0106] In the embodiments of the present invention, the following implementation modes are also provided.

[0107] Performing tensor product processing on the basic prior knowledge and the monitoring image attention weight matrix to obtain the monitoring image prior knowledge, wherein the attention weight matrix includes the monitoring image attention weight matrix;

[0108] The tensor product processing is performed on the basic prior knowledge and the action behavior attention weight matrix to obtain the action behavior prior knowledge, wherein the attention weight matrix includes the action behavior attention weight matrix.

[0109] In an embodiment of the present invention, exemplarily, the server first obtains basic prior knowledge and a monitoring image attention weight matrix. Basic prior knowledge may include general information about common elements and events in monitoring scenes, while the monitoring image attention weight matrix reflects the importance of different areas or features in monitoring images. For example, in a park monitoring scene, basic prior knowledge may include rules such as "pedestrians usually walk on the sidewalk" and "vehicles usually drive on the road". The monitoring image attention weight matrix can assign higher weights to areas with high traffic or key facilities (such as gates, important equipment). The server performs tensor product processing on the basic prior knowledge and the monitoring image attention weight matrix. This processing method is actually a weighted fusion, which combines the universality of basic prior knowledge and the specificity of the monitoring image attention weight matrix, thereby generating prior knowledge for a specific monitoring image, that is, monitoring image prior knowledge. Continuing the above example, through tensor product processing, the server can conclude that in a specific monitoring image, "pedestrians on the sidewalk" and "vehicles on the road" are more important features because these areas are given higher attention weights. Similar to the processing of surveillance image prior knowledge, the server also obtains basic prior knowledge related to action behaviors and an attention weight matrix for action behaviors. Basic prior knowledge can include general rules and features of various action behaviors, while the attention weight matrix for action behaviors reflects the importance of different action or behavior features in the recognition process. For example, when identifying whether a person is running, features such as "fast leg movement" and "body leaning forward" can be given higher attention weights. The server performs tensor product processing on these two parts of information to generate prior knowledge for specific action behaviors, namely, action behavior prior knowledge. This knowledge combines the general rules of action behaviors with the importance weights in specific scenarios, which helps to more accurately identify and classify action behaviors. Through the above steps, the server can use tensor product processing to effectively combine basic prior knowledge and attention weight matrices to generate more targeted and accurate surveillance image prior knowledge and action behavior prior knowledge, thereby improving the recognition and classification performance of subsequent surveillance images and action behaviors.

[0110] The embodiment of the present invention provides a computer device 100, which includes a processor and a non-volatile memory storing computer instructions. When the computer instructions are executed by the processor, the computer device 100 executes the aforementioned property safety prevention and control method based on artificial intelligence. Figure 2 As shown, Figure 2The block diagram of the computer device 100 provided in the embodiment of the present invention. The computer device 100 includes a memory 111, a processor 112 and a communication unit 113. To achieve data transmission or interaction, the memory 111, the processor 112 and the communication unit 113 are directly or indirectly electrically connected to each other. For example, these components can be electrically connected to each other through one or more communication buses or signal lines.

[0111] For illustrative purposes, the foregoing description is made with reference to specific embodiments. However, the above illustrative discussion is not intended to be exhaustive or to limit the present disclosure to the precise form disclosed. Numerous modifications and variations are possible in accordance with the above teachings. These embodiments are selected and described in order to best illustrate the principles of the present disclosure and its practical application, so that those skilled in the art can best utilize the present disclosure and utilize various embodiments with different modifications to suit the intended specific application.

Claims

1. A property safety control method based on artificial intelligence, characterized in that: include: Get the current surveillance video; Input the current surveillance video into a pre-trained target video frame analysis model to obtain image information of people in the current surveillance video; Extracting a partial surveillance image including the person from the current surveillance video according to the person image information; Inputting the local monitoring image into a pre-trained target park behavior classification model to obtain a park behavior type recognition result; When the park behavior type identification result is characterized as abnormal behavior, the current monitoring video is sent to the security server and a corresponding alarm message is generated; The target video frame analysis model is obtained by the following method, including: Obtain sample surveillance video; Sampling the sample surveillance video to obtain a plurality of sample video frames; According to each local feature focuser in the preset encoder, a feature extraction operation is performed on each video segment region in a first sample video frame to obtain a video frame feature of each video segment region in the first sample video frame, where the first sample video frame is any one of the multiple sample video frames; performing an integration operation on the video frame features of each of the video clip regions in the first sample video frame according to the feature fusion unit in the preset encoder to obtain the video frame features of the first sample video frame; Performing an integration operation on the video frame features of the plurality of sample video frames to obtain a video frame feature set of the sample surveillance video; Inputting the video frame feature set of the sample surveillance video, the time sequence information of multiple character ranges, and the target value of the character information set into the initial video frame analysis model, wherein the target value of the character information set includes the coordinate information target value of each character range and the character image information target value corresponding to each character range; The following steps are performed by the initial video frame analysis model to obtain a predicted person information set of the sample surveillance video: For a target person range, the coordinate information of the target person range is determined according to the coordinate information target values ​​of all person ranges arranged before the target person range according to the time sequence information and the person image information target values ​​of all person ranges; the initial video frame analysis model includes a plurality of state space models; For a target person range, the coordinate information of the target person range, the target values ​​of the coordinate information of all person ranges arranged before the target person range, and the target values ​​of the person image information of all person ranges are processed according to the multimodal fusion unit in the last state space model to obtain a fusion feature; Performing feature standardization on the fused features according to the feature standardization unit in the state space model of the last position to obtain a standardized feature; The standardized features are processed according to the feature parsing layer in the state-space model of the last position to obtain the recognition confidence of the target person range, the recognition confidence of the target person range is used to characterize the current person image information corresponding to the target person range, the person information set includes the coordinate information of each person range in a plurality of person ranges and the person image information corresponding to each person range, the coordinate information of each person range is used to characterize the position of the corresponding person range in the sample monitoring video; Calculating a deviation between the character information set and a target value of the character information set to obtain a cost parameter; The initial video frame analysis model is trained according to the cost parameter to obtain a target video frame analysis model, and the target video frame analysis model is used to process the current monitoring video to obtain the character image information in the current monitoring video.

2. The method according to claim 1, characterized in that The inputting of the local monitoring image into the pre-trained target park behavior classification model to obtain the park behavior type recognition result includes: Acquire a local monitoring image and a plurality of predefined motion behavior data, wherein one of the plurality of motion behavior data is used to describe a park behavior type in a preset park behavior type; Input the local monitoring image and the multiple action behavior data into a pre-trained target park behavior classification model to obtain a local monitoring image feature representation and multiple action behavior feature representations, wherein the target park behavior classification model includes an action behavior conversion network, a monitoring image conversion network, action behavior prior knowledge and monitoring image prior knowledge, the action behavior prior knowledge and the monitoring image prior knowledge are generated by basic prior knowledge, the monitoring image prior knowledge is used to be input into the monitoring image conversion network in coordination with the local monitoring image to obtain the local monitoring image feature representation, and the action behavior prior knowledge is used to be input into the action behavior conversion network in coordination with the multiple action behavior data to obtain the multiple action behavior feature representations; Acquire a first Mahalanobis distance between the local monitoring image feature representation and a first motion behavior feature representation and a second Mahalanobis distance between the local monitoring image feature representation and a second motion behavior feature representation, wherein the multiple motion behavior feature representations include the first motion behavior feature representation and the second motion behavior feature representation; When the first Mahalanobis distance is lower than the Mahalanobis distance threshold, determining that the first action behavior feature representation is a target action behavior feature representation, and the park behavior type of the local monitoring image is the target park behavior type; When the second Mahalanobis distance is higher than the Mahalanobis distance threshold, it is determined that the second action behavior characteristic representation is not the target action behavior characteristic representation, and the park behavior type of the local monitoring image is not the target park behavior type; The park behavior type is used as the park behavior type recognition result of the local monitoring image.

3. The method according to claim 2, characterized in that The inputting of the local monitoring image and the multiple action behavior data into the pre-trained target park behavior classification model to obtain the local monitoring image feature representation and the multiple action behavior feature representations includes: Performing attention weighting processing on the basic prior knowledge and the monitoring image attention weight matrix to determine the monitoring image prior knowledge, wherein the attention weighting processing is used to take the eigenvalue of the monitoring image prior knowledge corresponding to the first characteristic dimension in the matrix as the weighted sum of the eigenvalue on the first characteristic dimension in the basic prior knowledge and the eigenvalue on the first characteristic dimension in the monitoring image attention weight matrix; Cutting the local surveillance image into a plurality of surveillance image segments; Projecting the plurality of surveillance image segments onto a surveillance image feature layer to obtain a plurality of segment feature codes; Performing feature integration processing on the plurality of segment feature codes and the monitoring image prior knowledge to determine the monitoring image integrated feature vector; The integrated feature vector features of the monitoring image are converted to a target feature layer to determine the feature representation of the local monitoring image.

4. The method according to claim 2, characterized in that: The plurality of action behavior data are input into a pre-trained target park behavior classification model to obtain a plurality of action behavior feature representations, including: Performing attention weighting processing on the basic prior knowledge and the action behavior attention weight matrix to determine the action behavior prior knowledge, wherein the attention weighting processing is used to take the eigenvalue of the action behavior prior knowledge corresponding to the second characteristic dimension in the matrix as the weighted sum of the eigenvalue on the second characteristic dimension in the basic prior knowledge and the eigenvalue on the second characteristic dimension in the action behavior attention weight matrix; Based on the following steps, the plurality of action behavior data and the action behavior prior knowledge are respectively subjected to feature integration processing to determine a plurality of action behavior integrated feature vectors, wherein the action behavior data subjected to feature integration processing each time is determined as candidate action behavior data, and the action behavior integrated feature vector obtained by the feature integration processing each time is determined as a candidate action behavior integrated feature vector: Performing sub-action data action splitting processing on the candidate action behavior data to obtain multiple sub-action data; Projecting the plurality of sub-action data to the action behavior feature layer to obtain the candidate action behavior integrated feature vector; The plurality of action behavior integrated feature vectors are respectively feature-converted to a target feature layer to determine the plurality of action behavior feature representations.

5. The method according to claim 2, characterized in that: The method further comprises: Acquire a monitoring image instance and a plurality of action behavior data instances, wherein the monitoring image instance is pre-assigned to correspond to a target action behavior data instance, and the plurality of action behavior data instances include the target action behavior data instance; Obtaining native basic prior knowledge, native monitoring image attention weight matrix, and native action behavior attention weight matrix, wherein the native basic prior knowledge, the native monitoring image attention weight matrix, and the native action behavior attention weight matrix are tuning parameters for the training process of optimizing the native park behavior classification model; Performing attention weighting processing on the native basic prior knowledge and the native monitoring image attention weight matrix to determine the native monitoring image prior knowledge, wherein the attention weighting processing is used to take the eigenvalue of the native monitoring image prior knowledge corresponding to the third characteristic dimension in the matrix as the weighted sum of the eigenvalue on the third characteristic dimension in the native basic prior knowledge and the eigenvalue on the third characteristic dimension in the native monitoring image attention weight matrix; Performing the attention weighting processing on the native basic prior knowledge and the native action behavior attention weight matrix to determine the native action behavior prior knowledge, wherein the attention weighting processing is used to take the eigenvalue of the native action behavior prior knowledge corresponding to the fourth characteristic dimension in the matrix as the weighted sum of the eigenvalue on the fourth characteristic dimension in the native basic prior knowledge and the eigenvalue on the fourth characteristic dimension in the native action behavior attention weight matrix; Performing feature integration processing on the surveillance image instance and the native surveillance image prior knowledge to determine a surveillance image instance integrated feature vector, and performing feature integration processing on the multiple action behavior data instances and the native action behavior prior knowledge respectively to determine multiple sample action behavior integrated feature vectors; Convert the monitoring image instance integrated feature vector and the multiple sample action behavior integrated feature vector features to the native feature layer, determine the monitoring image instance feature representation and the multiple action behavior feature representation instances, wherein the native campus behavior classification model includes the action behavior conversion network, the monitoring image conversion network, native action behavior prior knowledge and native monitoring image prior knowledge, and the native action behavior prior knowledge and the native monitoring image prior knowledge are generated by native basic prior knowledge; Determine, by means of the native park behavior classification model, a sample feature distance between the monitoring image instance feature representation and each action behavior feature representation instance in the plurality of action behavior feature representation instances; The error parameter is calculated based on the sample feature distance, and the native basic prior knowledge and the native attention weight matrix are optimized using the error parameter until the preset training stop condition is met to obtain the target park behavior classification model.

6. The method according to claim 2, characterized in that The inputting of the local monitoring image and the multiple action behavior data into the pre-trained target park behavior classification model to obtain the local monitoring image feature representation and the multiple action behavior feature representations includes: In the case where the target park behavior classification model includes a multi-level surveillance image classification unit, the corresponding basic prior knowledge and attention weight matrix are allocated to the surveillance image classification units at each level, wherein the output of the n-th level surveillance image classification unit in the multi-level surveillance image classification unit and the surveillance image prior knowledge corresponding to the n+1-th level surveillance image classification unit are coordinated as the input of the n+1-th level surveillance image classification unit, and n is a positive integer greater than 1; In the case where the target park behavior classification model includes multi-level action behavior classification units, the corresponding basic prior knowledge and the attention weight matrix are allocated to the action behavior classification units at each level, wherein the output of the n-th level action behavior classification unit in the multi-level action behavior classification units and the action behavior prior knowledge corresponding to the n+1-th level action behavior classification unit are coordinated as the input of the n+1-th level action behavior classification unit.

7. The method according to claim 2, characterized in that: The method further comprises: In the case where the target park behavior type belongs to a dangerous park behavior type, determining that the local monitoring image is a dangerous monitoring image, wherein the preset park behavior type includes the dangerous park behavior type; If it is determined that the local monitoring image is the dangerous monitoring image, dangerous prior knowledge is generated, wherein the dangerous prior knowledge represents that the local monitoring image contains dangerous behavior.

8. The method according to claim 2, characterized in that: The method further comprises: Performing tensor product processing on the basic prior knowledge and the monitoring image attention weight matrix to obtain the monitoring image prior knowledge, wherein the attention weight matrix includes the monitoring image attention weight matrix; The tensor product processing is performed on the basic prior knowledge and the action behavior attention weight matrix to obtain the action behavior prior knowledge, wherein the attention weight matrix includes the action behavior attention weight matrix.

9. A server system, characterized in that: The method comprises a server, wherein the server is used to execute the method described in any one of claims 1 to 8.

Images