Intelligent security management system and method based on image recognition

Abstract

The application provides an intelligent security management system and method based on image recognition, and relates to the technical field of intelligent communities.The application sets a public security identification model, preliminarily detects the real-time images of each camera in the intelligent community, obtains suspicious images that may exist in behaviors that endanger public security, and realizes rapid identification of behaviors that endanger public security.Further, the application uses a target identification model that matches the public security type and environmental parameters to perform secondary identification on the suspicious images and video data thereof, determines the identification result, so that the target identification model can more accurately identify the cause of events that endanger public security and perform responsibility determination, and realizes accurate determination of behaviors that endanger public security.Compared with the mode of manual alarm and on-site processing by police officers, the application can automatically, quickly and accurately identify behaviors that endanger public security through preliminary identification and secondary identification by image recognition technology, and improves the security management efficiency of the intelligent community.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of smart community, and particularly relates to a smart security management system and method based on image recognition. BACKGROUND

[0002] With the acceleration of urbanization and the deepening of smart community construction, public security prevention and management has become an indispensable core element in community governance. However, the traditional security management mode relies too much on manual monitoring and patrol, which not only is inefficient, but also has obvious limitations in monitoring coverage, making it difficult to capture and effectively deal with potential public security risks in a timely manner.

[0003] In current security practice, once a security event occurs, it often relies on the traditional way of manual alarm and on-site disposal by police officers. This processing method not only significantly prolongs the processing time of the event, but also greatly reduces the processing efficiency, making it difficult to eliminate public security risks in a timely and effective manner, which brings not small hidden dangers to community safety.

[0004] More seriously, when the police officers arrive at the scene, in order to fully understand the event situation and make accurate responsibility determination, they need to conduct tedious inquiry and mediation work, and return to the monitoring center to manually retrieve relevant monitoring videos for detailed manual analysis. This process not only consumes time and effort, but also further prolongs the processing cycle, seriously restricting the improvement of processing efficiency.

[0005] In summary, the current public security prevention and management is faced with the problems of low efficiency and long processing cycle, which is difficult to meet the urgent needs of modern smart community for public security prevention and management. SUMMARY

[0006] The present application provides a smart security management system and method based on image recognition, which can automatically realize rapid identification and accurate determination of dangerous public security behavior, and improve the security management efficiency of smart community.

[0007] In the first aspect, the present application provides a smart security management method based on image recognition, which comprises: acquiring real-time images of multiple cameras in a smart community; determining a suspicious image and a public security type corresponding to the suspicious image based on the real-time images and a preset public security identification model, the public security identification model being used to identify suspicious images that may exist dangerous public security behavior; the public security type includes fire, criminal, public order and traffic; acquiring video data and environmental parameters corresponding to the suspicious image; performing model matching based on the public security type and the environmental parameters to determine a target identification model; determining an identification result of the suspicious image based on the suspicious image, the video data and the target identification model, the identification result including an event cause and a responsibility determination result; and performing smart security management based on the identification result of the suspicious image.

[0008] In a possible implementation, the suspicious image and the public security type corresponding to the suspicious image are determined based on the real-time images and the preset public security identification model, including: performing feature conversion based on the real-time images to determine feature vectors of the real-time images; determining output results corresponding to the real-time images based on the feature vectors of the real-time images and the preset public security identification model, the output results including probabilities that the real-time images exist in each type of public security; determining the suspicious image based on the output results corresponding to the real-time images and a preset probability threshold; and determining the public security type corresponding to each suspicious image based on the output results corresponding to the suspicious image.

[0009] In a possible implementation, the target identification model is determined by performing model matching based on the public security type and the environmental parameter, including: determining a sub-model related to the public security type from a plurality of sub-models of the image identification model based on the public security type and a mapping relationship between the public security type and a model type; performing correlation analysis based on the public security type and the environmental parameter to determine an environmental condition related to the public security type; the environmental parameter includes temperature, humidity, illumination, and crowd flow; determining a model parameter of the sub-model related to the public security type based on the environmental condition related to the public security type; and determining the target identification model based on the sub-model related to the public security type and the model parameter.

[0010] In a possible implementation, the identification result of the suspicious image is determined based on the suspicious image, the video data, and the target identification model, including: performing frame processing on the video data to obtain a frame image sequence arranged in chronological order; performing feature extraction based on the suspicious image and the frame image sequence to determine a to-be-identified feature vector; and inputting the to-be-identified feature vector into the target identification model to obtain the identification result of the suspicious image.

[0011] In a possible implementation, the intelligent security management is performed based on the identification result, including: if the identification type is criminal or fire, storing event data of the suspicious image and generating first alarm information, the first alarm information being used to instruct a center user to confirm a criminal event or a fire event; if the identification type is public order or traffic, storing event data of the suspicious image and generating second alarm information, the second alarm information being used to instruct the center user to evaluate an influence range of a public order event or a traffic event; receiving an evaluation result input by the center user, the evaluation result including a first influence range or a second influence range; if the evaluation result is the first influence range, generating an event processing instruction, the event processing instruction being used to instruct a patrol user to perform on-site mediation processing; and sending the event processing instruction to a processing terminal of the patrol user.

[0012] In a possible implementation, based on the recognition result, the intelligent security management is performed, including: in response to a screening operation instruction of the patrol user on the intelligent security terminal, screening a plurality of suspect images stored in the database based on the screening operation instruction to obtain one or more candidate images; the screening operation instruction includes a query location and a query time period selected by the patrol user; displaying the one or more candidate images; in response to a selection instruction of the patrol user on the one or more candidate images; based on the selection instruction, querying the database to obtain event data of a target suspect image, the event data including the target suspect image, video data, a public security type, an environmental parameter, and the recognition result; the selection instruction includes a target suspect image selected by the patrol user; and displaying the event data of the target suspect image.

[0013] In a possible implementation, the method further includes: receiving a target query instruction of the patrol user, the target query instruction being used to instruct to query a target person or a target object; the target query instruction including a real-time image of the target person or the target object; based on the target query instruction, querying the target person or the target object to obtain a query report; and displaying the query report.

[0014] In a possible implementation, based on the target query instruction, the target person or the target object is queried to obtain a query report, including: based on the target query instruction, extracting a feature of the target person or the target object; based on the target query instruction, performing model matching to determine a target query model; based on the real-time image of the target person or the target object, determining a query range and a plurality of cameras in the query range; obtaining real-time images of the plurality of cameras in the query range; based on the real-time images of the plurality of cameras in the query range, the feature of the target person or the target object, and the target query model, performing traversal matching to determine a query result, the query result including a plurality of target real-time images related to the target person or the target object; based on camera positions corresponding to the plurality of target real-time images and a map of the query range, drawing a trajectory graph of the target person or the target object; and based on the query result and the trajectory graph, generating the query report.

[0015] In a possible implementation, before the suspect image and the public security type corresponding to the suspect image are determined based on the real-time image and the preset public security recognition model, the method further includes: obtaining typical images of various public security type events occurred in a historical period and public security types corresponding to the typical images; based on the typical images of the various public security type events, performing feature conversion to obtain feature vectors of the typical images; taking the feature vectors of the typical images as inputs and taking the public security types corresponding to the typical images as outputs to generate a plurality of first training samples; and based on the plurality of first training samples, performing neural network training to obtain the public security recognition model.

[0016] In a possible implementation, before the model matching based on the public security type and the environment parameter and determining the target identification model, the method further includes: obtaining processing result data of various events occurred in a historical period, the processing result data including real-time images, video data, public security types, environment parameters, event causes, and responsibility determination results of various events handled by the patrol user in the patrol process; performing clustering analysis based on the public security type and the environment parameter to determine a plurality of environment conditions corresponding to each public security type, wherein any environment condition is a combination of one or more environment parameters; splitting the processing result data of various events based on the public security type and the environment parameter to obtain processing result data corresponding to various environment conditions under each public security type; for each environment condition under any public security type, generating a plurality of input feature vectors corresponding to the environment condition under the public security type based on real-time images and video data corresponding to the environment condition under the public security type; generating a plurality of output feature vectors corresponding to the environment condition under the public security type based on event causes and responsibility determination results corresponding to the environment condition under the public security type; generating a plurality of second training samples based on the plurality of input feature vectors and the plurality of output feature vectors; performing neural network training based on the plurality of second training samples to obtain a sub-model and model parameters corresponding to the environment condition under the public security type; performing neural network training based on the processing result data corresponding to various environment conditions under each public security type to obtain sub-models and model parameters corresponding to various environment conditions under each public security type; and fusing the sub-models and model parameters corresponding to various environment conditions under each public security type to obtain the image identification model.

[0017] In a possible implementation, the method further includes: obtaining processing result data of the intelligent security terminal in a historical period before the current moment, the processing result data including real-time images, video data, public security types, environment parameters, event causes, and responsibility determination results of various events handled by the patrol user in the patrol process; generating a plurality of first training samples and a plurality of second training samples based on the processing result data; updating the public security identification model based on the plurality of first training samples to obtain an updated public security identification model; updating each sub-model in the image identification model based on the plurality of second training samples to obtain an updated image identification model; and performing real-time monitoring on real-time images of the smart community based on the updated public security identification model and the image identification model.

[0018] In a second aspect, an embodiment of the present application provides an intelligent security management device based on image recognition, which comprises a communication module and a processing module. The communication module is configured to acquire real-time images of multiple cameras in an intelligent community. The processing module is configured to determine a suspicious image and a public security type corresponding to the suspicious image based on the real-time images and a preset public security identification model. The public security identification model is configured to identify the suspicious image that may exist in a behavior of endangering public security. The public security type includes fire, criminal, public order and traffic. The communication module is further configured to acquire video data and environmental parameters corresponding to the suspicious image. The processing module is further configured to determine a target identification model based on the public security type and the environmental parameters. The processing module is further configured to determine an identification result of the suspicious image based on the suspicious image, the video data and the target identification model. The identification result includes an event cause and a responsibility determination result. The intelligent security management is performed based on the identification result of the suspicious image.

[0019] In a third aspect, an embodiment of the present application provides an intelligent security management system based on image recognition, which comprises an electronic device. The electronic device comprises a memory and a processor. The memory stores a computer program. The processor is configured to call and run the computer program stored in the memory to perform the steps of the method according to the first aspect and any possible implementation manner of the first aspect.

[0020] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium. The computer readable storage medium stores a computer program. When the computer program is executed by a processor, the steps of the method according to the first aspect and any possible implementation manner of the first aspect are implemented.

[0021] The present application provides an intelligent security management system and method based on image recognition. The present application detects real-time images of cameras in an intelligent community by setting a public security identification model to obtain suspicious images that may exist in behaviors of endangering public security, and realizes rapid identification of behaviors of endangering public security. Further, the present application performs secondary identification of suspicious images and video data thereof by using a target identification model matched with a public security type and environmental parameters to determine an identification result, so that the target identification model can more accurately identify an event cause of behaviors of endangering public security and perform responsibility determination, and accurate determination of behaviors of endangering public security is realized. Compared with an artificial alarm and a mode of on-site processing by a police officer, the present application can automatically, rapidly and accurately identify behaviors of endangering public security by using image recognition technology for preliminary identification and secondary identification, and improves the efficiency of security management in an intelligent community. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description only constitute some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0023] Figure 1 is a flow diagram of a smart security management method based on image recognition provided by an embodiment of the present application;

[0024] Figure 2 is a structural diagram of a smart security management device based on image recognition provided by an embodiment of the present application;

[0025] Figure 3 is a structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0026] In the following description, specific details are set forth in order to provide a thorough understanding of embodiments of the present application. However, persons of ordinary skill in the art will readily recognize that embodiments of the present application can be practiced without these specific details. In other instances, well-known structures, devices, circuits, and methods have not been described in detail in order to avoid obscuring the present application.

[0027] In the description of the present application, unless otherwise specified, " / " means "or", for example, A / B can mean A or B. "And / or" in this paper is only a description of the relationship between the associated objects, which means that there can be three relationships, for example, A and / or B, which can mean that A exists alone, A and B exist together, and B exists alone. In addition, "at least one" "multiple" means two or more. "First", "second", etc. are not limited in quantity and execution order, and "first", "second", etc. are not necessarily different.

[0028] In the embodiments of the present application, the words "exemplary" or "for example" are used to mean serving as an example, instance, or illustration. Any embodiment or design presented as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or advantageous than other embodiments or design solutions. Rather, the use of "exemplary" or "for example" is intended to present concepts in a concrete manner for ease of understanding.

[0029] Moreover, the terms "comprise", "comprising", "have", "having", "include", "including", "contain", "containing", "provide", "providing", "offer", "offering", "specify", "specifying", "invent", "inventing", "create", "creating" and any variations thereof in the description and in the claims shall not be construed as excluding any additional step or material. For example, a process, method, system, product or apparatus that comprises a list of steps or modules is not necessarily limited to those steps or modules which have been expressly identified.

[0030] For the purpose of making the objects, technical solutions and advantages of the present application clearer, the following will be described with specific embodiments in conjunction with the accompanying drawings of the present application.

[0031] As described in the background, the current public security and prevention management of smart community has the technical problems of low processing efficiency and long processing period.

[0032] To solve the above technical problems, as shown in the description of the present application, Figure 1 The embodiment of the present application provides a smart security and prevention management method based on image recognition. The method comprises steps S101-S105.

[0033] S101, acquiring real-time images of multiple cameras in a smart community.

[0034] In some embodiments, the cameras can transmit real-time images to the control center through wired or wireless networks, ensuring network stability and avoiding image delay or loss. The control center receives real-time images from each camera and performs preliminary processing such as format conversion and clarity optimization.

[0035] S102, based on the real-time images and a preset public security recognition model, determining a suspicious image and a public security type corresponding to the suspicious image.

[0036] In the embodiment of the present application, the public security recognition model is used to identify suspicious images that may exist in behaviors that endanger public security. The public security types include fire, criminal, public order and traffic.

[0037] In some embodiments, the public security recognition model is obtained by neural network training based on typical images of various public security type events occurring in historical periods.

[0038] As a possible implementation manner, step S102 can be implemented as steps S1021-S1024.

[0039] S1021, based on the real-time images, performing feature conversion to determine the feature vectors of each real-time image.

[0040] Exemplarily, the embodiments of the present application can preprocess real-time images, including denoising, grayscale (if color images are not necessary feature sources), size adjustment, etc., to ensure consistency and accuracy of subsequent feature extraction. Key features in the images are extracted using image processing techniques (such as SIFT, SURF, HOG, etc.) or deep learning models (such as convolutional neural networks CNN). These features can include edges, textures, color distributions, shapes, etc. The extracted features are encoded into feature vectors. A feature vector is a high-dimensional array, where each element represents the value of the image in a certain feature dimension.

[0041] S1022, based on the feature vectors of each real-time image and the preset public security identification model, determine the output result corresponding to each real-time image.

[0042] In some embodiments, the output result includes the probability of the existence of each type of public security in the real-time image.

[0043] Exemplarily, the embodiments of the present application can load a pre-trained public security identification model. The model is a neural network or machine learning model that receives feature vectors as input and outputs a probability distribution of public security types (such as fire, criminal, public order, and traffic). The feature vectors of each real-time image are input into the public security identification model, and the model calculates the probability of the image belonging to each public security type based on these feature vectors. The model outputs a probability distribution vector, where each element represents the probability of the image belonging to the corresponding public security type. This vector is the output result corresponding to the real-time image.

[0044] S1023, based on the output result corresponding to each real-time image and the preset probability threshold, determine the suspect image.

[0045] Exemplarily, the embodiments of the present application can set one or more probability thresholds according to actual application scenarios and security requirements. These thresholds are used to determine whether an image may contain behavior that endangers public security. For the output result of each real-time image, check whether the probability of belonging to each public security type exceeds the preset threshold. If a probability exceeds the threshold, the image is marked as a suspect image. If the model supports multi-label classification (i.e., an image may belong to multiple public security types at the same time), a threshold needs to be set for each type, and the suspect image is screened accordingly.

[0046] S1024, based on the output result corresponding to each suspect image, determine the public security type corresponding to each suspect image.

[0047] For example, embodiments of the present invention can identify the police type with the highest probability in the output results for each suspect image and use that type as the corresponding police type for the suspect image. When more detailed judgment is required, the probability distribution in the output results of the suspect image can be analyzed, considering other high-probability police types and their relative relationships. Where possible, the accuracy of the model's judgment can be verified through manual review or additional sensor data.

[0048] S103. Obtain the video data and environmental parameters corresponding to the suspect image.

[0049] For example, embodiments of the present invention can extract video data within a time period corresponding to a suspected image from camera storage or live stream. Embodiments of the present invention can also collect environmental parameters at the time the suspected image occurred using sensors (such as temperature sensors, humidity sensors, light detectors, etc.).

[0050] S104. Based on the public security type and environmental parameters, perform model matching to determine the target recognition model.

[0051] As one possible implementation, step S104 can be specifically implemented as steps S1041-S1044.

[0052] S1041. Based on the public security type and the mapping relationship between the public security type and the model type, determine the sub-model related to the public security type from multiple sub-models of the image recognition model.

[0053] For example, embodiments of the present invention can establish a clear mapping table or database, which records the correspondence between different public security types (such as fire protection, criminal investigation, public order, and traffic) and various sub-models in the image recognition model. From the mapping table, one or more sub-models that match the current public security type are selected. These sub-models are specifically designed for a certain public security type and can more effectively process image features related to that type.

[0054] S1042. Based on the public security type and environmental parameters, conduct correlation analysis to determine the environmental conditions related to the public security type.

[0055] In some embodiments, environmental parameters include temperature, humidity, light intensity, and pedestrian traffic.

[0056] For example, embodiments of the present invention can collect environmental parameter data in real time using various sensors (such as temperature sensors, humidity sensors, light sensors, and people counters) installed in smart communities. Based on historical data and expert knowledge, association rules are defined between different public security types and environmental conditions. For example, fire safety types may be associated with environmental conditions such as high temperature and smoke concentration; public security types may be associated with conditions such as low light and high pedestrian traffic. The currently collected environmental parameters are matched with the association rules to determine the environmental conditions most relevant to the current public security type.

[0057] S1043. Based on the environmental conditions related to the public security type, determine the model parameters of the sub-models related to the public security type.

[0058] For example, embodiments of the present invention can establish a correspondence between environmental conditions and model parameters, where different environmental conditions correspond to different model parameters in the same sub-model.

[0059] S1044. Based on the sub-model related to the public security type and the model parameters, determine the target recognition model.

[0060] For example, embodiments of the present invention can apply model parameters to selected sub-models to configure and optimize them. This includes the weights of neural networks, parameters of image processing algorithms, etc.

[0061] S105. Based on the suspect image, video data, and target recognition model, determine the recognition result of the suspect image.

[0062] In this embodiment of the application, the identification results include the cause of the event and the result of liability determination.

[0063] As one possible implementation, step S105 can be specifically implemented as steps S1051-S1054.

[0064] S1051. Perform frame segmentation on the video data to obtain a sequence of frame images arranged in chronological order.

[0065] For example, embodiments of the present invention can decode video data, converting it from compressed formats (such as MP4, AVI, etc.) into raw image data sequences. This step is typically performed by video processing libraries (such as FFmpeg, OpenCV, etc.). In the decoded video data, frame images are extracted at fixed time intervals (e.g., N frames per second) or based on specific events (e.g., motion detection triggers). The extracted frame images are arranged in chronological order to form a frame image sequence. The extracted frame images are stored in memory or written to temporary files on disk for subsequent processing. The storage format can be the raw image format (such as BMP, PNG, etc.) or a compressed format (such as JPEG) to save storage space.

[0066] S1052. Based on the suspect image and the frame image sequence, perform feature extraction to determine the feature vector to be identified.

[0067] For example, embodiments of the present invention can find a region corresponding to a suspect image in a sequence of frame images. If the suspect image is a frame extracted from a video, that frame can be directly located; if the suspect image is a static image, the most matching frame needs to be found through similarity calculation. In the located frame image region, features identical to those in the suspect image are extracted. These include low-level features such as color, texture, shape, and edges, or higher-level features (such as features extracted through a deep learning model).

[0068] This invention can encode extracted features into feature vectors. For example, feature values ​​can be quantized into discrete numerical values ​​and combined into a high-dimensional array. For features extracted by a deep learning model, the feature vector may be the activation value of a certain layer in the network. The extracted feature vector is used as the feature vector to be identified and then input to the subsequent target recognition model.

[0069] S1053. Input the feature vector to be identified into the target recognition model to obtain the recognition result of the suspect image.

[0070] For example, embodiments of the present invention can load a previously determined target recognition model. The feature vectors to be recognized are input into the target recognition model, and the model calculates the recognition result based on these feature vectors. This includes steps such as forward propagation calculation, feature matching, and classification decision. The recognition result is parsed from the model's output, including specific information such as the event cause, responsibility determination, and behavior type. The recognition result is stored in a database or displayed on a user interface for subsequent analysis and processing.

[0071] S106. Based on the identification results of the suspect image, conduct intelligent security management.

[0072] As one possible implementation, step S105 can be specifically implemented as steps A1-A5.

[0073] A1. If the identification type is criminal or fire-related, store the event data of the suspect image and generate the first alarm information.

[0074] In some embodiments, the first alarm information is used to instruct the central user to confirm a criminal or fire incident.

[0075] In some embodiments, the control center determines the incident type. If the type is criminal or fire-related, it is considered an emergency; if the type is public order or traffic-related, it is considered a non-emergency but important incident.

[0076] For all identified suspect images and related events, the system needs to store event data. This includes, but is not limited to: the suspect image itself, the timestamp of the event, location information (such as camera location), identification type, and any additional information (such as environmental parameters, other sensor data, etc.). Event data should be stored in a secure and reliable database for subsequent querying and analysis.

[0077] For example, when the identification type is criminal or fire-related, the system immediately generates an initial alarm message. This message should include the urgency of the event, the location of the event, a brief description (such as "suspected criminal activity detected" or "fire source detected"), and clearly instruct the central user to confirm the event. The initial alarm message should be sent to the central user via a high-priority channel (such as SMS, instant messaging software, internal alarm system, etc.).

[0078] A2. If the identification type is public security or traffic, store the event data of the suspect image and generate a second alarm message.

[0079] In some embodiments, the second alarm information is used to instruct the central user to assess the scope of the impact of a security incident or a traffic incident.

[0080] For example, when the identification type is public security or traffic, the system generates a second alarm message. This message should include the type of event, the location of the event, a preliminary estimate of the possible impact range, etc., and instruct the central user to assess the impact range of the event. The second alarm message can be sent to the central user through a lower priority channel so that they can conduct the assessment while handling the emergency.

[0081] A3. Evaluation results input by the user at the receiving center.

[0082] In some embodiments, the assessment results include primary impact range or secondary impact range.

[0083] For example, after receiving an alarm message, the central user inputs the assessment results through a designated interface or system, based on the nature and severity of the event. The assessment results typically include a primary impact area (i.e., the area or population directly affected by the event) or a secondary impact area (i.e., the area or population that may be indirectly affected due to the spread of the event). Upon receiving the assessment results, the system determines the next steps based on the assessment level and type.

[0084] A4. If the assessment result is a Level 1 impact range, then generate an event handling instruction.

[0085] In some embodiments, the event handling instruction is used to instruct the patrol user to perform on-site mediation.

[0086] A5. Send event processing instructions to the processing terminal of the patrol user.

[0087] For example, if the assessment result indicates a Level 1 impact area and the event type is public security or traffic, the system generates an event handling instruction. This instruction includes detailed event information, handling requirements (such as on-site mediation, traffic control, crowd management, etc.), and possible auxiliary information (such as a site map, the location of nearby patrol users, etc.). The system sends the event handling instruction to the processing terminal (such as a smartphone, tablet, walkie-talkie, etc.) of the patrol user responsible for the area via an internal communication system or a dedicated mobile application. When sending the instruction, the accuracy and timeliness of the information should be ensured so that the patrol user can respond and handle the event quickly.

[0088] As another possible implementation, step S105 can be specifically implemented as steps B1-B4.

[0089] B1. In response to the screening operation command of the patrol user on the intelligent security terminal, based on the screening operation command, the system filters multiple suspected images stored in the database to obtain one or more candidate images.

[0090] In some embodiments, the filtering operation instruction includes inspecting the query location and query time period selected by the user.

[0091] For example, patrol users input filtering instructions through the graphical user interface (GUI) of a smart security terminal (such as a mobile application, desktop software, or dedicated hardware terminal). The instructions include the user-selected query location (such as a specific street, neighborhood, or building) and query time period (such as date and time period). The control center receives these instructions and performs initial verification to ensure the validity and correct format of the input data.

[0092] The control center filters multiple suspect images stored in the database based on received filtering instructions. The filtering process may involve geographic location matching (ensuring the image is relevant to the selected location), timestamp matching (ensuring the image was captured within a selected time period), and possible additional conditions (such as specific police type or environmental parameter ranges). The filtering result may be one or more candidate images that highly match the patrol user's query criteria.

[0093] B2. Display one or more alternative images.

[0094] For example, the control center organizes the selected candidate images into a list or grid view, with basic information (such as capture time, location overview, etc.) displayed next to each image. The list or grid view should be easy to navigate, allowing patrol users to quickly browse and select images. The control center displays the candidate image list or grid view through the GUI of the smart security terminal. The display interface should be clear, intuitive, and include necessary navigation and selection controls.

[0095] B3. In response to the patrol user's selection instruction for one or more candidate images, based on the selection instruction, query the database to obtain event data of the target suspect image.

[0096] In some embodiments, event data includes target suspect images, video data, police type, environmental parameters, and identification results. Selection instructions include target suspect images selected by the patrol user.

[0097] For example, a patrol user selects one or more target suspect images from a list of candidate images or a grid view. Selection can be done by clicking, double-clicking, or dragging. The control center receives the selection instruction and records the selected target suspect images. Based on the selected target suspect images, the control center queries the database for corresponding event data. Event data includes the target suspect image itself, associated video data (which may be a complete video file or related clips), the public security type (e.g., criminal, public order, traffic, fire, etc.), environmental parameters (e.g., temperature, humidity, lighting, etc.), and previous identification results. The query process should be efficient and accurate, ensuring that users can quickly obtain the information they need.

[0098] B4. Event data that displays images of the target suspect.

[0099] For example, the system organizes the retrieved event data into an easy-to-understand and readable format. This may include image previews, a video player interface, detailed event descriptions, public security type labels, environmental parameter charts, and textual or graphical representations of the identification results. The system displays the event data of the target suspect image through the GUI of the intelligent security terminal. The display interface should contain all necessary information and be presented in an easy-to-navigate and readable manner. For video data, the system should provide playback controls (such as play, pause, fast forward, rewind, etc.) to allow patrol users to view the video content as needed.

[0100] This invention provides a smart security management method based on image recognition. By setting up a public safety identification model, it performs preliminary detection on real-time images from various cameras within a smart community to obtain suspected images of behaviors that may endanger public safety, achieving rapid identification of such behaviors. Furthermore, this invention uses a target identification model matched with the type of public security incident and environmental parameters to perform secondary identification on the suspected images and their video data, determining the identification results. This allows the target identification model to more accurately identify the cause of the event endangering public safety and determine responsibility, achieving accurate determination of behaviors endangering public security. Compared to manual alarms and on-site police handling, this invention, through preliminary and secondary identification using image recognition technology, can automatically, quickly, and accurately identify behaviors endangering public security, improving the efficiency of security management in smart communities.

[0101] Optionally, the image recognition-based smart security management method provided in this embodiment of the invention further includes steps S201-S203.

[0102] S201, Receive the target query command from the patrol user.

[0103] In some embodiments, a target query instruction is used to instruct a query for a target person or object.

[0104] For example, a target query instruction may include a real-time image of the target person or object.

[0105] For example, patrol users input target query commands through the graphical user interface (GUI) of a smart security terminal (such as a mobile application, desktop software, or dedicated hardware terminal). These commands are used to query specific target persons or objects, such as searching for missing persons, locating specific vehicles or items, etc. The user-input command includes a real-time image of the target person or object, which could be a scene image captured by a camera or a picture selected by the user from their photo album.

[0106] After receiving the user's input, the system first verifies the data to ensure its integrity and validity. Simultaneously, the system checks user permissions to ensure the user has the authority to perform this type of query.

[0107] S202. Based on the target query command, query the target personnel or target objects and obtain a query report.

[0108] As one possible implementation, step S202 can be specifically implemented as steps S2021-S2027.

[0109] S2021. Based on the target query command, extract the characteristics of the target person or target object.

[0110] For example, when a real-time image is received from a target query command, the system first uses image processing techniques to extract key features from the image. These features may include color, texture, shape, edges, and key points (such as corners and spots). For objects with specific structures, such as faces and license plates, the system may employ more advanced feature extraction methods, such as deep learning-based face recognition algorithms and license plate recognition algorithms.

[0111] S2022. Based on the target query command, perform model matching to determine the target query model.

[0112] For example, the target query model is one of the sub-models in the image recognition model. This sub-model achieves target matching query by comparing the feature similarity of specific regions of two images.

[0113] S2023. Based on real-time images of the target person or object, determine the query range and multiple cameras within the query range.

[0114] In some embodiments, the system determines the query range based on the geographic location information (such as latitude and longitude, address, etc.) in the target query instruction, combined with map data and camera layout information. This range may be a circular area, a rectangular area, or an area of ​​other shapes. The size of the query range may be adjusted according to factors such as the movement speed of the target person or object, the urgency of the query, etc. Within the determined query range, the system filters out all available cameras and acquires their real-time images or video streams. These cameras may come from different surveillance networks, such as urban traffic monitoring, community monitoring, commercial venue monitoring, etc.

[0115] S2024. Obtain real-time images from multiple cameras within the query range.

[0116] For example, the system can establish communication with selected cameras via a network connection to acquire their images or video streams in real time.

[0117] S2025. Based on real-time images from multiple cameras within the query range, the characteristics of the target person or object, and the target query model, perform traversal matching to determine the query results.

[0118] In some embodiments, the query results include multiple real-time images of targets related to the target person or object.

[0119] For example, the system uses a traversal matching approach, matching the features of the target person or object with real-time images from each camera within the query range. The matching process may involve steps such as feature comparison and similarity calculation. To improve matching efficiency, the system may employ optimization strategies such as parallel processing and block matching. Based on the matching results, the system selects multiple real-time images of targets related to the target person or object. These images may come from different cameras, recording the target's appearance at different times and locations.

[0120] S2026. Based on the camera locations corresponding to multiple real-time images of targets and the map of the query range, draw the trajectory map of the target personnel or target objects.

[0121] For example, the system draws a trajectory map of the target person or object based on the camera positions corresponding to multiple real-time images of the target and map information of the query range. This trajectory map may represent the target's movement path in the form of lines, arrows, etc. To improve the accuracy and readability of the trajectory map, the system may use algorithms such as map matching and path smoothing.

[0122] S2027. Based on the query results and trajectory map, generate a query report.

[0123] In some embodiments, the query report includes detailed information such as real-time images, trajectory maps, query time, query range, and camera information of multiple targets related to the target person or object. This information helps users gain a comprehensive understanding of the target's movement and location. The report may also include additional information such as the target's identification results (e.g., name, ID number), and records of relevant security incidents.

[0124] S203, Display query report.

[0125] In this way, the embodiments of the present invention can complete the query instructions of patrol users through image recognition and matching, providing convenience for the query of personnel and target objects, and improving the security management efficiency of smart communities.

[0126] Optionally, the image recognition-based smart security management method provided in this embodiment of the invention further includes steps S301-S304 before step S102.

[0127] S301. Obtain typical images of various types of public security incidents that occurred during historical periods, as well as the corresponding public security type for each typical image.

[0128] For example, embodiments of the present invention can directly obtain typical images from historical periods and their corresponding public security types from the database of the control center.

[0129] S302. Based on typical images of various types of public security incidents, perform feature transformation to obtain feature vectors for each typical image.

[0130] S303. Using the feature vectors of each typical image as input and the corresponding public security type of each typical image as output, generate multiple first training samples.

[0131] S304. Based on multiple first training samples, perform neural network training to obtain a public safety identification model.

[0132] For example, embodiments of the present invention may select a suitable neural network architecture, such as a convolutional neural network (CNN), a recurrent neural network (RNN), or a deep residual network (ResNet). The number of network layers, nodes, and other parameters are determined based on the complexity of the problem and the scale of the data. Training samples are input into the neural network, and prediction results are calculated through forward propagation. A loss function (such as cross-entropy loss) is used to measure the difference between the prediction results and the true labels. The weights and biases of the neural network are updated using a backpropagation algorithm to reduce the value of the loss function. The above process is repeated until the value of the loss function converges or reaches a preset number of training epochs. The trained model is evaluated using a validation set or test set to check its performance. Evaluation metrics include accuracy, recall, F1 score, etc. The model is optimized based on the evaluation results, such as adjusting the network structure, increasing training data, and using regularization techniques. The trained public safety identification model is saved to disk for subsequent use or deployment.

[0133] In this way, embodiments of the present invention can deploy a public security identification model through neural network training before image detection of smart communities, thereby facilitating the identification of suspicious images and public security types.

[0134] Optionally, the image recognition-based smart security management method provided in this embodiment of the invention further includes steps S401-S409 before step S104.

[0135] S401. Obtain the processing results data of various events that occurred during the historical period.

[0136] In some embodiments, the processing result data includes real-time images, video data, public security type, environmental parameters, incident cause, and responsibility determination results of various events handled by the patrol user during the patrol process.

[0137] For example, embodiments of the present invention can directly obtain the processing result data of various events from the database of the control center.

[0138] S402. Based on the public security type and environmental parameters, perform cluster analysis to determine the various environmental conditions corresponding to each public security type.

[0139] Any environmental condition is a combination of one or more environmental parameters.

[0140] For example, embodiments of the present invention can use clustering algorithms such as K-means, hierarchical clustering, and DBSCAN to cluster data based on public security type and environmental parameters. This identifies common combinations of environmental conditions for each public security type; for instance, traffic accidents are more common on urban roads in rainy weather and at night, while criminal offenses may occur at night in sparsely populated areas.

[0141] S403. Based on the public security type and environmental parameters, the processing result data of various types of events are split to obtain the processing result data corresponding to various environmental conditions under each public security type.

[0142] For example, the present invention can split the original data into data subsets under different environmental conditions for different public security types based on the clustering results. This ensures that each subset contains a sufficient number of samples for subsequent model training.

[0143] S404. For each environmental condition under any public security type, generate multiple input feature vectors corresponding to that environmental condition under that public security type based on the real-time image and video data corresponding to that environmental condition under that public security type.

[0144] For example, this invention can extract features such as color, texture, shape, and motion trajectory from real-time image and video data. The extracted features are encoded into vector form for input into a neural network. The most representative features are selected based on importance or relevance to reduce model complexity and improve training efficiency.

[0145] S405. Based on the event cause and responsibility determination results corresponding to the environmental conditions under the public security type, generate multiple output feature vectors corresponding to the environmental conditions under the public security type.

[0146] For example, the present invention can convert the cause of an event into a quantifiable feature vector, such as using one-hot encoding or word embedding techniques. The responsibility determination result can be converted into a feature vector, such as using numerical encoding to represent the responsible party and the degree of responsibility.

[0147] S406. Generate multiple second training samples based on multiple input feature vectors and multiple output feature vectors.

[0148] For example, the input feature vector and the output feature vector are combined to form a training sample, i.e., (input feature vector, output feature vector). Ensure that each sample is correctly labeled with the corresponding public security type, environmental conditions, event cause, and responsibility determination result.

[0149] S407. Based on multiple second training samples, perform neural network training to obtain the sub-model and model parameters corresponding to the environmental conditions under the public security type.

[0150] For example, this invention can select a suitable neural network architecture, such as a convolutional neural network (CNN) for image feature extraction, a recurrent neural network (RNN) for processing time series data, or a fully connected neural network (FNN) for classification and regression tasks. The neural network is trained using training samples, prediction results are calculated through forward propagation, and network weights and biases are updated through backpropagation. The model's performance, such as accuracy, recall, and F1 score, is evaluated using a validation set. Based on the evaluation results, the network structure, learning rate, regularization parameters, etc., are adjusted to improve model performance.

[0151] S408. Based on the processing result data corresponding to various environmental conditions under each type of public security, perform neural network training to obtain sub-models and model parameters corresponding to various environmental conditions under each type of public security.

[0152] S409. Based on the sub-models and model parameters corresponding to various environmental conditions under each type of public security, an image recognition model is obtained by fusion.

[0153] In this way, by setting up an image recognition model and its sub-models, the embodiments of the present invention realize image recognition and prediction under different public security types and environmental conditions, which facilitates intelligent security management and improves the security management efficiency of smart communities.

[0154] Optionally, the image recognition-based smart security management method provided in this embodiment of the invention further includes steps S501-S505.

[0155] S501. Obtain the processing result data of the intelligent security terminal in the historical period before the current moment.

[0156] In some embodiments, the processing result data includes real-time images, video data, public security type, environmental parameters, incident cause, and responsibility determination results of various events handled by the patrol user during the patrol process.

[0157] S502. Based on the processed data, generate multiple first training samples and multiple second training samples.

[0158] For example, the first training sample can extract real-time image and video data, along with corresponding public security type information, from the processed data. Feature extraction is performed on the image and video data to generate feature vectors. These feature vectors and the public security type information are then combined to form the first training sample, i.e., (feature vector, public security type).

[0159] For example, the second training sample can extract real-time image and video data, as well as the corresponding event cause and responsibility determination results, from the processed result data. Feature extraction is also performed on the image and video data to generate feature vectors. The feature vectors, event cause, and responsibility determination results are combined to form the second training sample, i.e., (feature vector, event cause, responsibility determination result).

[0160] S503. Based on multiple first training samples, update the public safety identification model to obtain the updated public safety identification model.

[0161] For example, an existing public safety identification model is loaded. The model is trained using the first training samples, predictions are computed through forward propagation, and model weights are updated through backpropagation. The model's performance, such as accuracy and recall, is evaluated using a validation set. Based on the evaluation results, if the model performance improves, the model is updated to a new public safety identification model.

[0162] S504. Based on multiple second training samples, update each sub-model in the image recognition model to obtain the updated image recognition model.

[0163] For example, load the individual sub-models from an existing image recognition model. For each sub-model (corresponding to different public security types and environmental conditions), train it using the corresponding second training samples. Evaluate the performance of each sub-model using a validation set. Based on the evaluation results, if the sub-model performance improves, update the corresponding sub-model. If the image recognition model uses an ensemble learning method to fuse multiple sub-models, the model fusion needs to be performed again to obtain the updated image recognition model.

[0164] S505, based on the updated public safety identification model and image recognition model, performs real-time monitoring of real-time images of the smart community.

[0165] For example, this invention can acquire image and video data in real time from cameras or other intelligent security terminals in a smart community. The real-time image and video data undergoes preprocessing, such as format conversion and noise removal. An updated public security identification model is used to identify the preprocessed images and determine their corresponding public security type. For specific public security type events identified, the updated image recognition model is used to further analyze the cause of the event and determine responsibility. The identification and analysis results are output to the monitoring center or the terminal devices of relevant personnel to enable timely response measures.

[0166] In this way, the embodiments of the present invention can periodically or in real time update the public safety identification model and the image recognition model, ensuring the accuracy of model recognition in the process of intelligent security management.

[0167] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0168] The following are device embodiments of the present invention. For details not described in detail, please refer to the corresponding method embodiments described above.

[0169] Figure 2 A schematic diagram of an image recognition-based smart security management device 600 provided by an embodiment of the present invention is shown. The smart security management device 600 includes a communication module 601 and a processing module 602.

[0170] The communication module 601 is used to acquire real-time images from multiple cameras in the smart community.

[0171] The processing module 602 is used to determine the suspected image and the corresponding public security type based on real-time images and a preset public security identification model. The public security identification model is used to identify suspected images that may involve acts that endanger public safety. The public security types include fire protection, criminal, public order, and traffic.

[0172] The communication module 601 is also used to acquire video data and environmental parameters corresponding to the suspect image.

[0173] The processing module 602 is also used to perform model matching based on public security type and environmental parameters to determine the target recognition model; based on the suspect image, video data, and target recognition model, determine the recognition result of the suspect image, including the cause of the incident and the responsibility determination result; and based on the recognition result of the suspect image, perform intelligent security management.

[0174] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. For example... Figure 3 As shown, the electronic device 700 includes: a processor 701, a memory 702, and a computer program 703 stored in the memory 702 and executable on the processor 701. When the processor 701 executes the computer program 703, it implements the steps in the above-described method embodiments, for example... Figure 1 The steps S101-S106 are shown. Alternatively, when the processor 701 executes the computer program 703, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 2 The functions of the communication module 601 and the processing module 602 shown are illustrated.

[0175] For example, the computer program 703 can be divided into one or more modules / units, which are stored in the memory 702 and executed by the processor 701 to complete the present invention. The one or more modules / units can be capable of performing specific tasks... Figure 3 A series of computer program instruction segments with defined functions, which describe the execution process of the computer program 703 in the electronic device 700. For example, the computer program 703 can be divided into... Figure 2 The communication module 601 and the processing module 602 are shown.

[0176] The processor 701 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0177] The memory 702 can be an internal storage unit of the electronic device 700, such as a hard disk or memory of the electronic device 700. The memory 702 can also be an external storage device of the electronic device 700, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the electronic device 700. Furthermore, the memory 702 can include both internal and external storage units of the electronic device 700. The memory 702 is used to store the computer program and other programs and data required by the terminal. The memory 702 can also be used to temporarily store data that has been output or will be output.

[0178] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A smart security management method based on image recognition, characterized in that, include: Acquire real-time images from multiple cameras in a smart community; Based on the real-time images and a preset public safety identification model, a suspect image and the corresponding public security type are determined. The public safety identification model is used to identify suspect images that may involve acts endangering public safety. The public security types include fire protection, criminal investigation, public order, and traffic. Obtain the video data and environmental parameters corresponding to the suspected image; Based on the public security type and the environmental parameters, model matching is performed to determine the target recognition model, including: based on the public security type and the mapping relationship between the public security type and the model type, determining the sub-model related to the public security type from multiple sub-models of the image recognition model; based on the public security type and the environmental parameters, performing correlation analysis to determine the environmental conditions related to the public security type; the environmental parameters include temperature, humidity, light intensity, and pedestrian traffic; based on the environmental conditions related to the public security type, determining the model parameters of the sub-model related to the public security type; and based on the sub-model related to the public security type and the model parameters, determining the target recognition model. Based on the suspected image, the video data, and the target recognition model, the recognition result of the suspected image is determined, and the recognition result includes the cause of the event and the responsibility determination result; Based on the identification results of the suspected images, intelligent security management is carried out.

2. The intelligent security management method based on image recognition according to claim 1, characterized in that, The process of determining a suspect image and its corresponding public security type based on the real-time image and a preset public security identification model includes: Based on the real-time images, feature transformation is performed to determine the feature vector of each real-time image; Based on the feature vectors of each real-time image and the preset public security identification model, the output results corresponding to each real-time image are determined, and the output results include the probability that the real-time image contains various types of public security. Based on the output results corresponding to each real-time image and the preset probability threshold, the suspect image is determined. Based on the output results corresponding to each suspect image, the corresponding public security type for each suspect image is determined.

3. The intelligent security management method based on image recognition according to claim 1, characterized in that, The step of determining the recognition result of the suspect image based on the suspect image, the video data, and the target recognition model includes: The video data is processed into frames to obtain a sequence of frame images arranged in chronological order. Based on the suspected image and the frame image sequence, feature extraction is performed to determine the feature vector to be identified; The feature vector to be identified is input into the target recognition model to obtain the recognition result of the suspect image.

4. The intelligent security management method based on image recognition according to claim 1, characterized in that, The intelligent security management based on the identification results of the suspected image includes: If the identification type is criminal or fire-related, the event data of the suspected image is stored, and a first alarm message is generated; the first alarm message is used to instruct the central user to confirm the criminal or fire-related event. If the identification type is public security or traffic, the event data of the suspected image is stored and a second alarm message is generated. The second alarm message is used to instruct the central user to assess the impact range of the public security or traffic incident. The evaluation results are input by the user at the receiving center; the evaluation results include the primary scope of impact or the secondary scope of impact. If the assessment result is within the first level of impact, an event handling instruction is generated, which is used to instruct the patrol user to conduct on-site mediation. Send the event processing command to the processing terminal of the patrol user.

5. The intelligent security management method based on image recognition according to claim 1, characterized in that, The intelligent security management based on the identification results of the suspected image includes: In response to the patrol user's filtering operation command on the intelligent security terminal, the system filters multiple suspected images stored in the database based on the filtering operation command to obtain one or more candidate images; the filtering operation command includes the query location and query time period selected by the patrol user. Display one or more alternative images; In response to the patrol user's selection instruction on one or more candidate images; based on the selection instruction, the database is queried to obtain event data of the target suspect image, the event data including the target suspect image, video data, public security type, environmental parameters and recognition results; the selection instruction includes the target suspect image selected by the patrol user; Event data that displays the image of the target suspect.

6. The intelligent security management method based on image recognition according to any one of claims 1 to 5, characterized in that, The method further includes: The system receives a target query instruction from a patrol user, which instructs the user to query a target person or object. The target query instruction includes a real-time image of the target person or object. Based on the target query command, a query is performed on the target person or target object to obtain a query report; Display the query report.

7. The intelligent security management method based on image recognition according to claim 6, characterized in that, The process of querying target personnel or objects based on the target query instruction and obtaining a query report includes: Based on the target query command, extract the features of the target person or target object; Based on the target query instruction, model matching is performed to determine the target query model; Based on the real-time images of the target person or object, the query range and multiple cameras within the query range are determined; Obtain real-time images from multiple cameras within the query range; Based on real-time images from multiple cameras within the query range, the characteristics of the target person or object, and the target query model, a traversal matching is performed to determine the query results, which include multiple real-time images of the target person or object related to the target. Based on the camera locations corresponding to the real-time images of the multiple targets, and the map of the query range, a trajectory map of the target person or object is drawn. Based on the query results and the trajectory map, a query report is generated.

8. The intelligent security management method based on image recognition according to any one of claims 1 to 5, characterized in that, The method further includes: The system acquires the processing result data of intelligent security terminals from the historical period prior to the current moment. The processing result data includes real-time images, video data, public security type, environmental parameters, event causes, and responsibility determination results of various events handled by the patrol user during the patrol process. Based on the processing result data, multiple first training samples and multiple second training samples are generated; Based on the multiple first training samples, the public safety identification model is updated to obtain the updated public safety identification model; Based on the multiple second training samples, each sub-model in the image recognition model is updated to obtain the updated image recognition model; Based on the updated public safety identification model and image recognition model, real-time monitoring of real-time images of the smart community is performed.

9. A smart security management system based on image recognition, characterized in that, The intelligent security management system includes an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor is used to call and run the computer program stored in the memory to perform the method as described in any one of claims 1 to 8.

Images