A personnel intrusion monitoring method for multi-service cooperative operation and related equipment

Abstract

The application discloses a personnel intrusion monitoring method for multi-service cooperative operation and related equipment, and the method comprises the following steps: in response to the channel identifier and the stream address of the camera issued by the front end, starting the inference service process of the corresponding channel based on the channel identifier, and obtaining real-time video frames from the stream address through the inference service process; based on the real-time video frames, using a pre-trained target detection model to perform personnel intrusion detection to obtain a detection result; when the detection result includes a target object with a category of intruding personnel, calling a large model based on the local image corresponding to the bounding box of the target object to verify and analyze to obtain a judgment text; and performing adaptive alarm processing according to the judgment text. The application can improve the resource utilization efficiency and system flexibility in a multi-path monitoring scene; and the two-stage personnel intrusion detection steps can effectively distinguish between ordinary personnel and uniformed staff to significantly reduce false positives, and can be widely applied in the field of image processing technology.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and related equipment for monitoring personnel intrusion in a multi-service collaborative operation. Background Technology

[0002] Detecting and identifying intruders in surveillance footage has become an important requirement for security management. However, traditional intrusion detection mainly relies on manual continuous monitoring of multiple surveillance feeds. This manual identification method is inefficient and prone to missed detections or misjudgments due to fatigue.

[0003] With the development of machine learning technology, existing technologies have further introduced target detection technology based on deep convolutional neural networks for personnel intrusion detection and identification. For example, single-stage detectors such as YOLO and SSD or two-stage detectors such as Faster R-CNN are used to achieve frame-by-frame identification and classification of personnel targets in surveillance video streams. However, there are still many shortcomings in practical applications: due to the limitations of the model's own discrimination ability, it may still generate a large number of false alarms in complex scenarios; in addition, the allocation of computing resources for each monitoring channel in a multi-channel video surveillance environment places extremely high demands on computing hardware, resulting in high system construction and maintenance costs. Summary of the Invention

[0004] The main objective of this invention is to provide a method, device, electronic device, storage medium, and program product for monitoring personnel intrusion in a multi-service collaborative operation, aiming to solve at least one problem of the prior art.

[0005] To achieve the above objectives, one aspect of this invention proposes a method for monitoring personnel intrusion during multi-service collaborative operations, the method comprising: In response to the camera channel identifier and stream address sent by the front end, the inference service process of the corresponding channel is started based on the channel identifier, and the real-time video frames are obtained from the stream address through the inference service process; Based on real-time video frames, a pre-trained target detection model is used to detect personnel intrusion and obtain detection results. The detection results include the bounding box and category of the target object, which is either an intruder or a non-intruder. When the detection results include a target object categorized as an intruder, the large model is invoked based on the local image corresponding to the bounding box of the target object to verify and analyze the judgment text, which is either "intrusion" or "security". Adaptive alert processing is performed based on the identified text.

[0006] In some embodiments, in response to the camera channel identifier and stream address sent by the front end, the inference service process for the corresponding channel is started based on the channel identifier, including the following steps: Obtain the camera's channel identifier and stream address sent by the front end through a preset communication protocol, and transmit them to the service management module; The service management module retrieves the code process identifier corresponding to the channel identifier by looking up the preset mapping table. Start the corresponding inference service process based on the code process identifier; The service management module continuously collects the running status information of the inference service process; If the running status information indicates that the inference service process is not performing the image detection task, the inference service process will be shut down according to the code process identifier. The process returns to the step of retrieving the code process identifier corresponding to the channel identifier from the preset mapping table through the service management module, in order to restart the inference service process until the running status information indicates that the inference service process is executing the image detection task.

[0007] In some embodiments, the method further includes the following steps: Obtain a personnel image dataset; in which each personnel image in the personnel image dataset is labeled with a bounding box label and a category label. The category label includes the labeled category as ordinary personnel and uniformed staff. Ordinary personnel represent intruders, and uniformed staff represent non-intruders. The model was trained based on YOLOv10n configuration; The training model was optimized by using a dataset of people images to obtain a target detection model.

[0008] In some embodiments, training a model based on YOLOv10n configuration includes the following steps: In YOLOv10n, a multi-layer convolutional structure is inserted at the first skip connection between the backbone network and the neck. The PSA of the backbone network in YOLOv10n is replaced with C2PSA, and the C2f of the shallow and deep layers in the neck of YOLOv10n is replaced with C3Ghost. The training model is then configured. The multi-layer convolutional structure includes parallel convolutional layers, splicing layers, and convolutional layers for channel number recovery connected in sequence.

[0009] In some embodiments, the training model is optimized by using a people image dataset to obtain an object detection model, including the following steps: People images are used as input data to train a model so that the model can process and output predicted targets; the predicted targets include predicted bounding boxes and predicted categories. A loss function is constructed based on the first difference between the predicted bounding box and the bounding box label, and the second difference between the predicted category and the category label. The target detection model is obtained by backpropagating the training model based on the loss function to tune and optimize the parameters.

[0010] In some embodiments, based on the local image corresponding to the bounding box of the target object, a large model is invoked for verification analysis to obtain the judgment text, including the following steps: Using the bounding box corresponding to the target object whose category is intruder as a reference, the real-time video frame is scaled according to the preset scaling rules to obtain a local image; The local image, along with preset prompts, is input into the large model so that the large model can perform personnel intrusion verification on the local image based on the preset prompts and obtain the judgment text.

[0011] In some embodiments, adaptive alarm processing based on the determined text includes the following steps: If the text is determined to be an intrusion, an alarm message is generated based on the timestamp corresponding to the real-time video frame and associated with the channel identifier. The alarm message is then pushed to the monitoring terminal and an alarm operation is performed. If the text is deemed safe, the real-time video frame is marked as a non-intrusive event.

[0012] To achieve the above objectives, another aspect of the present invention provides a personnel intrusion monitoring device for multi-service collaborative operation, the device comprising: The first module is used to respond to the channel identifier and stream address of the camera sent by the front end, start the inference service process of the corresponding channel based on the channel identifier, and obtain real-time video frames from the stream address through the inference service process; The second module is used to perform intrusion detection based on real-time video frames using a pre-trained target detection model, and obtain the detection results. The detection results include the bounding box and category of the target object, which is either an intruder or a non-intruder. The third module is used to, when the detection results include a target object categorized as an intruder, call a large model based on the local image corresponding to the bounding box of the target object to verify and analyze the judgment text, where the judgment text is either intrusion or security. The fourth module is used for adaptive alarm processing based on the judgment text.

[0013] To achieve the above objectives, another aspect of the present invention provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the aforementioned method.

[0014] To achieve the above objectives, another aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned method.

[0015] To achieve the above objectives, another aspect of the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the aforementioned method.

[0016] The embodiments of the present invention include at least the following beneficial effects: The present invention provides a method, device, electronic device, storage medium, and program product for personnel intrusion monitoring with multi-service collaborative operation. This solution responds to the channel identifier and stream address of the camera issued by the front end, starts the inference service process of the corresponding channel based on the channel identifier, and obtains real-time video frames from the stream address through the inference service process; based on the real-time video frames, personnel intrusion detection is performed using a pre-trained target detection model to obtain detection results; wherein, the detection results include the bounding box and category of the target object, and the category is intruder or non-intruder; when the detection results include a target object of the category of intruder, based on the local image corresponding to the bounding box of the target object, a large model is called for verification and analysis to obtain judgment text; wherein, the judgment text is intrusion or security; adaptive alarm processing is performed based on the judgment text. This invention dynamically starts the inference service process and acquires video streams for each camera through a service startup step, enabling adaptive startup and independent operation of multi-channel inference services. This improves resource utilization efficiency and system flexibility in multi-channel monitoring scenarios. The personnel intrusion detection in this invention employs a two-level process: the first level uses a target detection model for preliminary personnel classification; the second level only calls a large model to verify the scene when a suspected intruder is initially detected, utilizing the fine-grained understanding capabilities of the large model to effectively distinguish between ordinary personnel and uniformed staff, significantly reducing false alarms; the final alarm decision step uses the output of the large model as the basis for judgment to ensure high alarm accuracy. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of an implementation environment for a personnel intrusion monitoring method for multi-service collaborative operations provided in an embodiment of the present invention; Figure 2 This is a flowchart illustrating the personnel intrusion monitoring method for multi-service collaborative operation provided in this embodiment of the invention; Figure 3 This is a schematic diagram of an example of the original YOLOv10n network structure provided in an embodiment of the present invention; Figure 4 This is a schematic diagram illustrating an example of an improved YOLOv10n network design structure provided in an embodiment of the present invention; Figure 5 This is a schematic diagram illustrating the overall process of the personnel intrusion monitoring method for multi-service collaborative operation provided in this embodiment of the invention; Figure 6 This is a schematic diagram of the structure of the personnel intrusion monitoring device for multi-service collaborative operation provided in an embodiment of the present invention; Figure 7 This is a schematic diagram of the structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of this invention; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this invention as detailed in the appended claims.

[0019] It is understood that the terms "first," "second," etc., used in this invention may be used to describe various concepts, but unless specifically stated otherwise, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of embodiments of this invention, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the words "if" or "when" as used herein may be interpreted as "when," "in response to determination," or "in the event of a determination."

[0020] The terms “at least one,” “multiple,” “each,” “any,” etc., used in this invention, “at least one” includes one, two, or more than two; “multiple” includes two or more than two; “each” refers to each of the corresponding multiple; and “any” refers to any one of the multiple.

[0021] Unless otherwise defined, all technical and scientific terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used in this invention is for descriptive purposes only and is not intended to limit the invention.

[0022] In related technologies, due to the limitations of the model's own discrimination capabilities, it may still generate a large number of false alarms in complex scenarios; in addition, the allocation of computing resources for each monitoring channel in a multi-channel video surveillance environment places extremely high demands on computing hardware.

[0023] In view of this, this invention provides a multi-service collaborative method and related equipment for monitoring personnel intrusion. This method responds to the channel identifier and stream address of the camera sent from the front end, initiates an inference service process for the corresponding channel based on the channel identifier, and obtains real-time video frames from the stream address through the inference service process. Based on the real-time video frames, a pre-trained target detection model is used to perform personnel intrusion detection, obtaining detection results. The detection results include the bounding box and category of the target object, where the category is either an intruder or a non-intruder. When the detection results include a target object categorized as an intruder, a large model is invoked based on the local image corresponding to the bounding box of the target object to verify and analyze and obtain a judgment text. The judgment text is either "intrusion" or "security." Adaptive alarm processing is performed based on the judgment text. This invention dynamically starts the inference service process and acquires video streams for each camera through a service startup step, enabling adaptive startup and independent operation of multi-channel inference services. This improves resource utilization efficiency and system flexibility in multi-channel monitoring scenarios. The personnel intrusion detection in this invention employs a two-level process: the first level uses a target detection model for preliminary personnel classification; the second level only calls a large model to verify the scene when a suspected intruder is initially detected, utilizing the fine-grained understanding capabilities of the large model to effectively distinguish between ordinary personnel and uniformed staff, significantly reducing false alarms; the final alarm decision step uses the output of the large model as the basis for judgment to ensure high alarm accuracy.

[0024] It is understood that the multi-service collaborative intrusion monitoring method provided by this invention can be applied to any computer device with data processing and computing capabilities, and this computer device can be various terminals or servers. When the computer device in the embodiment is a server, the server is an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. Optionally, the terminal can be a smartphone, tablet, laptop, or desktop computer, but it is not limited to these.

[0025] like Figure 1 The diagram shown is a schematic representation of an implementation environment provided by an embodiment of the present invention. (Refer to...) Figure 1 The implementation environment includes at least one terminal 102 and a server 101. The terminal 102 and the server 101 can be connected via a network, either wirelessly or via a wired connection, to complete data transmission and exchange.

[0026] Server 101 can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0027] Additionally, server 101 can also be a node server in a blockchain network. Blockchain is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms.

[0028] Terminal 102 can be a smartphone, tablet computer, laptop computer, desktop computer, smart speaker, smartwatch, etc., but is not limited to these. Terminal 102 and server 101 can be directly or indirectly connected via wired or wireless communication, and this embodiment of the invention does not impose any limitations.

[0029] For example, based on Figure 1 The implementation environment shown in this embodiment of the invention provides a method for monitoring personnel intrusion in multi-service collaborative operations. The following description uses the application of this method in server 101 as an example. It can be understood that this method can also be applied to terminal 102.

[0030] Reference Figure 2 , Figure 2 This is an optional flowchart of a multi-service collaborative operation personnel intrusion monitoring method provided in an embodiment of the present invention. The executing entity of the multi-service collaborative operation personnel intrusion monitoring method can be any of the aforementioned computer devices (including servers or terminals). Figure 2 The method may include, but is not limited to, steps S100 to S400.

[0031] Step S100: In response to the channel identifier and stream address of the camera sent by the front end, start the inference service process of the corresponding channel based on the channel identifier, and obtain real-time video frames from the stream address through the inference service process; It should be noted that in some embodiments, responding to the channel identifier and stream address of the camera sent by the front end, starting the inference service process of the corresponding channel based on the channel identifier may include the following steps: obtaining the channel identifier and stream address of the camera sent by the front end through a preset communication protocol, and transmitting them to the service management module; obtaining the code process identifier corresponding to the channel identifier by looking up a preset mapping table through the service management module; starting the corresponding inference service process based on the code process identifier; continuously collecting the running status information of the inference service process through the service management module; if the running status information indicates that the inference service process is not performing an image detection task, closing the inference service process based on the code process identifier; returning to the step of obtaining the code process identifier corresponding to the channel identifier by looking up a preset mapping table through the service management module to restart the inference service process, until the running status information indicates that the inference service process is performing an image detection task.

[0032] For example, in some specific implementations, due to the presence of multiple video surveillance channels, it is necessary to enable multi-service collaborative operation. To further reduce model inference overhead, the front end sets up multiple commands to pre-start loading the model inference service, and issues the video surveillance channel as the unique identifier for that video recognition channel. The front end can obtain the stream address of the camera, and each camera corresponds to a specific channel ID. Recognition is started via HTTP or HTTPS communication protocol. Each time a service is started, the code process ID of that channel is automatically obtained. If the service is detected as not automatically detecting images, the process ID is killed and restarted.

[0033] Step S200: Based on real-time video frames, perform intrusion detection using a pre-trained target detection model to obtain detection results; The detection results include the bounding box and category of the target object, which is either an intruder or a non-intruder. It should be noted that in some embodiments, the method may further include the following steps: acquiring a personnel image dataset; wherein each personnel image in the personnel image dataset is labeled with a bounding box label and a category label, the category label including the labeled category as ordinary personnel and uniformed staff, ordinary personnel representing intruders, and uniformed staff representing non-intruders; configuring and training a model based on YOLOv10n; using the personnel image dataset to perform parameter tuning and optimization training on the training model to obtain a target detection model.

[0034] For example, in some specific implementations, a dataset of personnel from different scenarios is collected and labeled. LabelImg is used as the labeling tool. The main labeling targets are ordinary personnel and uniformed staff. Uniformed staff are those who are permitted to intrude without triggering an alarm. Based on the labeled data, the system categorizes personnel as intruders or non-intruders, and uses this information to train a model that outputs whether the output indicates an intruder or a non-intruder.

[0035] It should be noted that in some embodiments, configuring a training model based on YOLOv10n may include the following steps: inserting a multi-layer convolutional structure at the first skip connection between the backbone network and the neck in YOLOv10n, replacing the PSA of the backbone network in YOLOv10n with C2PSA, and replacing the C2f of the shallow and deep layers in the neck in YOLOv10n with C3Ghost, thereby configuring a training model; wherein, the multi-layer convolutional structure includes a parallel convolutional layer, a splicing layer, and a convolutional layer for channel number recovery connected in sequence.

[0036] Exemplary examples, such as in some specific implementations, Figure 3 The diagram shows the original YOLOv10n network structure. This invention mainly improves the YOLOv10n algorithm to monitor whether there are intruders in the intrusion area of ​​the scene. Figure 4 As shown, this is the improved YOLOv10n network design provided by the present invention. The feature map of the last layer of the backbone network is relatively small. To make fuller use of the feature map information flow, the PSA layer is replaced with C2PSA. The shallow feature map of the original YOLOv10n is relatively large. It is concatenated with the deep feature map. Three convolutions are added in the middle for feature map concatenation, and a convolution is used to restore the original number of channels before concatenating with the deep convolution feature map. To further improve detection accuracy and inference efficiency, the C2f of the deep network is replaced with a C3Ghost convolution.

[0037] The addition of three convolutions for feature map concatenation can be implemented as follows: Assuming the original input image is 640×640×3, after two convolutions, one C2F, and then another convolution and one C2F, the feature map size becomes 80×80×256. The input to the three convolutions is this 80×80 convolutional feature map. The number of convolutions can be determined according to different scenes and camera heights, and can be 1×1, 3×3, or 5×5 convolutions. Finally, after feature map concatenation, a 1×1 convolution is used to restore the original number of channels (Note: In the improved YOLOv10n network design provided in this invention, the scale of the convolutional layers is not specifically declared, and the scale parameters of the original YOLOv10n are used by default).

[0038] in, Figure 3 and Figure 4 The meanings of each module in the network structure are shown in Table 1 below: Table 1

[0039] It should be noted that in some embodiments, the target detection model is obtained by training and optimizing the training model using a people image dataset, which may include the following steps: inputting people images as input data into the training model so that the training model processes and outputs predicted targets; wherein, the predicted targets include predicted bounding boxes and predicted categories; constructing a loss function based on a first difference between the predicted bounding box and the bounding box label and a second difference between the predicted category and the category label; and optimizing the training model through backpropagation based on the loss function to obtain the target detection model.

[0040] For example, in some specific implementations, the prediction box loss (i.e., the first difference) can adopt CIoU Loss (Complete IoU, including overlapping area, center point distance, and aspect ratio penalty), and the prediction class loss (i.e., the second difference) can adopt Cross-Entropy Loss. Then, the two losses are weighted and summed to construct the final loss function used for parameter tuning and optimization. The weight coefficients of the two losses can be set and adjusted according to actual needs, and the default is 1:1.

[0041] Step S300: When the detection result includes a target object classified as an intruder, the large model is called to verify and analyze the local image corresponding to the bounding box of the target object to obtain the judgment text. Among these, the text is determined to be either an intrusion or a security issue; It should be noted that in some embodiments, the judgment text is obtained by calling a large model based on the local image corresponding to the bounding box of the target object and then verifying and analyzing it. This may include the following steps: using the bounding box corresponding to the target object whose category is intruder as a reference, scaling the real-time video frame according to a preset scaling rule to obtain a local image; inputting the local image together with a preset prompt word into the large model so that the large model can verify the intrusion of personnel by referring to the preset prompt word and obtain the judgment text.

[0042] For example, in some specific implementations, the identified personnel image is fed into a large model for analysis to determine whether a regular person has entered the area. The application of the large model further confirms this and reduces false positives in the target recognition algorithm. The large model's input is the entire image, described by prompts. If the person is not a uniformed worker, the output is "Intrusion." If the person is a uniformed worker, the output is "Safe." The large model is invoked when the detection algorithm detects an intruder and resizes the image (the purpose of image resizing is to multi-path analysis algorithms, enabling real-time use of one LLM path, as larger images slow down LLM). Finally, the output of the large model is used as the final standard.

[0043] Step S400: Perform adaptive alarm processing based on the judged text; It should be noted that in some embodiments, adaptive alarm processing based on the determined text may include the following steps: if the determined text is an intrusion, generate alarm information based on the timestamp corresponding to the real-time video frame and associate it with the channel identifier, push the alarm information to the monitoring terminal and perform alarm operation; if the determined text is safe, mark the real-time video frame as a non-intrusion event.

[0044] For example, in some specific implementations, if the text is determined to be an intrusion, the channel identifier bound during the service startup step is obtained from the runtime context of the current inference service process, and the current system timestamp is recorded. Additionally, the corresponding real-time video frame can be obtained as auxiliary information. Subsequently, the channel identifier, timestamp, and real-time video frame are encapsulated into alarm information and pushed to the remote monitoring terminal via HTTP or WebSocket communication protocols, enabling security personnel to quickly locate the incident camera based on the channel identifier and verify the situation based on the image. If the text is determined to be safe, the detection results of all intruders detected in the first-level step (target detection model) in the real-time video frame can be discarded, and the real-time video frame can be marked as a non-intrusion event without triggering any alarm prompts.

[0045] To explain in detail the principle of the technical solution of the present invention, the overall process of the present invention will be described below with reference to some specific embodiments. It is easy to understand that the following is an explanation of the technical principle of the present invention and should not be regarded as a limitation of the present invention.

[0046] First, it should be noted that traditional manual diagnostic methods are obviously inefficient for the design of existing video surveillance intrusion systems and cannot meet the needs of current intrusion detection methods.

[0047] With the continuous development of the economy and computer technology, artificial intelligence (AI) technology has been widely developed. The combination of AI technology and computers has received widespread attention. Existing deep convolutional neural networks have been greatly advanced. In the field of intrusion detection, many scholars have designed different network architectures to solve intrusion detection problems in different scenarios, all aiming to achieve faster and better automated intrusion detection, thereby replacing manual labor and improving diagnostic efficiency and accuracy.

[0048] To improve the efficiency of automated intrusion detection and monitoring systems, this invention provides a method for intrusion monitoring that combines multi-service collaborative operation with YOLOv10n, such as... Figure 5 The diagram illustrates an optional process example of a multi-service collaborative operation personnel intrusion monitoring method according to an embodiment of the present invention. In some specific implementations, the method of the present invention can be implemented as follows: This invention mainly solves the following problems: 1. This invention mainly improves the YOLOv10n algorithm to monitor whether there are intruders in the intrusion area of ​​the scene.

[0049] 2. The feature map in the last layer of the backbone network is relatively small. To make better use of the feature map information flow, the PSA layer is replaced with C2PSA. The shallow feature maps of the original YOLOv10n are relatively large. They are concatenated with the deep feature maps, with three convolutions added in between for feature map concatenation. A convolution is then used to restore the original number of channels before concatenating with the deep convolution feature maps. To further improve detection accuracy and inference efficiency, the C2f layer of the deep network is replaced with a C3Ghost convolution.

[0050] Adding three convolutions for feature map concatenation can be implemented as follows: Assuming the original input image is 640×640×3, after two convolutions, one C2F, and then another convolution and one C2F, the feature map size becomes 80×80×256. The input to the three convolutions is this 80×80 convolutional feature map. The number of convolutions can be determined based on different scenes and camera heights, choosing 1×1, 3×3, or 5×5 convolutions. Finally, after feature map concatenation, a 1×1 convolution is used to restore the original number of channels.

[0051] In some specific application scenarios, the method of the present invention can be implemented as follows: S1: Dataset preparation phase.

[0052] We collected and labeled a dataset of people from different scenarios. LabelImg was used for labeling. The main targets for labeling were ordinary people and staff in uniform.

[0053] Among these, uniformed staff are allowed to bypass the alarm. Based on data labeling, the system categorizes intruders and non-intruders, and uses this information to train a model that outputs whether the user is an intruder or a non-intruder.

[0054] S2: Model training phase.

[0055] The dataset is divided into training, testing, and validation sets for training. Training parameters can be adjusted based on the available training equipment to obtain a better model.

[0056] S3: Multi-service coordination operation.

[0057] Because multiple video surveillance channels exist, multi-service collaborative operation needs to be enabled. To further reduce model inference overhead, the improved YOLOv10n model of this invention uses a front-end configuration to issue multiple commands to pre-start loading the model inference service, and issues the video surveillance channel as the unique identifier for that video channel.

[0058] The front-end can obtain the camera's stream address, with each camera corresponding to a specific channel ID. Recognition is initiated via HTTP or HTTPS communication protocols. Each time a service is started, it automatically obtains the process ID for that channel. If the service is detected as not automatically detecting images, the process ID is killed and restarted.

[0059] S4: Application of large models.

[0060] The identified personnel images are fed into a large model for analysis to determine whether ordinary personnel have entered the area.

[0061] Among these, applying large models further clarifies and reduces false alarms in target recognition algorithms.

[0062] The large model takes the entire image as input, described by prompts. If the person is not a uniformed worker, the output is "Intrusion." If the person is a uniformed worker, the output is "Safe."

[0063] The large model is invoked when the detection algorithm detects an intruder and rescales the image (the purpose of scaling the image is to enable multi-path analysis algorithms to use one LLM in real time, because the larger the image, the slower the LLM becomes). Finally, the output of the large model is used as the final standard.

[0064] In summary, this invention dynamically initiates the inference service process and acquires video streams for each camera through a service startup step, enabling adaptive startup and independent operation of multi-channel inference services. This improves resource utilization efficiency and system flexibility in multi-channel monitoring scenarios. Specifically, the personnel intrusion detection in this invention employs a two-level process: the first level uses a target detection model for preliminary personnel classification; the second level only calls a large model to verify the scene when a suspected intruder is initially detected, leveraging the fine-grained understanding capabilities of the large model to effectively distinguish between ordinary personnel and uniformed staff, significantly reducing false alarms; the final alarm decision step uses the output of the large model as the basis for judgment to ensure high alarm accuracy.

[0065] like Figure 6 As shown, this embodiment of the invention also provides a multi-service collaborative operation personnel intrusion monitoring device 900, which can implement the above-described method. This device may include: The first module 901 is used to respond to the channel identifier and stream address of the camera sent by the front end, start the inference service process of the corresponding channel based on the channel identifier, and obtain real-time video frames from the stream address through the inference service process; The second module 902 is used to perform personnel intrusion detection based on real-time video frames using a pre-trained target detection model, and obtain detection results; wherein, the detection results include the bounding box and category of the target object, and the category is intruder or non-intruder; The third module 903 is used to, when the detection result includes a target object of the category of intruder, call the large model based on the local image corresponding to the bounding box of the target object to verify and analyze the judgment text; wherein the judgment text is either intrusion or security. The fourth module, 904, is used for adaptive alarm processing based on the judgment text.

[0066] It is understood that the content of the above method embodiments is applicable to the present device embodiments. The specific functions implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0067] This invention also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described above. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0068] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0069] like Figure 7 As shown, Figure 7 The hardware structure of an electronic device 1000 according to another embodiment is illustrated. The electronic device 1000 includes: The processor 1001 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (aSIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention. The memory 1002 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RaM). The memory 1002 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1002 and is called and executed by the processor 1001. Input / output interface 1003 is used to implement information input and output; The communication interface 1004 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 1005 transmits information between various components of the device (e.g., processor 1001, memory 1002, input / output interface 1003, and communication interface 1004); The processor 1001, memory 1002, input / output interface 1003 and communication interface 1004 are connected to each other within the device via bus 1005.

[0070] The electronic device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0071] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0072] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0073] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0074] It is understood that the content of the above method embodiments is applicable to the embodiments of this program product. The specific functions implemented by the embodiments of this program product are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0075] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0076] The multi-service collaborative operation personnel intrusion monitoring method, device, electronic device, storage medium, and program product provided in this invention respond to the channel identifier and stream address of the camera sent by the front end, and start the inference service process of the corresponding channel based on the channel identifier. The inference service process obtains real-time video frames from the stream address. Based on the real-time video frames, it uses a pre-trained target detection model to perform personnel intrusion detection and obtains the detection result. The detection result includes the bounding box and category of the target object, which is either an intruder or a non-intruder. When the detection result includes a target object of the category of an intruder, it calls a large model based on the local image corresponding to the bounding box of the target object to verify and analyze and obtain the judgment text. The judgment text is either intrusion or security. Adaptive alarm processing is performed based on the judgment text. This invention dynamically starts the inference service process and acquires video streams for each camera through a service startup step, enabling adaptive startup and independent operation of multi-channel inference services. This improves resource utilization efficiency and system flexibility in multi-channel monitoring scenarios. The personnel intrusion detection in this invention employs a two-level process: the first level uses a target detection model for preliminary personnel classification; the second level only calls a large model to verify the scene when a suspected intruder is initially detected, utilizing the fine-grained understanding capabilities of the large model to effectively distinguish between ordinary personnel and uniformed staff, significantly reducing false alarms; the final alarm decision step uses the output of the large model as the basis for judgment to ensure high alarm accuracy.

[0077] The embodiments described in this invention are for the purpose of more clearly illustrating the technical solutions of the embodiments of this invention, and do not constitute a limitation on the technical solutions provided by the embodiments of this invention. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this invention are also applicable to similar technical problems.

[0078] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present invention, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0079] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0080] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0081] The preferred embodiments of the present invention have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present invention. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and spirit of the present invention should be within the scope of the claims of the present invention.

Claims

1. A method for monitoring personnel intrusion during multi-service collaborative operations, characterized in that, The method includes the following steps: In response to the camera's channel identifier and stream address sent by the front end, the inference service process for the corresponding channel is started based on the channel identifier, and real-time video frames are obtained from the stream address through the inference service process; Based on the real-time video frames, a pre-trained target detection model is used to perform personnel intrusion detection, and the detection results are obtained; wherein, the detection results include the bounding box and category of the target object, and the category is intruder or non-intruder; When the detection result includes a target object classified as an intruder, a large model is invoked based on the local image corresponding to the bounding box of the target object to verify and analyze the judgment text; wherein, the judgment text is either intrusion or security. Adaptive alarm processing is performed based on the determined text.

2. The method according to claim 1, characterized in that, The step of responding to the camera's channel identifier and stream address sent from the front end, and starting the inference service process for the corresponding channel based on the channel identifier, includes the following steps: Obtain the channel identifier and stream address of the camera sent by the front end through a preset communication protocol, and transmit them to the service management module; The service management module retrieves the code process identifier corresponding to the channel identifier by looking up a preset mapping table. Based on the code process identifier, start the corresponding inference service process; The service management module continuously collects the running status information of the inference service process; If the running status information indicates that the inference service process is not performing the image detection task, the inference service process is shut down according to the code process identifier. Return to the step of retrieving the code process identifier corresponding to the channel identifier by looking up the table according to the preset mapping table through the service management module, so as to restart the inference service process until the running status information indicates that the inference service process is executing the image detection task.

3. The method according to claim 1, characterized in that, The method further includes the following steps: Obtain a personnel image dataset; wherein, each personnel image in the personnel image dataset is labeled with a bounding box label and a category label, the category label including the labeled category as ordinary personnel and uniformed staff, the ordinary personnel representing intruders, and the uniformed staff representing non-intruders; The model was trained based on YOLOv10n configuration; The target detection model is obtained by tuning and optimizing the parameters of the training model using the personnel image dataset.

4. The method according to claim 3, characterized in that, The YOLOv10n-based configuration training model includes the following steps: In the YOLOv10n, a multi-layer convolutional structure is inserted at the first skip connection between the backbone network and the neck, the PSA of the backbone network in the YOLOv10n is replaced with C2PSA, and the C2f of the shallow and deep layers in the neck of the YOLOv10n is replaced with C3Ghost, thus obtaining the training model. The multi-layer convolutional structure includes a parallel convolutional layer, a splicing layer, and a convolutional layer for channel number recovery connected in sequence.

5. The method according to claim 3, characterized in that, The step of using the personnel image dataset to perform parameter tuning and optimization training on the training model to obtain the target detection model includes the following steps: The personnel image is input into the training model as input data, so that the training model processes and outputs a predicted target; wherein, the predicted target includes a prediction box and a prediction category; A loss function is constructed based on the first difference between the predicted bounding box and the bounding box label, and the second difference between the predicted category and the category label. The target detection model is obtained by backpropagation to tune and optimize the training model based on the loss function.

6. The method according to claim 1, characterized in that, The process of obtaining the judgment text by calling a large model based on the local image corresponding to the bounding box of the target object includes the following steps: Using the bounding box corresponding to the target object of the category of intruder as a reference, the real-time video frame is scaled according to a preset scaling rule to obtain the local image; The local image, along with a preset prompt, is input into a large model so that the large model performs personnel intrusion verification on the local image with reference to the preset prompt and obtains the judgment text.

7. The method according to claim 1, characterized in that, The adaptive alarm processing based on the determined text includes the following steps: If the determined text is an intrusion, an alarm message is generated based on the timestamp corresponding to the real-time video frame and associated with the channel identifier. The alarm message is pushed to the monitoring terminal and an alarm operation is performed. If the determined text is safe, the real-time video frame is marked as a non-intrusive event.

8. A personnel intrusion monitoring device for multi-service collaborative operation, characterized in that, The apparatus, applicable to the method according to any one of claims 1 to 7, comprises: The first module is used to respond to the channel identifier and stream address of the camera sent by the front end, start the inference service process of the corresponding channel based on the channel identifier, and obtain real-time video frames from the stream address through the inference service process; The second module is used to perform personnel intrusion detection based on the real-time video frames using a pre-trained target detection model, and obtain detection results; wherein, the detection results include the bounding box and category of the target object, and the category is intruder or non-intruder; The third module is used to, when the detection result includes a target object whose category is intruder, call a large model based on the local image corresponding to the bounding box of the target object to verify and analyze and obtain the judgment text; wherein, the judgment text is intrusion or security. The fourth module is used for adaptive alarm processing based on the judgment text.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method according to any one of claims 1 to 7.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images