Service real-time monitoring system based on video images

Abstract

The invention discloses a service real-time monitoring system based on a video image, relates to the technical field of image processing, and solves the problem that a manual recognition mode is difficult to achieve a good effect in the aspects of efficiency and accuracy. The device is provided with the edge calculation unit and the central processing unit, the central processing unit is used for training and updating the model, and the edge calculation unit is used for downloading the trained model to carry out actual recognition processing, so that the calculation loss of the edge calculation unit is effectively reduced, namely, the calculation efficiency is improved. At the moment, the edge computing unit does not need to train and update a model per se, and only needs to use all computing resources for processing the monitoring video data in real time, so that the pressure of edge computing is effectively reduced, and the processing efficiency is improved. In the aspect of accuracy, the model is further retrained and optimized through the central processing unit on the basis of the uploaded data of each edge computing unit in combination with the actual feedback of the terminal, so that the accuracy of the model is further ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a service real-time monitoring system based on video images. Background Art

[0002] Nowadays, the popularity of surveillance equipment is very high, and they are widely used in many fields such as public security, business management, and traffic monitoring. These devices not only help with accident tracking, behavior analysis, etc., but also in the field of urban public safety, surveillance systems are widely used in crowded places such as streets, squares, and stations to improve social security and the ability to respond to emergencies. In addition, the market size of video surveillance equipment is huge. In 2023, the market size of video surveillance equipment in China is about 127.15 billion yuan, a year-on-year increase of 6.54%, showing that the popularity and application of surveillance equipment are growing.

[0003] Although the popularity of surveillance equipment has brought a large amount of video data, the identification of target persons and characteristic behaviors is mostly done manually. This process is time-consuming and labor-intensive, and it is difficult to process the extremely large amount of surveillance data. With the continuous increase in video surveillance equipment, the amount of data generated is also rising sharply, and the manual identification method faces challenges in efficiency and accuracy. The development of intelligent analysis technology will enable the function of the surveillance system to shift from post-tracing to prevention, effectively improving the function and efficiency of the surveillance system itself, but manual identification is still the main means, especially when processing large-scale surveillance data, this process is particularly difficult and inefficient.

[0004] In view of this, a real-time monitoring system for services based on video images is needed. Summary of the invention

[0005] In view of the problem in the prior art that with the continuous increase of video surveillance equipment, the amount of data generated is also rising sharply, and the manual identification method is difficult to achieve good results in terms of efficiency and accuracy, the present invention provides a real-time service monitoring system based on video images, which can automatically identify whether the target person and other key personnel appear in the monitoring data and whether there are unconventional behaviors based on the trained face recognition model and unconventional behavior recognition model, thereby providing a reference for relevant personnel to carry out corresponding processing, which plays a huge role in data screening. The specific technical solution is as follows:

[0006] A service real-time monitoring system based on video images, comprising:

[0007] The edge computing unit includes a video acquisition module, an algorithm use module and an algorithm update module. The video acquisition module acquires real-time monitoring video based on a camera. The algorithm use module stores algorithms for processing real-time monitoring video. The algorithm use module calls these algorithms to process the monitoring video and transmits the processing results to the central processing unit. The algorithm update module is connected to the central processing unit and the algorithm use module, and is used to download the updated algorithm from the central processing unit according to the set time to update the algorithm in the algorithm use module.

[0008] The central processing unit is connected to the edge computing unit and the information acquisition terminal. The central processing unit receives output results from several edge computing units, and updates the model based on feedback results received from the information acquisition terminal and in combination with the feedback results and the output information of the edge computing units.

[0009] The information acquisition terminal is used to obtain the alarm information of the central processing unit, and after manual confirmation through on-site or online processing, the information after manual confirmation is fed back to the central processing unit.

[0010] Preferably, the video acquisition module includes several cameras arranged at different positions, and the real-time monitoring videos collected and uploaded by different cameras are real-time monitoring videos with each camera number and position attached, and the real-time monitoring videos also include the video acquisition time.

[0011] Preferably, the initially trained model stored in the edge computing module is trained by the central processing unit, which is the training of the face recognition model and the training of the unconventional behavior recognition model, and the training process of these two models is as follows:.

[0012] Preferably, the initially trained model stored in the edge computing module is trained by the central processing unit, specifically a face recognition model, and the training and use process of the face recognition model is as follows:

[0013] S1: Prepare a data set containing images of target persons, annotate them, and add category labels of target persons. Category labels of target persons include those containing target persons and those excluding target persons.

[0014] S2: Use the YOLOv8 model for training. Start training on the specified device by loading the pre-trained YOLOv8 model and setting the training parameters, including the training cycle, input image size, and batch size.

[0015] S3: After the training is completed, the trained YOLOv8 model is used to detect the video stream in real time. The camera or video file is opened, the video content is read frame by frame, and the model is used to predict each frame to identify the target person.

[0016] S4: During the video stream detection process, the timestamp of each frame is recorded. When the model detects the target person, the timestamp of the frame is recorded as the time point when the target person appears;

[0017] S5: Output the conclusion of the detected target person and the specific time point of their appearance, where the time point is a specific frame number or converted into a time axis of the video;

[0018] S6: According to the recorded time point, use a video processing tool to capture the video clip where the target person appears.

[0019] Preferably, the preliminarily trained model stored in the edge computing module is trained by the central processing unit, specifically an unconventional behavior recognition model. The training and use process of the unconventional behavior recognition model is as follows:

[0020] S01: Collect and annotate video datasets containing unconventional behaviors, add time tags and behavior categories to unconventional behavior frames in the video, and the behavior categories include those containing unconventional behaviors and those not containing unconventional behaviors;

[0021] S02: Use deep learning frameworks and pre-trained models to train a model that can identify unconventional behaviors;

[0022] S03: Obtain real-time video streams from surveillance cameras, or read recorded video files, use trained models to perform frame-level analysis on video streams, and detect irregular behaviors in the video in real time;

[0023] S04: During the detection process, for each frame of the video, the time point at which the unusual behavior is detected is recorded, represented by a frame number or a timestamp of the video;

[0024] S05: Output the results of the detected unconventional behavior and the corresponding time nodes, specifically as a text report or directly annotating on the video;

[0025] S06: Based on the recorded time nodes, use video editing software or programming library to capture the segments with unusual behaviors in the video.

[0026] Preferably, the central processing unit supplements the data set based on the feedback signal of the information acquisition terminal for training and optimization of the model. At this time, if the feedback signal of the information acquisition terminal is "correct recognition", then the central processing unit composes a data set according to the alarm signal of the edge computing unit and the corresponding video. If the feedback signal of the information acquisition terminal is "error in recognition", the alarm signal is changed to a non-alarm signal and then the corresponding captured video is added as a data set.

[0027] Preferably, a map marking module is provided on the central processing unit, and the map marking module is used to mark the location of each alarming camera on the map, and the operation steps are as follows:

[0028] S10: Register a developer account of a map service provider and obtain an API key;

[0029] S20: Define an HTML element as a map container in the front-end page, set its size and style, and use <script>标签或者模块化的方式加载地图服务的JavaScript库；

[0030] S30：使用API密钥和地图服务提供的初始化函数创建地图实例，并设置中心点坐标和缩放级别；

[0031] S40：编写一个函数，该函数接受经纬度参数，并创建一个标记对象，并设置标记的属性，包括位置、图标、标题等，所述函数具体如下：

[0032] S401：确定函数的名称，例如addMarker，确定函数的参数，至少包括经纬度坐标、标记的标题、图标等属性；

[0033] S402：使用地图API提供的Marker构造函数创建一个新的标记对象；

[0034] S403：将创建的标记对象添加到地图实例上；

[0035] S50：调用标记添加函数，将创建的标记对象添加到地图实例上，通过循环遍历经纬度数据数组，为每个数据项调用标记添加函数。

[0036] 优选的，中央处理单元上设有地图标记模块，所述地图标记模块用于在地图上标记人脸识别模型识别出来的人员的行走路径，其实现过程如下：

[0037] S100：收集包含经纬度坐标和对应时间或时间范围的数据，这些数据以列表或数组的形式组织，每个条目都包含一个地点的经度、纬度和时间戳；

[0038] S200：每个地点可以用一个包含经度、纬度和时间属性的对象来表示；

[0039] S300：根据每个地点的时间戳对整个数据集进行排序；

[0040] S400：设置一个地图实例；

[0041] S500：遍历排序后的数据集，并为每个地点在地图上添加一个标记，这些标记将代表数据集中的每个地点；

[0042] S600：对于排序后的数据集中的每对连续地点，在地图上绘制一条线或箭头，以显示它们之间的连接。

[0043] 一种计算机可读存储介质，所述计算机可读存储介质包括存储的程序，其中，在所述程序运行时控制所述计算机可读存储介质所在设备执行如上所述的种基于视频图像的服务实时监控系统。

[0044] 一种处理器，所述处理器用于运行程序，其中，所述程序运行时执行如上所述的种基于视频图像的服务实时监控系统。

[0045] 与现有技术相比，本发明的有益效果为：

[0046] 本发明训练有能够基于训练好的人脸识别模型以及非常规行为识别模型来自动识别监控数据中是否出现目标人物等重点关注人员以及是否存在非常规行为等，进而有利于提供参考以便让相关人员去进行相应的处理，在数据筛查方面起到巨大的作用。而且，再次基础上，本发明设置有边缘计算单元和中央处理单元，所述中央处理单元主要用于模型的训练和更新，所述边缘计算单元用于下载训练好的模型进行实际的识别处理，这有效减少饿了边缘计算单元的计算损耗，即，边缘计算单元此时不需要自身进行模型的训练和更新，只需要将所有的计算资源用于实时处理监控视频数据即可，有效减轻了边缘计算的压力，提升了处理效率。在准确性方面，本发明通过中央处理单元基于各边缘计算单元的上传数据，以及结合终端的实际反馈，进一步对模型进行再训练和优化，进一步保证了模型的准确程度。附图说明

[0047] 为了更清楚地说明本发明具体实施方式或现有技术中的技术方案，下面将对具体实施方式或现有技术描述中所需要使用的附图作简单地介绍。在所有附图中，类似的元件或部分一般由类似的附图标记标识。附图中，各元件或部分并不一定按照实际的比例绘制。

[0048] 图1为本发明人脸识别模型的训练及应用流程图；

[0049] 图2为本发明非常规行为识别模型的训练及应用流程图。具体实施方式

[0050] 下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。

[0051] 应当理解，当在本说明书和所附权利要求书中使用时，术语"包括”和"包含”指示所描述特征、整体、步骤、操作、元素和 / 或组件的存在，但并不排除一个或多个其它特征、整体、步骤、操作、元素、组件和 / 或其集合的存在或添加。

[0052] 还应当理解，在本发明说明书中所使用的术语仅仅是出于描述特定实施例的目的而并不意在限制本发明。如在本发明说明书和所附权利要求书中所使用的那样，除非上下文清楚地指明其它情况，否则单数形式的"一”、"一个”及"该”意在包括复数形式。

[0053] 还应当进一步理解，在本发明说明书和所附权利要求书中使用的术语"和 / 或”是指相关联列出的项中的一个或多个的任何组合以及所有可能组合，并且包括这些组合。

[0054] 在本发明的一个实施例中，提供一种基于视频图像的服务实时监控系统，包括：

[0055] 边缘计算单元，包括视频获取模块、算法使用模块和算法更新模块，所述视频获取模块基于摄像头获取实时监控视频，所述算法使用模块中存储有用于处理实时监控视频的算法，所述算法使用模块调用这些算法对监控视频进行处理，并且将处理结果传输给所述中央处理单元。所述算法更新模块与中央处理单元以及算法使用模块连接，用于按照设定时间从中央处理单元下载更新后的算法对算法使用模块中的算法进行更新。

[0056] 中央处理单元，与边缘计算单元以及信息获取终端连接，中央处理单元接收来自若干个边缘计算单元的输出结果，并基于输出结果决定是否向信息获取终端发送告警信号，此外，中央处理单元，接收来自信息获取终端的反馈结果，并结合反馈结果以及边缘计算单元的输出信息进行模型的更新。

[0057] 信息获取终端，用于获取中央处理单元的告警信息，便于基于告警信息进行相应的处理，且通过现场或线上处理的方式，经由人工确认之后，向中央处理单元反馈经由人工确认之后的信息。

[0058] 总之，以上三个单元模块的处理过程是：首先由中央处理单元基于从边缘计算单元获取到的实时监控视频以及第三方的数据库的原始数据进行模型的训练，包括人脸识别模型以及非常规行为识别模型，初步训练完成后传输到个边缘计算单元中进行实际的识别应用。其次，边缘计算单元基于从中央处理单元出获取到的模型来识别自身获取到的视频监控中有无相应人脸以及相应的非常规行为，进而给说中央处理单元发送异常情况的识别结果，并且附上异常情况的视频截取部分，中央处理单元接收识别结果以及视频截取部分并存储，随后基于识别结果向信息获取终端发送告警信息。最后，信息获取终端基于告警信息安排人员进行相应的核查以及处理，并且向所述中央处理单元发送反馈，所述反馈包括"识别正确”以及"识别错误”的反馈。中央处理单元接收到反馈之后，将反馈为"识别正确”的视频截取以及相应的识别结果再一次作为数据集进一步训练和优化模型，将"识别错误”的视频截取作为正常情况的数据集送入模型进一步训练和优化。

[0059] 下面进一步介绍以上各个单元。

[0060] 首先是边缘计算单元，所述边缘计算单元包括多个摄像头，且对每一个摄像头的编号和位置进行记录，即，不同的摄像头采集并上传的实时监控视频是附带有每一个摄像头编号和位置的实时监控视频。应知晓的，所述实时监控视频上还包括视频的获取时间。若干个设置在不同位置的摄像头组成视频获取模块。

[0061] 算法使用模块上存储有初步训练好的模型，包括人脸识别模型和非常规行为识别模型，所述人脸识别模型用于识别摄像头拍摄到的视频影像中是否存在目标人物等警方需要查找到的或重点关注的人物，所述非常规行为识别模型用于识别摄像头拍摄到的视频影像中是否存在相应的非常规行为。

[0062] 算法更新模块用于和中央处理单元连接用于定期获取中央处理单元上经过进一步的数据集训练和优化之后，更新过的模型，用以对算法使用模块上使用的模型进行更细迭代，以优化模型的识别效率或准确度。

[0063] 中央处理单元主要用于训练模型以及接收模型的用于训练或优化的数据集，所述边缘计算模块中存储的初步训练完毕的模型即由中央处理单元训练，分别为人脸识别模型的训练和非常规行为识别模型的训练，这两个模型的训练过程如下：

[0064] 如图1所示，人脸识别模型的训练及应用的过程如下：

[0065] S1：准备一个包含目标人物图像的数据集，并对其进行标注，添加目标人物的类别标签，目标人物的类别标签包括含目标人物以及不含目标人物。

[0066] S2：使用YOLOv8模型进行训练，通过加载预训练的YOLOv8模型，并设置训练参数，如训练周期、输入图像尺寸和批次大小，在指定的设备上开始训练。

[0067] S3：训练完成后，使用训练好的YOLOv8模型对视频流进行实时检测。打开摄像头或视频文件，逐帧读取视频内容，并使用模型对每一帧进行预测，识别出目标人物。

[0068] S4：在视频流检测过程中，记录下每一帧的时间戳。当模型检测到目标人物时，记录下该帧的时间戳，这将作为目标人物出现的时间点。

[0069] S5：将检测到的目标人物的结论以及他们出现的具体时间点输出。时间点可以是具体的帧编号或者转换为视频的时间轴(如秒数)。

[0070] S6：根据记录的时间点，使用视频处理工具截取目标人物出现的视频片段。这可以通过视频编辑软件或者编程库(如OpenCV)来实现。

[0071] 此外，可将上述目标人物的图片替换成想要识别出来的人脸的图片，如走失儿童或其他重点关注对象等。

[0072] 如图2所示，下面是非常规行为识别模型的训练以及应用过程：

[0073] S01：收集并标注包含非常规行为的视频数据集，为视频中的非常规行为帧添加时间标签和行为类别，行为类别包括含非常规行为和不含非常规行为。

[0074] S02：使用深度学习框架(如TensorFlow或PyTorch)和预训练模型(如YOLO、SSD或Faster R-CNN)来训练一个能够识别非常规行为的模型。

[0075] S03：从监控摄像头获取实时视频流，或者读取已经录制好的视频文件。使用训练好的模型对视频流进行帧级分析，实时检测视频中的非常规行为。

[0076] S04：在检测过程中，对于每一帧视频，记录下检测到非常规行为的时间节点，这通常是以视频的帧编号或者时间戳来表示。

[0077] S05：将检测到的非常规行为的结果以及对应的时间节点输出，可以是文本形式的报告或者直接在视频上标注。

[0078] S06：根据记录的时间节点，使用视频编辑软件或编程库(如FFmpeg或OpenCV)来截取视频中存在非常规行为的段落。可以设置一个时间窗口，比如在非常规行为发生前后的几秒钟内，以确保截取的视频段落包含了非常规行为的全过程。

[0079] 以上是中央处理单元中模型训练过程以及边缘计算单元中利用模型进行实际视频识别的过程。在边缘计算单元里，若识别到有目标人物，则会向中央处理单元发送"含目标人物”的告警信号并且附上剪辑出来的含嫌疑人的视频。相似的，如果识别出有非常规行为，则向中央处理单元发送"含非常规行为”的告警信号并且附上简剪辑出来的含非常规行为的视频。也就是说，此时"含目标人物”+相应的视频片段，以及"含非常规行为”+相应的视频片段会存储在中央处理单元中。

[0080] 在边缘计算单元向中央处理单元发送告警信号的时候，也会同步向信息获取终端发送告警信号以及相应的视频片段。信息获取终端收到告警信息的时候，安排相应的警务人员对告警信号以及相应的视频片段进行确认，以排除边缘计算单元的误报现象，并且在排除了边缘计算单元没有误报现象的时候进行相应的后续处理并且向中央处理单元返回"识别正确”的反馈信号。此时，如果确认告警信号为边缘计算单元的误报现象，则向中央处理单元返回"识别错误”的反馈信号。

[0081] 中央处理单元基于信息获取终端的反馈信号进行数据集的补充以便进行模型的训练和优化。此时，若信息获取终端的反馈信号是"识别正确”，那么，中央处理单元按边缘计算单元的告警信号以及相应视频来组成数据集。若信息获取终端的反馈信号是"识别错误”，则将告警信号改为非告警信号然后加上相应的截取视频作为数据集，即，截取视频的标签从"含目标人物”或"含非常规行为”变为"不含目标人物”或"不含非常规行为”，进而进一步训练优化并更新模型。

[0082] 在此过程中，中央处理单元是一直在根据信息获取终端的反馈信号并基于边缘计算单元上传的截取视频来进行模型的进一步训练优化，进而实现模型的更新，使得模型能够更加适应于当下的识别，以及提高识别的准确程度。

[0083] 边缘计算单元中设置有模型更新模块，用于在一定的时间间隔内下载中央处理单元中的最新训练好的模型进而对模型使用单元中存储的模型进行更新和替换，进而用最新训练好的，准确度更高的模型来进行后续的视频影像的识别。上面所述的时间间隔根据说实际情况设定一个固定的时间，也可根据边缘计算单元的告警次数来区别化这个固定的时间，如，告警次数越高的边缘计算单元，这个设置的固定的时间间隔就越短。

[0084] 由于边缘计算单元中对每一个摄像头的编号和位置进行记录，即，不同的摄像头采集并上传的实时监控视频是附带有每一个摄像头编号和位置的实时监控视频。因此中央处理单元中所获取到的告警信号以及与其相对应的监控视频截取是携带者摄像头的编号和位置的，这是由于摄像头在上传实时影像给边缘计算单元计算的时候就携带该摄像头的编号，而边缘计算单元以及中央处理单元中均存储有摄像头编号及其对应的位置信息，即，从摄像头编号可直接从存储的其与位置信息的对应关系中识别出对应的位置信息来。这个位置信息具体为摄像头所处位置的经纬度坐标。

[0085] 识别出位置后，中央处理单元上设有地图标记模块，所述地图标记模块用于将每一个告警的摄像头的位置均在地图上标记出来，其操作步骤如下：

[0086] S10：注册地图服务提供商的开发者账号，获取API密钥。

[0087] S20：在前端页面中定义一个HTML元素作为地图的容器，设置其大小和样式，通过<script>标签或者模块化的方式加载地图服务的JavaScript库；

[0088] S30：使用API密钥和地图服务提供的初始化函数创建地图实例，并设置中心点坐标和缩放级别；

[0089] S40：编写一个函数，该函数接受经纬度参数，并创建一个标记对象。并设置标记的属性，如位置、图标、标题等。该函数具体如下：

[0090] S401：确定函数的名称，例如addMarker。确定函数的参数，至少包括经纬度坐标、标记的标题、图标等属性；

[0091] S402：使用地图API提供的Marker构造函数创建一个新的标记对象。

[0092] S403：将创建的标记对象添加到地图实例上，在本实施例中，该点由一个方法来实现，如map.addMarker(marker)；

[0093] S50：调用标记添加函数，将创建的标记对象添加到地图实例上，通过循环遍历经纬度数据数组，为每个数据项调用标记添加函数。

[0094] 以上适用于人脸识别模型和非常规识别模型中任意一个模型识别后出现告警信号的情形，即，将告警信号发送给信息获取终端的同时，中央处理单元还将标记有相应摄像头位置的地图传输给信息获取终端。

[0095] 进一步的，在人脸识别模型识别后发出告警信号的情况中，除以上所描述的将摄像头的位置在地图上标记出来之外，还根据目标人物等被识别人员在相应摄像头捕捉位置的先后顺序，在地图上标记处目标人物等被识别人员的行走路径，其实现过程如下：

[0096] S100：收集包含经纬度坐标和对应时间或时间范围的数据。这些数据以列表或数组的形式组织，每个条目都包含一个地点的经度、纬度和时间戳。

[0097] S200：每个地点可以用一个包含经度、纬度和时间属性的对象来表示。

[0098] S300：根据每个地点的时间戳对整个数据集进行排序。这通常涉及到比较时间值，并重新排列数据集中的条目，以确保它们按照时间的先后顺序排列。

[0099] S400：设置一个地图实例；

[0100] S500：遍历排序后的数据集，并为每个地点在地图上添加一个标记。这些标记将代表数据集中的每个地点。

[0101] S600：对于排序后的数据集中的每对连续地点(除了最后一个地点)，在地图上绘制一条线或箭头，以显示它们之间的连接。这表示地点之间的顺序和方向。

[0102] S700：为地图上的标记和线条设置交互功能。例如，当用户点击一个标记时，可以显示一个信息窗口，提供更多关于该地点的详细信息，包括该地点的摄像头捕捉到的原视频以及经由边缘计算单元处理后的告警信号以及视频截取。

[0103] 综上所述，本发明训练有能够基于训练好的人脸识别模型以及非常规行为识别模型来自动识别监控数据中是否出现目标人物等重点关注人员以及是否存在非常规行为等，进而有利于提供参考以便让相关人员去进行相应的处理，在数据筛查方面起到巨大的作用。而且，再次基础上，本发明设置有边缘计算单元和中央处理单元，所述中央处理单元主要用于模型的训练和更新，所述边缘计算单元用于下载训练好的模型进行实际的识别处理，这有效减少饿了边缘计算单元的计算损耗，即，边缘计算单元此时不需要自身进行模型的训练和更新，只需要将所有的计算资源用于实时处理监控视频数据即可，有效减轻了边缘计算的压力，提升了处理效率。在准确性方面，本发明通过中央处理单元基于各边缘计算单元的上传数据，以及结合终端的实际反馈，进一步对模型进行再训练和优化，进一步保证了模型的准确程度。

[0104] 本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元，能够以电子硬件、计算机软件或者二者的结合来实现，为了清楚地说明硬件和软件的可互换性，在上述说明中已经按照功能一般性地描述了各示例的组成。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本发明的范围。

[0105] 在本发明所提供的实施例中，应该理解到，单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元可结合为一个单元，一个单元可拆分为多个单元，或一些特征可以忽略等。

[0106] 另外，在本发明各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。

[0107] 所述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本发明的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可为个人计算机、服务器或者网络设备等)执行本发明各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、只读存储器(ROM,Read-0nlyMemory)、随机存取存储器(RAM,RandomAccessMemory)、移动硬盘、磁碟或者光盘等各种可以存储程序代码的介质。

[0108] 最后应说明的是：以上各实施例仅用以说明本发明的技术方案，而非对其限制；尽管参照前述各实施例对本发明进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分或者全部技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本发明各实施例技术方案的范围，其均应涵盖在本发明的权利要求和说明书的范围当中。< / script>

Claims

1. A real-time monitoring system for services based on video images, characterized in that: include: The edge computing unit includes a video acquisition module, an algorithm use module and an algorithm update module. The video acquisition module acquires real-time monitoring video based on a camera. The algorithm use module stores algorithms for processing real-time monitoring video. The algorithm use module calls these algorithms to process the monitoring video and transmits the processing results to the central processing unit. The algorithm update module is connected to the central processing unit and the algorithm use module, and is used to download the updated algorithm from the central processing unit according to the set time to update the algorithm in the algorithm use module. A central processing unit is connected to the edge computing unit and the information acquisition terminal, the central processing unit receives output results from the plurality of edge computing units, and updates the model based on feedback results received from the information acquisition terminal and in combination with the feedback results and the output information of the edge computing units; The information acquisition terminal is used to obtain the alarm information of the central processing unit, and after manual confirmation through on-site or online processing, the information after manual confirmation is fed back to the central processing unit.

2. A video image-based service real-time monitoring system according to claim 1, characterized in that: The video acquisition module includes a plurality of cameras arranged at different positions, and the real-time monitoring videos collected and uploaded by different cameras are real-time monitoring videos with the number and position of each camera attached, and the real-time monitoring videos also include the acquisition time of the video.

3. A real-time monitoring system for services based on video images according to claim 1, characterized in that: The initially trained model stored in the edge computing module is trained by the central processing unit, and the training of the central processing unit includes the training of the face recognition model and the training of the unconventional behavior recognition model.

4. A real-time monitoring system for services based on video images according to claim 1, characterized in that: The training and use process of the face recognition model is as follows: S1: Prepare a data set containing images of target persons, annotate them, and add category labels of target persons. Category labels of target persons include those containing target persons and those excluding target persons. S2: Use the YOLOv8 model for training. Start training on the specified device by loading the pre-trained YOLOv8 model and setting the training parameters, including the training cycle, input image size, and batch size. S3: After the training is completed, the trained YOLOv8 model is used to detect the video stream in real time. The camera or video file is opened, the video content is read frame by frame, and the model is used to predict each frame to identify the target person. S4: During the video stream detection process, the timestamp of each frame is recorded. When the model detects the target person, the timestamp of the frame where the target person is detected is recorded as the time point when the target person appears. S5: Output the conclusion of the detected target person and the specific time point of their appearance, where the time point is a specific frame number or converted into a time axis of the video; S6: According to the recorded time point, use a video processing tool to capture the video clip where the target person appears.

5. A real-time monitoring system for services based on video images according to claim 4, characterized in that: The training and use process of the unconventional behavior recognition model is as follows: S01: Collect and annotate video datasets containing unconventional behaviors, add time tags and behavior categories to unconventional behavior frames in the video, and the behavior categories include those containing unconventional behaviors and those not containing unconventional behaviors; S02: Use deep learning frameworks and pre-trained models to train a model that can identify unconventional behaviors; S03: Obtain real-time video streams from surveillance cameras, or read recorded video files, use trained models to perform frame-level analysis on video streams, and detect irregular behaviors in the video in real time; S04: During the detection process, for each frame of the video, the time point at which the unusual behavior is detected is recorded, represented by a frame number or a timestamp of the video; S05: Output the results of the detected unconventional behavior and the corresponding time nodes, specifically as a text report or directly annotating on the video; S06: Based on the recorded time nodes, use video editing software or programming library to capture the segments with unusual behaviors in the video.

6. A real-time monitoring system for services based on video images according to claim 5, characterized in that: The central processing unit supplements the data set based on the feedback signal of the information acquisition terminal for model training and optimization. At this time, if the feedback signal of the information acquisition terminal is "correct recognition", then the central processing unit forms a data set according to the alarm signal of the edge computing unit and the corresponding video. If the feedback signal of the information acquisition terminal is "error in recognition", the alarm signal is changed to a non-alarm signal and the corresponding intercepted video is added as the data set.

7. A real-time monitoring system for services based on video images according to claim 1, characterized in that: The central processing unit is provided with a map marking module, which is used to mark the location of each alarmed camera on the map. The operation steps are as follows: S10: Register a developer account of a map service provider and obtain an API key; S20: Define an HTML element as a map container in the front-end page, set its size and style, and <script>标签或者模块化的方式加载地图服务的JavaScript库；S30：使用API密钥和地图服务提供的初始化函数创建地图实例，并设置中心点坐标和缩放级别；S40：编写一个函数，该函数接受经纬度参数，并创建一个标记对象，并设置标记的属性，包括位置、图标、标题等，所述函数具体如下：S401：确定函数的名称，例如addMarker，确定函数的参数，至少包括经纬度坐标、标记的标题、图标等属性；S402：使用地图API提供的Marker构造函数创建一个新的标记对象；S403：将创建的标记对象添加到地图实例上；S50：调用标记添加函数，将创建的标记对象添加到地图实例上，通过循环遍历经纬度数据数组，为每个数据项调用标记添加函数。8.根据权利要求1所述的一种基于视频图像的服务实时监控系统，其特征在于，中央处理单元上设有地图标记模块，所述地图标记模块用于在地图上标记人脸识别模型识别出来的人员的行走路径，其实现过程如下：S100：收集包含经纬度坐标和对应时间或时间范围的数据，这些数据以列表或数组的形式组织，每个条目都包含一个地点的经度、纬度和时间戳；S200：每个地点可以用一个包含经度、纬度和时间属性的对象来表示；S300：根据每个地点的时间戳对整个数据集进行排序；S400：设置一个地图实例；S500：遍历排序后的数据集，并为每个地点在地图上添加一个标记，这些标记将代表数据集中的每个地点；S600：对于排序后的数据集中的每对连续地点，在地图上绘制一条线或箭头，以显示它们之间的连接。9.一种计算机可读存储介质，其特征在于，所述计算机可读存储介质包括存储的程序，其中，在所述程序运行时控制所述计算机可读存储介质所在设备执行权利要求1所述的基于视频图像的服务实时监控系统。10.一种处理器，其特征在于，所述处理器用于运行程序，其中，所述程序运行时执行权利要求1所述的基于视频图像的服务实时监控系统。< / script>

Images