A fusion intelligent AI visualization monitoring method and system

Abstract

The present invention provides a fusion intelligent AI visualization monitoring method and system, which includes: constructing a feature recognition model, acquiring monitoring video in real time, and importing the feature parameters of the monitoring image under the corresponding frame number into the feature recognition model to obtain status information corresponding to the feature parameters under the current frame number; based on the obtained status information, generating a monitoring result and judging whether the monitoring result is abnormal; if the monitoring result is abnormal, retrieving the parameter information of the monitoring image under the corresponding frame number and feeding back the abnormal monitoring result to the visualization monitoring platform for evaluation, and predicting the trend of the abnormal monitoring result; otherwise, continuing monitoring. According to the present invention, after obtaining the abnormal status information, further judging its detection result and judging whether the monitoring result is abnormal; if so, retrieving the parameter information of the monitoring image under the corresponding frame number and feeding back the abnormal monitoring result to the visualization monitoring platform, and the visualization monitoring platform evaluates the abnormal event.

Description

Technical Field

[0001] The present invention relates to the field of analysis technology, and in particular to a fusion intelligent AI visualization monitoring method and system. Background Art

[0002] The fusion intelligent AI visualization monitoring method and system refers to the use of artificial intelligence technology and visualization technology, combined with multiple data sources and sensor information, to achieve real-time monitoring, analysis and prediction of complex systems or environments; with the continuous development of social processes today, smart factories have gradually become a development trend. In the operation of factories, the safety management of personnel and equipment is of vital importance. By setting monitoring points, personnel and equipment can be monitored to ensure their safety; however, since a large number of monitoring points need to be set up in the factory, the amount of video that needs to be monitored is huge, video alarms have no intelligent judgment, and the workload of manual identification and judgment is large, which easily leads to repeated operations, fatigue and slow response speed for operators; at the same time, the storage and retrieval function is weak, there is no effective retrieval and positioning, and only files can be viewed one by one, which has low application efficiency; at the same time, it is easy to make misjudgments by only using video monitoring. Therefore, the present invention studies and designs a fusion intelligent AI visualization monitoring method and system. Summary of the Invention

[0003] Therefore, the technical problem to be solved by the present invention is to overcome the defects mentioned above, thereby providing a fusion intelligent AI visualization monitoring method and system.

[0004] In order to solve the above problems, the present invention provides a fusion intelligent AI visualization monitoring method, which includes:

[0005] S1: Obtain surveillance video samples from the visual monitoring platform based on the cloud server, segment them into surveillance images for each frame, and extract the feature parameters and parameter information corresponding to each frame of surveillance image;

[0006] S2: Based on the feature parameters of the monitoring image as feature values ​​and the state information corresponding to each feature parameter as the target variable, a feature recognition model is constructed, and the state information corresponding to each abnormal feature parameter is marked as an abnormal result set;

[0007] S3: Acquire surveillance video in real time and import the feature parameters of the surveillance image at the corresponding frame number into the feature recognition model to obtain the state information corresponding to the feature parameters at the current frame number;

[0008] S4: Based on the obtained status information, generate monitoring results and determine whether the monitoring results are abnormal. If the monitoring results are abnormal, retrieve the parameter information of the monitoring image under the corresponding frame number and feed it back to the visual monitoring platform for evaluation together with the abnormal monitoring results, and predict the trend of the abnormal monitoring results. Otherwise, continue monitoring.

[0009] Preferably, in said S1, the parameter information of the monitoring image is: time information and location information;

[0010] The feature parameters are: edge, corner, area, and ridge features of the monitoring image.

[0011] Preferably, in S2, the state information is: a plurality of feature parameter subsets of different shapes formed according to types of feature parameters, and the plurality of feature parameter subsets correspond to one type of state information.

[0012] Preferably, in S3, the characteristic parameters of the surveillance image corresponding to the number of frames are:

[0013] Based on the surveillance video, surveillance images of the ath frame and the random bth frame are obtained, where a<b, and corresponding feature parameters are extracted respectively. The feature parameters are imported into the feature recognition model to obtain state information of the ath frame and state information of the bth frame respectively;

[0014] When the characteristic parameters of the surveillance image at the corresponding frame number in the surveillance video sample change, the frame number of the current surveillance image is marked as a.

[0015] Preferably, in S4, based on the obtained status information, a monitoring result is generated as follows:

[0016] Based on the monitoring image of frame a and the monitoring image of frame b, determine whether the status information of frame a belongs to the abnormal result set. If not, continue monitoring. Otherwise, compare the obtained status information of frame a with the status information of frame b to determine whether they are the same.

[0017] If they are not the same, a new random number c is re-established, where a<c<b, and the characteristic parameters of the c-th frame are extracted, the state information of the c-th frame is obtained, and the state information of the c-th frame is compared with the state information of the a-th frame again until the state information of the c-th frame is the same as the state information of the a-th frame;

[0018] If the status information of the ath frame is the same as the status information of the bth frame or the status information of the cth frame, the status information of the ath frame is judged to be an abnormal result, and the status information of the ath frame, the parameter information of the ath frame, the parameter information of the bth frame or the parameter information of the cth frame is transmitted to the visual monitoring platform.

[0019] Preferably, in S4, the trend of the predicted abnormal monitoring result is:

[0020] If the status information of the a-th frame is the same as the status information of the b-th frame or the status information of the c-th frame, the difference calculation is performed using the feature parameter subset of the a-th frame and the feature parameter subset of the b-th frame or the feature parameter subset of the c-th frame to obtain a difference feature parameter set, and the time from the a-th frame to the b-th frame or the c-th frame is obtained to obtain the change trend of the difference feature parameter set per unit time, and generate a predicted trend of the abnormal monitoring results.

[0021] Preferably, the visual monitoring platform is evaluated as follows: the visual monitoring platform evaluates the abnormal monitoring results and feeds back the actual results, compares the actual results with the abnormal monitoring results, and if the deviation between the actual results and the abnormal monitoring results is within the deviation threshold range, the feature recognition model is trained using the feature parameters and corresponding status information of the monitoring images from frame a to frame b or the monitoring image from frame c in the abnormal monitoring results.

[0022] The present invention further provides a fusion intelligent AI visualization monitoring system, which adopts the fusion intelligent AI visualization monitoring method described in any of the preceding items, including:

[0023] A visual monitoring platform, wherein the integrated intelligent AI visual monitoring system is connected to the visual monitoring platform via the Internet;

[0024] An information acquisition module, configured to acquire monitoring videos from the visual monitoring platform;

[0025] An extraction module is used to generate a surveillance image at a corresponding frame number from the surveillance video, and extract feature parameters and parameter information of the surveillance image at the corresponding frame number from the surveillance image at the corresponding frame number;

[0026] The feature recognition model module is used to generate state information of the corresponding frame number based on the feature parameters of the corresponding frame number obtained in real time, and obtain monitoring results based on the state information;

[0027] The control module is respectively connected to the information acquisition module, the extraction module and the feature recognition model module. The control module is also configured as a control storage subunit with a time storage function to store parameter information, status information and monitoring results in the corresponding process.

[0028] Preferably, the extraction module further comprises: an extraction marking unit, configured to mark corresponding frame numbers when characteristic parameters of the surveillance images at corresponding frame numbers in the surveillance video samples change.

[0029] Preferably, the system further includes a processing module: the processing module is connected to the control module, and the processing module includes a judgment and comparison module and a solution module, the judgment and comparison module is used to judge whether the monitoring result is abnormal, whether the characteristic parameters of the monitoring image at the corresponding frame number in the monitoring video sample have changed, whether the state information at the corresponding frame number is the same, and whether the deviation between the actual result and the abnormal monitoring result is within the deviation threshold range;

[0030] The solution module performs difference calculation using a feature parameter subset of the ath frame and a feature parameter subset of the bth frame or a feature parameter subset of the cth frame to obtain a difference feature parameter set, and obtains the time from the ath frame to the bth frame or the cth frame to obtain a change trend of the difference feature parameter set per unit time.

[0031] The present invention provides a fusion intelligent AI visualization monitoring method and system with the following beneficial effects:

[0032] 1. After obtaining abnormal status information, the present invention further determines its detection results and whether the monitoring results are abnormal. If so, the parameter information of the monitoring image under the corresponding frame number is retrieved and the abnormal monitoring results are fed back to the visual monitoring platform. The visual monitoring platform evaluates the abnormal event and also sends the predicted trend of the abnormal monitoring results for auxiliary judgment, so that the staff of the visual monitoring platform can quickly judge and respond to it, facilitating the staff to monitor various abnormal events. If not, the monitoring continues until a new abnormal situation is triggered.

[0033] 2. The present invention also uses real-time monitoring, when the state information of the triggered frame a is an abnormal result set, the state information of the frame b is compared with the state information of the frame a using dynamic randomness. When the state information of the frame a is the same as the state information of the frame b, a new random number is called, and the state information and parameter information of the frame a and the frame b are output, and the monitoring under the abnormal result is no longer continued until a new abnormal result is regenerated, and the new abnormal result is monitored, which effectively reduces the calculation and processing of the same abnormal result at continuous time nodes; if the state information of the frame a is different from the state information of the frame b, a new dynamic random number c is used, wherein a<c<b, until it is satisfied, the state information of the frame a and the parameter information of the frame a and the frame c are output, thereby improving the accurate feedback of the same abnormal result at continuous time nodes and avoiding misjudgment due to external interference factors;

[0034] 3. The present invention can also extract monitoring images from frame a to frame b (or frame c) from the monitoring results through the system, and obtain the feature parameters and corresponding status information of these images. These feature parameters and status information can be used as input data of the model to train the model to identify similar abnormal situations. During the training process, through continuous iteration and optimization, the system can gradually improve the accuracy and generalization ability of the feature recognition model. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 This is a flow chart of the integrated intelligent AI visualization monitoring method of the present invention;

[0036] Figure 2 This is an abnormal condition diagram of the present invention. Figure 1 ;

[0037] Figure 3 This is an abnormal condition diagram of the present invention. Figure 2 ;

[0038] Figure 4 This is an abnormal condition diagram of the present invention. Figure 3 . DETAILED DESCRIPTION

[0039] like Figure 1-4 As shown, the present invention provides a fusion intelligent AI visualization monitoring method, which includes:

[0040] S1: Obtain surveillance video samples from the visual monitoring platform based on the cloud server, segment them into surveillance images for each frame, and extract the feature parameters and parameter information corresponding to each frame of surveillance image;

[0041] S2: Based on the feature parameters of the monitoring image as feature values ​​and the state information corresponding to each feature parameter as the target variable, a feature recognition model is constructed, and the state information corresponding to each abnormal feature parameter is marked as an abnormal result set;

[0042] S3: Acquire surveillance video in real time and import the feature parameters of the surveillance image at the corresponding frame number into the feature recognition model to obtain the state information corresponding to the feature parameters at the current frame number;

[0043] S4: Based on the status information obtained, generate monitoring results and determine whether the monitoring results are abnormal. If the monitoring results are abnormal, retrieve the parameter information of the monitoring image under the corresponding frame number and feed it back to the visual monitoring platform for evaluation together with the abnormal monitoring results, and predict the trend of the abnormal monitoring results. Otherwise, continue monitoring. Figure 1-4As shown in FIG, the fusion intelligent AI visualization monitoring method is connected through a cloud server and a visualization monitoring platform to obtain monitoring video samples of various clients and monitoring terminals connected to the visualization monitoring platform, wherein the visualization monitoring platform is Grafana, Grape City Wyn, FineReport, NetEase and Tencent Cloud platforms, which are open source visualization monitoring platforms that can obtain data from multiple data sources, and divide the acquired monitoring video samples into monitoring images of each frame, and extract the feature parameters and feature information corresponding to each frame of the monitoring image, wherein the feature parameters are extracted using SIFT algorithm, SURF algorithm, ORB algorithm, Harris corner detection algorithm, etc., and the feature parameters of each frame of the monitoring image are selected as feature values ​​respectively, and the state information corresponding to each feature parameter is used as the target variable to construct a feature recognition model, so that after extracting the feature parameters, the current corresponding state information can be obtained by comparing the feature recognition model, as shown in FIG. Figure 2 , real-time monitoring of employees not wearing safety helmets, such as Figure 3 , conduct real-time monitoring of fires within the factory area, such as Figure 4 , real-time monitoring of oil leakage in workshop equipment within the factory area is carried out. By importing the feature parameters under the corresponding frame into the model, the state information corresponding to the feature parameters is obtained and abnormal results such as employees not wearing safety helmets, fires, and oil leakage are obtained. In the process of building the model, the state information corresponding to the abnormal feature parameters is marked as an abnormal result set. When using the state model, it prevents the processing and calculation process from being occupied, so that only abnormal results are output, which can improve the efficiency of monitoring;

[0044] Specifically, in the actual monitoring process, by acquiring monitoring videos from the visual monitoring platform in real time, and extracting feature parameters and parameter information from the monitoring images under the corresponding frame number, the feature parameters are imported into the feature recognition model. When the feature parameters under the corresponding frame number are abnormal, it can output the corresponding status information. The abnormal status information corresponds to the abnormal situation of personnel or equipment. In other cases, no status information is output, so as to improve the efficiency of monitoring and reduce the occupation of the monitoring process.

[0045] Specifically, after obtaining the abnormal status information, the monitoring results are further judged to determine whether the monitoring results are abnormal. If so, the parameter information of the monitoring image under the corresponding frame number is retrieved and fed back to the visual monitoring platform together with the abnormal monitoring results, and the predicted trend of the abnormal monitoring results is also sent together. The visual monitoring platform evaluates the abnormal event so that the staff of the visual monitoring platform can quickly judge and respond to it, which is convenient for the staff to monitor various abnormal events. If not, the monitoring will continue until a new abnormal situation is triggered.

[0046] In some embodiments, in S1, the parameter information of the monitoring image is: time information and location information;

[0047] The feature parameters are: edge, corner, area, and ridge features of the monitoring image.

[0048] Specifically, the parameter information of the monitoring image is time information and location information. The time information can be used to obtain the node where the event occurred, and the location information is convenient for obtaining the location of the monitoring area, so as to facilitate the analysis and rapid response to abnormal events. When the abnormal event is triggered, the time information and location information are sent to the visual monitoring platform together. After the staff first checks the monitoring image, time and location information, they will further judge and evaluate the abnormal event.

[0049] Specifically, in the processing of surveillance images, feature parameters are used for subsequent image analysis, recognition, or tracking. Edge features are locations where grayscale features in an image change significantly, typically corresponding to the boundaries or outlines of objects. In surveillance images, when surveillance images at different frame rates change, changes in the shape and position of objects in the scene can be quickly captured. Corner features are located at the corners or intersections of objects where grayscale values ​​in surveillance images change significantly in multiple directions. Corner features remain relatively stable when the surveillance image is rotated, scaled, or changes brightness. Regional features are displayed as the overall properties of a region in a surveillance image, such as color or texture. Regional features can identify objects or scenes with specific attributes in surveillance images. Ridge features can describe long strip structures or linear features in an image. In surveillance images, spine features can correspond to linear objects such as road signs, utility poles, and tree branches. Ridge feature point detection algorithms (such as the Hough transform) can accurately extract these feature points from the image, thereby enabling the recognition and analysis of linear structures in the scene.

[0050] In some embodiments, in S2, the state information is: multiple feature parameter subsets of different shapes formed according to the types of feature parameters, and the multiple feature parameter subsets correspond to one type of state information.

[0051] Specifically, after extracting feature parameters such as edges, corners, regions, and ridge features from the monitoring image, multiple feature parameter subsets of different shapes are formed according to each type of feature parameter, and at the corresponding number of frames, feature parameter subsets of different types are formed. Multiple feature parameter subsets form feature parameters at the corresponding number of frames to form a corresponding state information.

[0052] In some embodiments, in S3, the characteristic parameters of the monitoring image at the corresponding frame number are: based on the monitoring video, the monitoring images of the ath frame and the random bth frame are respectively obtained, where a<b, and the corresponding characteristic parameters are respectively extracted, and the characteristic parameters are imported into the feature recognition model to obtain the status information of the ath frame and the status information of the bth frame respectively; wherein, when the characteristic parameters of the monitoring image at the corresponding frame number in the monitoring video sample change, the frame number of the current monitoring image is marked as a.

[0053] Specifically, when determining the characteristic parameters of a surveillance image at a corresponding frame number, based on the real-time acquired surveillance video, surveillance images of the ath frame and the dynamically random bth frame are acquired, where a<b. The ath frame surveillance image is the one where the characteristic parameters of the surveillance image at the corresponding frame number in the real-time acquired surveillance image change, and the frame number is recorded as the ath frame. The characteristic parameters are extracted respectively, and the state information of the ath frame and the bth frame are acquired using a feature recognition model. The surveillance images of the ath frame and the bth frame are used to form a comparison, thereby avoiding misjudgment due to interference from external factors during real-time monitoring, making the monitoring results more accurate. At the same time, the change from the ath frame to the bth frame can be used to predict the subsequent change trend of the abnormal condition and provide a basis for evaluation. For example, when oil leaks occur in equipment in a factory, shadows may be cast by outdoor buildings when exposed to sunlight. By introducing the surveillance image of the ath frame at the time of the change and the dynamically random bth frame surveillance image, a further comparison is formed, providing a scientific basis for the evaluation of the results and preventing misjudgment of abnormal events.

[0054] In some embodiments, in S4, based on the obtained status information, a monitoring result is generated as follows: based on the monitoring image of the a-th frame and the monitoring image of the b-th frame, determine whether the status information of the a-th frame belongs to the abnormal result set; if not, continue monitoring; otherwise, compare the obtained status information of the a-th frame and the status information of the b-th frame respectively to determine whether they are the same; if they are not the same, re-establish a new random number c, where a<c<b, extract the characteristic parameters of the c-th frame, obtain the status information of the c-th frame, and re-compare it with the status information of the a-th frame until the status information of the c-th frame is the same as the status information of the a-th frame; if the status information of the a-th frame is the same as the status information of the b-th frame or the status information of the c-th frame, determine that the status information of the a-th frame is an abnormal result, and transmit the status information of the a-th frame, the parameter information of the a-th frame, the parameter information of the b-th frame or the parameter information of the c-th frame to the visualization monitoring platform.

[0055] Specifically, when the state parameters under the corresponding frame number generate corresponding state information using the feature recognition model, based on the monitoring images of the a-th frame and the b-th frame, determine whether the state information of the a-th frame belongs to the abnormal result set. If not, continue monitoring. Otherwise, compare the state information of the a-th frame with the state information of the b-th frame to determine whether the state information used is the same. If not, re-establish a new dynamic random number c, where a<c<b, extract the state information of the c-th frame and compare it with the state information of the a-th frame again, and compare the types and quantities of the feature parameter subsets until the state information of the c-th frame is the same. If the state information of the a-th frame is the same as the state information of the a-th frame, and the state information under the corresponding frame number is the abnormal event corresponding to the monitoring image of the a-th frame, then the event corresponding to the state information of the a-th frame is output as the abnormal result, and the state information of the a-th frame, the parameter information of the a-th frame, the parameter information of the b-th frame, or the parameter information of the c-th frame is transmitted to the visual monitoring platform. The visual monitoring platform retrieves the video under the corresponding time node, obtains the abnormal result according to the fed-back state information of the a-th frame, compares it with the video information, and uses the trend of the fed-back abnormal result to make further judgments and generate a response to the abnormal event;

[0056] Specifically, in real-time monitoring, when the state information of the triggered frame a is an abnormal result set, the state information of the b frame is compared with the state information of the a frame using dynamic randomness. When the state information of the a frame is the same as the state information of the b frame, a new random number is called, and the state information and parameter information of the a frame and the b frame are output. The monitoring under the abnormal result is no longer continued until a new abnormal result is regenerated and the new abnormal result is monitored, which effectively reduces the calculation and processing of the same abnormal results at continuous time nodes. If the state information of the a frame is different from the state information of the b frame, a new dynamic random number c is used, where a<c<b, until it is satisfied, and the state information of the a frame and the parameter information of the a frame and the c frame are output, which improves the accurate feedback of the same abnormal results at continuous time nodes and avoids misjudgment due to external interference factors.

[0057] In some embodiments, in S4, the trend of the predicted abnormal monitoring results is: if the status information of the a-th frame is the same as the status information of the b-th frame or the status information of the c-th frame, the difference calculation is performed using the feature parameter subset of the a-th frame and the feature parameter subset of the b-th frame or the feature parameter subset of the c-th frame to obtain a difference feature parameter set, and the time from the a-th frame to the b-th frame or the c-th frame is obtained to obtain the change trend of the difference feature parameter set per unit time, and generate a predicted trend of the abnormal monitoring results.

[0058] Specifically, when predicting the abnormal monitoring results, when the status information of the a-th frame is the same as the status information of the c-th frame or the status information of the a-th frame is the same as the status information of the c-th frame, the difference calculation is performed using the feature parameter subset of the a-th frame and the feature parameter subset of the b-th frame or the feature parameter subset of the c-th frame to obtain a difference feature parameter set, and based on the corresponding time node, the corresponding change trend of the difference feature parameter set per unit time is obtained. According to historical data and statistical models, a threshold range of the change trend corresponding to the difference feature parameter set per unit time can be established, and different response levels can be set within the corresponding threshold range. When the predicted trends corresponding to different abnormal events are monitored, the corresponding response levels are generated to improve the processing capabilities of different abnormal events.

[0059] In some embodiments, the visual monitoring platform performs evaluation as follows: the visual monitoring platform evaluates the abnormal monitoring results and feeds back the actual results, compares the actual results with the abnormal monitoring results, and if the deviation between the actual results and the abnormal monitoring results is within the deviation threshold range, the feature recognition model is trained using the feature parameters and corresponding status information of the a-th frame monitoring image to the b-th frame monitoring image or the c-th frame monitoring image in the abnormal monitoring results.

[0060] Specifically, after obtaining the monitoring results, the staff will combine the monitoring results and video information to generate the corresponding actual results. After comparing the actual results with the monitoring results, the system will calculate the deviation between them and judge whether the accuracy of the monitoring results is within an acceptable range based on the dynamic deviation threshold set by the system. The setting of the deviation threshold is based on comprehensive consideration of historical data, domain knowledge and business needs. A threshold that is too loose may cause the system to ignore abnormal situations, while a threshold that is too tight may cause the system to be overly sensitive and generate too many false alarms. If the deviation between the actual result and the abnormal monitoring result is within the set deviation threshold, then the monitoring result can be considered relatively accurate. In this case, the system can use these accurate monitoring results to train the feature recognition model.

[0061] Specifically, the system can extract monitoring images from frame a to frame b (or frame c) from the monitoring results, and obtain the feature parameters and corresponding status information of these images. These feature parameters and status information can be used as input data for the model to train the model to identify similar abnormal situations. During the training process, the system can adopt various machine learning algorithms and techniques, such as neural networks, support vector machines, decision trees, etc.; through continuous iteration and optimization, the accuracy and generalization ability of the feature recognition model can be gradually improved.

[0062] Specifically, after training is completed, the feature recognition model needs to be evaluated based on the accuracy, efficiency and prediction trend of each abnormal event. This can be done by applying the model to new monitoring data and comparing its predicted results with the actual results. If there is a large deviation between the model's predicted results and the actual results, adjust the model parameters or adopt a new algorithm to optimize the model.

[0063] The present invention also provides a fusion intelligent AI visualization monitoring system, which adopts the fusion intelligent AI visualization monitoring method described in any of the previous items, including: a visualization monitoring platform, the fusion intelligent AI visualization monitoring system is connected to the visualization monitoring platform via the Internet; an information acquisition module, used to obtain monitoring video from the visualization monitoring platform; an extraction module, used to generate monitoring images at a corresponding frame number from the monitoring video, and extract feature parameters and parameter information of the monitoring images at a corresponding frame number from the monitoring images at a corresponding frame number; a feature recognition model module, used to generate status information at the corresponding frame number based on the feature parameters at the corresponding frame number obtained in real time, and obtain monitoring results based on the status information; a control module, respectively connected to the information acquisition module, the extraction module and the feature recognition model module, and the control module is also configured as a control storage subunit with a time storage function to store parameter information, status information and monitoring results in the corresponding process.

[0064] Specifically, during real-time monitoring, the control module controls the information acquisition module to obtain monitoring video in real time from the visual monitoring platform through the Internet, the control module controls the extraction module to segment the monitoring images into corresponding frame numbers, and extracts the feature parameters and parameter information of the monitoring images under the corresponding frame numbers from the monitoring images under the corresponding frame numbers, the control module controls the import of the corresponding feature parameters into the feature recognition model module, generates the status information under the corresponding frame number based on the feature parameters under the corresponding frame number obtained in real time, and obtains the monitoring results based on the status information; wherein, during real-time monitoring, the parameter information, status information and monitoring results under the corresponding frame number of the abnormal event are stored in the control module, so as to facilitate the subsequent feedback to the visual monitoring platform for evaluation.

[0065] In some embodiments, the extraction module further includes: an extraction marking unit, configured to mark corresponding frame numbers when characteristic parameters of surveillance images at corresponding frame numbers in the surveillance video sample change.

[0066] Specifically, in real-time monitoring, when the characteristic parameters of the monitoring image corresponding to the frame number in the monitoring image are changed, the extraction and marking unit marks the frame number as the a-th frame, and begins to judge whether the status information corresponding to the frame number is an abnormal result. When the judgment is completed at the frame number, when the characteristic parameters of the monitoring image corresponding to the frame number in the new monitoring image are changed, a new judgment is started. In this way, when there is no change in the monitoring image, the subsequent judgment processing is not performed, reducing false touches and occupancy of the monitoring process.

[0067] In some embodiments, a processing module is further included: the processing module is connected to the control module, and the processing module includes a judgment and comparison module and a solution module, the judgment and comparison module is used to judge whether the monitoring result is abnormal, whether the characteristic parameters of the monitoring image at the corresponding frame number in the monitoring video sample have changed, whether the state information at the corresponding frame number is the same, and whether the deviation between the actual result and the abnormal monitoring result is within the deviation threshold range;

[0068] The solution module performs difference calculation using a feature parameter subset of the ath frame and a feature parameter subset of the bth frame or a feature parameter subset of the cth frame to obtain a difference feature parameter set, and obtains the time from the ath frame to the bth frame or the cth frame to obtain a change trend of the difference feature parameter set per unit time.

[0069] Specifically, the judgment and comparison module determines whether the monitoring result is abnormal. If so, the parameter information of the monitoring image under the corresponding frame number is retrieved and the abnormal monitoring result is fed back to the visual monitoring platform; the judgment and comparison module extracts the marking unit to mark the frame number as the a-th frame according to whether the feature parameters of the monitoring image under the corresponding frame number in the monitoring video sample have changed; the judgment and comparison module determines whether the state information under the corresponding frame number is the same, and the judgment and comparison module determines whether the deviation between the actual result and the abnormal monitoring result is within the deviation threshold range. If the deviation between the actual result and the abnormal monitoring result is within the set deviation threshold range, then it can be considered that the monitoring result is relatively accurate. In this case, the system can use these accurate monitoring results to train the feature recognition model;

[0070] Specifically, the solution module performs difference calculation using the characteristic parameters of the a-th frame and the characteristic parameters of the b-th frame or a subset of the characteristic parameters of the c-th frame to obtain a difference characteristic parameter set, and obtains the time from the a-th frame to the b-th frame or the c-th frame to obtain the change trend of the difference characteristic parameter set per unit time, and based on the corresponding time node, obtains the corresponding change trend of the difference characteristic parameter set per unit time. According to historical data and statistical models, a threshold range of the change trend corresponding to the difference characteristic parameter set per unit time can be established, and different response levels can be set within the corresponding threshold range. When the predicted trends corresponding to different abnormal events are monitored, the corresponding response levels are generated to improve the processing capabilities of different abnormal events.

[0071] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention. The above description is merely a preferred embodiment of the present invention. It should be noted that those skilled in the art can make various improvements and variations without departing from the technical principles of the present invention, and such improvements and variations shall also be considered within the scope of protection of the present invention.

Claims

1. A fusion intelligent AI visualization monitoring method, characterized in that: include: S1: Obtain surveillance video samples from the visual monitoring platform based on the cloud server, segment them into surveillance images for each frame, and extract the feature parameters and parameter information corresponding to each frame of surveillance image; S2: Based on the feature parameters of the monitoring image as feature values ​​and the state information corresponding to each feature parameter as the target variable, a feature recognition model is constructed, and the state information corresponding to each abnormal feature parameter is marked as an abnormal result set; S3: Acquire surveillance video in real time and import the feature parameters of the surveillance image at the corresponding frame number into the feature recognition model to obtain the state information corresponding to the feature parameters at the current frame number; S4: Based on the obtained status information, a monitoring result is generated and whether the monitoring result is abnormal is determined. If the monitoring result is abnormal, parameter information of the monitoring image at the corresponding frame number is retrieved and fed back to the visual monitoring platform together with the abnormal monitoring result for evaluation, and the trend of the abnormal monitoring result is predicted. Otherwise, monitoring continues; In S3, the characteristic parameters of the surveillance image corresponding to the number of frames are: Based on the surveillance video, surveillance images of the ath frame and the random bth frame are obtained, where a<b, and corresponding feature parameters are extracted respectively. The feature parameters are imported into the feature recognition model to obtain state information of the ath frame and state information of the bth frame respectively; Among them, when the characteristic parameters of the surveillance image at the corresponding frame number in the surveillance video sample change, the frame number of the current surveillance image is marked as a; In S4, based on the obtained status information, a monitoring result is generated: Based on the monitoring image of frame a and the monitoring image of frame b, determine whether the status information of frame a belongs to the abnormal result set. If not, continue monitoring. Otherwise, compare the obtained status information of frame a with the status information of frame b to determine whether they are the same. If they are not the same, a new random number c is re-established, where a<c<b, and the characteristic parameters of the c-th frame are extracted, the state information of the c-th frame is obtained, and the state information of the c-th frame is compared with the state information of the a-th frame again until the state information of the c-th frame is the same as the state information of the a-th frame; If the status information of the a-th frame is the same as the status information of the b-th frame or the status information of the c-th frame, the status information of the a-th frame is determined to be an abnormal result, and the status information of the a-th frame, the parameter information of the a-th frame, the parameter information of the b-th frame or the parameter information of the c-th frame are transmitted to the visual monitoring platform; In S4, the trend of the predicted abnormal monitoring result is: If the status information of the a-th frame is the same as the status information of the b-th frame or the status information of the c-th frame, the difference calculation is performed using the feature parameter subset of the a-th frame and the feature parameter subset of the b-th frame or the feature parameter subset of the c-th frame to obtain a difference feature parameter set, and the time from the a-th frame to the b-th frame or the c-th frame is obtained to obtain the change trend of the difference feature parameter set per unit time, and generate a predicted trend of the abnormal monitoring results.

2. The integrated intelligent AI visualization monitoring method according to claim 1 is characterized by: In said S1, the parameter information of the monitoring image is: time information and location information; The feature parameters are: edge, corner, area, and ridge features of the monitoring image.

3. The integrated intelligent AI visualization monitoring method according to claim 1 is characterized by: In the above S2, the state information is: a plurality of feature parameter subsets of different shapes formed according to the types of feature parameters, and the plurality of feature parameter subsets correspond to one type of state information.

4. The integrated intelligent AI visualization monitoring method according to claim 1 is characterized by: The visual monitoring platform performs evaluation as follows: the visual monitoring platform evaluates the abnormal monitoring results and feeds back the actual results, compares the actual results with the abnormal monitoring results, and if the deviation between the actual results and the abnormal monitoring results is within a deviation threshold range, uses the feature parameters and corresponding state information of the a-th frame monitoring image to the b-th frame monitoring image or the c-th frame monitoring image in the abnormal monitoring results to train a feature recognition model.

5. A fusion intelligent AI visual monitoring system, characterized by: The fusion intelligent AI visualization monitoring method according to any one of claims 1 to 4 comprises: A visual monitoring platform, wherein the integrated intelligent AI visual monitoring system is connected to the visual monitoring platform via the Internet; An information acquisition module, configured to acquire monitoring videos from the visual monitoring platform; An extraction module is used to generate a surveillance image at a corresponding frame number from the surveillance video, and extract feature parameters and parameter information of the surveillance image at the corresponding frame number from the surveillance image at the corresponding frame number; The feature recognition model module is used to generate state information of the corresponding frame number based on the feature parameters of the corresponding frame number obtained in real time, and obtain monitoring results based on the state information; The control module is respectively connected to the information acquisition module, the extraction module and the feature recognition model module. The control module is also configured as a control storage subunit with a time storage function to store parameter information, status information and monitoring results in the corresponding process.

6. The integrated intelligent AI visual monitoring system according to claim 5 is characterized by: The extraction module further includes: an extraction marking unit, which is used to mark the corresponding frame number when the feature parameters of the monitoring image at the corresponding frame number in the monitoring video sample change.

7. The integrated intelligent AI visual monitoring system according to claim 5 is characterized by: The system further includes a processing module: the processing module is connected to the control module, and the processing module includes a judgment and comparison module and a solution module, the judgment and comparison module is used to judge whether the monitoring result is abnormal, whether the characteristic parameters of the monitoring image at the corresponding frame number in the monitoring video sample have changed, whether the state information at the corresponding frame number is the same, and whether the deviation between the actual result and the abnormal monitoring result is within the deviation threshold range; The solution module performs difference calculation using a feature parameter subset of the ath frame and a feature parameter subset of the bth frame or a feature parameter subset of the cth frame to obtain a difference feature parameter set, and obtains the time from the ath frame to the bth frame or the cth frame to obtain a change trend of the difference feature parameter set per unit time.

Images