Risk identification method and device based on conversation video, equipment and medium

Abstract

The embodiment of the invention provides a conversation video-based risk identification method, device, equipment and medium, and the method comprises the steps: firstly obtaining video data of a conversation room, decomposing the video data into a video image and audio data, then selecting an ROI region based on the video image, combining with background modeling, and then employing an IP-GoDec method to identify the risk of the conversation video. And comparing pixel values in the foreground and the grey-scale map, judging the state of the conversation room environment according to the risk identification threshold parameter of the conversation room environment to obtain a conversation room environment risk identification result, and performing conversation room object risk identification by using the risk identification threshold parameter of the conversation room object to obtain a conversation room object risk identification result. And finally obtaining and displaying a total risk identification result. According to the method, background modeling is carried out by using a Gaussian mixture model, foreground segmentation is carried out in combination with IP-GoDec to improve robustness, and finally risk identification of a video image is completed, so that the method has excellent performance in video monitoring and target tracking application.

Description

Technical Field

[0001] The present application relates to the technical field of risk identification, and in particular to a risk identification method, device, equipment and medium based on conversation video. Background Art

[0002] With the rapid development of information technology, video data plays an increasingly critical role in the field of digital government and government supervision. In particular, conversation videos have been widely used in multiple scenarios due to their rich information and intuitive features. However, these video contents may also contain hidden risks, thus posing potential security risks. Faced with such challenges, existing solutions mostly rely on manual review mechanisms, which are not only inefficient but also costly.

[0003] In recent years, some studies have begun to explore the use of image and sound pattern recognition technology to automatically detect risky content in videos. Although this direction shows the possibility of automated processing, current methods often require complex algorithm design and rely heavily on large amounts of labeled data sets for training, which makes their practical application face two major obstacles: accuracy and resource investment. In particular, in the government data management scenario, the principle of keeping data out of the system and the difficulty of obtaining relevant data further limit the effective deployment and utilization of the above advanced technologies.

[0004] Therefore, how to develop a solution that meets the requirements of government security risk management and is efficient and feasible has become a technical problem that needs to be solved urgently. Summary of the invention

[0005] The present application provides a risk identification method, device, equipment and medium based on conversation video to complete risk identification of video images, so that the method can perform well in video monitoring and target tracking applications.

[0006] In order to achieve the above purpose, the technical solution adopted in the embodiment of the present application is as follows:

[0007] In a first aspect, an embodiment of the present application provides a risk identification method based on a conversation video, the method comprising:

[0008] Acquire video data of the conversation room and decompose the video data into video images and audio data;

[0009] Based on the video image, the ROI area is selected and combined with the background modeling. Then the IP-GoDec method is used to compare the pixel values ​​in the foreground and grayscale images. The state of the conversation room environment is judged according to the risk identification threshold parameters of the conversation room environment, and the conversation room environment risk identification results are obtained.

[0010] Based on the video image, the risk identification threshold parameter of the conversation room object is used to perform the conversation room object risk identification to obtain the conversation room object risk identification result;

[0011] Based on the conversation room environment risk identification results and the conversation room object risk identification results, the overall risk identification results are obtained and displayed.

[0012] In a possible implementation, based on the video image, the ROI area is selected and combined with the background modeling, and then the IP-GoDec method is used to compare the pixel values ​​in the foreground and the grayscale image, and the state of the conversation room environment is judged according to the risk identification threshold parameter of the conversation room environment, and the steps of obtaining the conversation room environment risk identification result include:

[0013] Determine the ROI image area of ​​the conversation room environment based on the video image;

[0014] Use Gaussian mixture model to perform background modeling on the ROI image area to obtain a background model;

[0015] Use IP-GoDec to perform foreground segmentation to obtain the foreground area, and perform difference analysis between the pixel value of the foreground area and the background grayscale image in the background model to obtain the grayscale difference;

[0016] The grayscale difference is compared with the risk identification threshold parameter of the conversation room environment to determine the state of the conversation room environment.

[0017] In a possible implementation, the step of determining the ROI image area of ​​the conversation room environment based on the video image includes:

[0018] Performing an initial frame selection on the video image to obtain an initial frame selection area;

[0019] The secondary selection area is obtained by performing area segmentation and polygon selection on the primary selection area;

[0020] The secondary framed area is manually annotated to determine the ROI image area of ​​the conversation room environment.

[0021] In a possible implementation, the step of decomposing the video data into video images and audio data includes:

[0022] Import the video data into the video processing software or development environment, and decode it through the decoding library to obtain image frames and audio samples;

[0023] Convert the audio samples into a processable format, index the image frames in time order, and then extract the image frames at a time point or within a time range to obtain the video image;

[0024] Align audio samples and video images using timestamps.

[0025] In a possible implementation manner, after the step of aligning the audio sample and the video image using the timestamp, the method further includes:

[0026] Performing format conversion, scaling, and cropping on the video image to obtain the processed video image;

[0027] The audio samples are denoised, data enhanced, and format converted to obtain processed audio samples.

[0028] In a possible implementation, the video data of the conversation room is automatically accessed through the GB28181 standard, or the video stream address of the camera is manually obtained and configured into the system.

[0029] In a second aspect, the embodiment of the present application further proposes a risk identification device based on a conversation video, the device comprising:

[0030] An acquisition module, used for acquiring video data of the conversation room and decomposing the video data into video images and audio data;

[0031] The first risk identification module is used to select the ROI area based on the video image and combine it with the background modeling, and then use the IP-GoDec method to compare the pixel values ​​in the foreground and the grayscale image, and judge the state of the conversation room environment according to the risk identification threshold parameter of the conversation room environment to obtain the conversation room environment risk identification result;

[0032] The second risk identification module is used to perform risk identification of objects in the conversation room based on the video image and using the risk identification threshold parameter of the objects in the conversation room to obtain a risk identification result of the objects in the conversation room;

[0033] The display module is used to obtain and display the total risk identification result based on the conversation room environment risk identification result and the conversation room object risk identification result.

[0034] In a third aspect, the present application further proposes a computer device, comprising a processor and a memory, wherein the memory stores a computer program, and the computer program is loaded and executed by the processor to implement the risk identification method as described in any one of the first aspects.

[0035] In a fourth aspect, the present application further proposes a computer-readable storage medium, in which a computer program is stored. The computer program is loaded and executed by a processor to implement the risk identification method as described in any one of the first aspects.

[0036] The above-mentioned main scheme of the present application and its further options can be freely combined to form multiple schemes, all of which are schemes that can be adopted and claimed for protection in the present application; and in the present application, (non-conflicting options) options and other options can also be freely combined. After understanding the scheme of the present application, those skilled in the art can understand that there are multiple combinations based on the prior art and common knowledge, all of which are technical schemes to be protected by the present application, and they are not exhaustively listed here.

[0037] Compared with the prior art, the embodiments of the present application provide a risk identification method, device, equipment and medium based on conversation video, firstly, the video data of the conversation room is obtained and decomposed into video image and audio data, then based on the video image, the ROI area is selected and combined with the background modeling, and then the IP-GoDec method is used to compare the pixel values ​​in the foreground and the grayscale image, and the state of the conversation room environment is judged according to the risk identification threshold parameters of the conversation room environment to obtain the conversation room environment risk identification result, and the conversation room object risk identification is performed using the risk identification threshold parameters of the conversation room objects to obtain the conversation room object risk identification result, and finally the total risk identification result is obtained and displayed. The Gaussian mixture model is used for background modeling, and the foreground segmentation is combined with IP-GoDec to improve the robustness, and finally the risk identification of the video image is completed, so that the method can perform well in video monitoring and target tracking applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0039] Figure 1 A flow chart of a risk identification method based on conversation video proposed in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0041] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0042] It should be noted that, in the absence of conflict, the features in the embodiments of the present application may be combined with each other.

[0043] In order to solve the problems of low efficiency, high cost and insufficient accuracy in the prior art when processing whether there are security risks in large amounts of video data in the field of digital government and government supervision, the embodiments of the present application propose a risk identification method, device, equipment and medium based on conversation video to perform risk identification of video images, which can perform well in video surveillance and target tracking applications, and are described below.

[0044] The risk identification method is applicable to a variety of scenarios, including: interrogation in the disciplinary inspection and supervision interview room, customs anti-smuggling interview room, traffic accident mediation interview, border defense interview of the National Immigration Administration, emergency fire disaster assessment interview, judicial police legal education interview of the Justice Bureau, urban renewal demolition resettlement compensation interview, property coordination neighborhood interview, arbitration and reconciliation coordination interview, and financial business interview with qualified investors. In these scenarios, the present invention can provide efficient and accurate risk identification services to help institutions and organizations better manage and control potential risks in the content of the conversation.

[0045] Please refer to Figure 1 , Figure 1 A flow chart of a risk identification method based on conversation video proposed in an embodiment of the present application is shown, and the method includes:

[0046] Step S1, obtaining video data of a conversation room and decomposing the video data into video images and audio data.

[0047] The video data acquisition of the conversation room is automatically accessed through the GB28181 standard, or the video stream address of the camera is manually obtained and configured into the system.

[0048] Automatic system access through GB28181 standard is applicable to the situation that the existing system already supports GB28181 standard and is not occupied by other systems. First confirm whether the existing system supports GB28181 standard, then configure the relevant parameters of GB28181 (such as SIP server address, port, etc.), and finally start the automatic system access function to complete the access of video data.

[0049] Manually obtain the video stream address of the camera and configure it to the system. It is applicable to various systems and can connect multiple cameras at the same time. First, obtain the video stream address of the camera (usually RTSP or HTTP URL), configure these video stream addresses in the system, ensure that the system can correctly read the video data, and then test to ensure that the video stream can be transmitted normally.

[0050] According to the specific environment and needs of the conversation room, choose a high-definition camera, night vision camera or wide-angle camera, install the camera at the entrance, exit and problem-prone areas of the conversation room, fix the camera to a suitable location such as the ceiling or wall, ensure that the angle covers the area that needs to be monitored, connect the camera's power cord and network cable to ensure that the camera can start and transmit data normally. Then configure a static or dynamic IP address for the camera so that it can communicate with other devices in the network, connect the camera to a switch or router, ensure that the camera can access the Internet or LAN normally, configure port forwarding or VPN to access the surveillance camera from a remote location, and perform a network test on the camera to ensure smooth data transmission.

[0051] Select local storage or cloud storage. Local storage includes hard disk video recorders (DVR) or network attached storage (NAS). Cloud storage relies on third-party services. If you choose local storage, connect the storage device to the camera through the local area network and allocate appropriate storage space; if you choose cloud storage, register the relevant service and create a bucket, and configure data encryption and backup policies to ensure data security. Get the video stream address in the DVR or cloud storage service. If you use a DVR, get the video stream address of the DVR (usually an RTSP URL); if you use cloud storage, get the video stream address provided by the cloud storage service.

[0052] The steps of decomposing video data into video images and audio data include:

[0053] Import the video data into the video processing software or development environment, and decode it through the decoding library to obtain image frames and audio samples;

[0054] Convert the audio samples into a processable format, index the image frames in time order, and then extract the image frames at a time point or within a time range to obtain the video image;

[0055] Align audio samples and video images using timestamps.

[0056] Share a single video file into an audio data file and a video file. The whole process will store the extracted video images and audio data in suitable formats, for example, image frames can be stored in image formats such as JPEG and PNG, and audio data can be stored in audio formats such as WAV and MP3. The present invention focuses on risk identification of video images, so the audio data is not further processed. If necessary, corresponding processing and risk identification operations can also be performed.

[0057] Import the recorded video data into the video processing software or development environment. You can use a variety of tools, such as FFmpeg, GStreamer, OpenCV, etc., and then use the decoding library to decode the video data. The decoding library includes FFmpeg, GStreamer, OpenCV, etc. The decoding process converts the compressed video code (such as H.264, H.265, MPEG-4, etc.) into raw image frames and audio samples. After decoding, the video data is decomposed into a series of image frames, each frame is a separate picture, usually represented in RGB, BGR or other pixel formats. The frames in the video stream are indexed in chronological order, and can be indexed by frame number or timestamp to facilitate the extraction of image frames at a specific time point or time range.

[0058] The audio data in the video stream is extracted and saved in pulse code modulation (PCM) or other audio coding format. The audio data is extracted and converted into a processable format, such as waveform data or other digital audio format, for further analysis and processing. Finally, it is necessary to ensure the synchronization of image frames and audio data in time, and use timestamps or other synchronization mechanisms to ensure that image frames are aligned in time with corresponding audio samples.

[0059] After the step of aligning the audio samples and video images using timestamps, also include:

[0060] Performing format conversion, scaling, and cropping on the video image to obtain the processed video image;

[0061] The audio samples are denoised, data enhanced, and format converted to obtain processed audio samples.

[0062] Post-process the extracted video images to convert the original video into a form more suitable for specific application scenarios. Specific operations include format conversion (such as converting from MOV to MP4), scaling (adjusting the resolution to adapt to the display requirements of different devices), and cropping (removing unnecessary parts and focusing on the area of ​​interest). Process the extracted audio data to improve the sound quality or ensure that the audio meets the requirements of subsequent analysis tools. The processing methods may involve removing background noise, improving speech clarity (such as through volume equalization), and converting the audio into different encoding formats (for example, from WAV to the more compact MP3 format).

[0063] The processed video frames are usually saved as common image file formats (such as JPEG or PNG), while the audio clips can be encoded into WAV, MP3 and other formats for easy management and further use. In order to simplify the entire workflow and improve work efficiency, you can develop scripts or use existing automation software to implement one-click execution of all the above steps, which greatly reduces the need for manual intervention and reduces the possibility of errors caused by human factors.

[0064] Executing the above steps can effectively decompose the video obtained from the surveillance camera or video file into independent video images and audio data, thereby providing data support for various applications such as computer vision analysis, speech recognition, and security monitoring.

[0065] For conversation videos, the privacy of the conversation environment is an important risk identification factor for whether the conversation video is compliant. If the entire conversation process is compliant, but the conversation environment itself is not compliant or the content of the conversation has the risk of being eavesdropped and leaked, this conversation video still needs to be defined as a risky conversation video, so risk identification of the conversation room environment becomes very important.

[0066] Step S2: Based on the video image, select the ROI area and combine it with the background modeling. Then use the IP-GoDec method to compare the pixel values ​​in the foreground and grayscale images, judge the state of the conversation room environment according to the risk identification threshold parameters of the conversation room environment, and obtain the conversation room environment risk identification result.

[0067] The conversation room environment can be set as a door, and the risk identification threshold parameter of the conversation room environment can be set to a (0,1) value. The default value is set to 0.45. The greater the deviation of the threshold from the default value, the lower the sensitivity of risk identification; the smaller the absolute value deviation of the threshold from the default value, the higher the sensitivity of risk identification.

[0068] Based on the video image, the ROI area is selected and combined with the background modeling, and then the IP-GoDec method is used to compare the pixel values ​​in the foreground and the grayscale image, and the state of the conversation room object is judged according to the risk identification threshold parameter of the conversation room object to obtain the conversation room environment risk identification result, including:

[0069] Determine the ROI image area of ​​the conversation room environment based on the video image;

[0070] Use Gaussian mixture model to perform background modeling on the ROI image area to obtain a background model;

[0071] Use IP-GoDec to perform foreground segmentation to obtain the foreground area, and perform difference analysis between the pixel value of the foreground area and the background grayscale image in the background model to obtain the grayscale difference;

[0072] The grayscale difference is compared with the risk identification threshold parameter of the conversation room object to determine the state of the conversation room object.

[0073] First, determine the ROI (Region of Interest) to identify the part of the image that needs special attention, that is, the area where the door of the conversation room is located. Initially, manually or by using preset parameters to select the approximate area, perform more detailed area segmentation and polygon selection to accurately define the ROI, and manually annotate the ROI in the screenshot based on prior knowledge to avoid interference from moving objects. Background modeling techniques such as Gaussian Mixture Model (GMM) are used to distinguish between static background and foreground objects that may change, in which the modeling is performed using the precisely defined ROI and the information of the entire image.

[0074] After obtaining a stable background model, the video frame is input to the IP-GoDec algorithm, which focuses on the gradient information in the image, that is, the rate of change of pixel values. By analyzing these gradient differences, the method can better capture detailed features such as object edges and textures. Combined with the background information provided by GMM, IP-GoDec can consider more dimensions of information when identifying the foreground, so that even objects with similar colors or textures to the background can be accurately segmented.

[0075] After using GMM to determine the approximate background, IP-GoDec is used to further optimize the foreground detection results. For situations that are difficult to distinguish (such as the glass room scene in the conversation room), the solution that combines GMM and IP-GoDec technology is very suitable for complex monitoring environments, such as the presence of reflective surfaces (such as glass), changing lighting conditions, or the presence of occlusions. It can not only effectively extract moving targets from the background, but also ensure high performance under various challenging conditions.

[0076] Then, we compare the pixel values ​​and the grayscale difference between the foreground and the background. We calculate the difference between the pixels in the foreground area and the background grayscale image. Finally, we set a reasonable threshold to distinguish the different states of the door (such as closed and open), and adjust it based on experimental data and actual application scenario requirements.

[0077] The generation process of the risk identification results of the conversation room environment based on the above is: obtain video data under various conditions (such as lighting changes, door opening and closing processes, etc.), perform preliminary processing such as noise reduction and conversion to grayscale images, develop adaptive algorithms to accurately find the location of the door in different situations, and use models such as GMM to continuously learn and adjust the understanding of the normal background. Apply IP-GoDec technology to enhance the clarity of the boundary between the moving object and the fixed background, quantify the difference in color intensity between the foreground and the known background, and determine whether the door is open or closed in the current state based on pre-set standards, continuously improve system performance, and ensure efficient and accurate completion of tasks.

[0078] The advantages of using computer vision-based conversation room environment (door) state recognition are:

[0079] First, it is non-contact. There is no need to modify the overall structure of the existing conversation room, no need to pull wires, no high voltage involved, and no new fire hazard risk factors caused by modifying the physical decoration environment of the conversation room. Therefore, it will not bring original safety hazards to the overall physical environment of the conversation room.

[0080] Second, it has strong scalability and supports the recognition of the status of the conversation room door and the detection of people entering and exiting through different algorithms. There is no need to install new sensor hardware, and it is convenient to expand new functions.

[0081] Third, traceability: if the door of the conversation room is not opened or closed normally, the real-time status can be saved by taking screenshots, which can serve as strong evidence for accident accountability. In addition, due to the popularity of video surveillance equipment, the use of existing surveillance equipment can further reduce economic costs and avoid repeated investment in hardware updates.

[0082] In addition, although the facility layout and environment in different conversation rooms may vary, by training computer vision algorithms, the door status in different environments can be effectively identified, which has stronger generalization capabilities.

[0083] The step of determining the ROI image area of ​​the conversation room environment based on the video image includes:

[0084] Performing an initial frame selection on the video image to obtain an initial frame selection area;

[0085] The secondary selection area is obtained by performing area segmentation and polygon selection on the primary selection area;

[0086] The secondary framed area is manually annotated to determine the ROI image area of ​​the conversation room environment.

[0087] Select a rough area of ​​interest through preliminary observation or based on certain assumptions. For example, in a video stream, if the goal is to track a door, first roughly mark the range where the door is located. In order to further improve the accuracy, more detailed operations are performed within the selected rough area. This can be done by splitting the large area into several small parts or directly using the polygon tool to outline a small area that fits the actual shape better. In order to ensure that the selected ROI is absolutely accurate, it is very useful to fine-tune or confirm the results manually, especially in the face of complex scenes. For example, it may be necessary to exclude some temporary obstacles (such as passers-by) or irrelevant factors such as timestamps.

[0088] The steps of using the Gaussian mixture model to perform background modeling on the ROI image region to obtain the background model include:

[0089] Select a frame in the video image as an initial frame;

[0090] Initialize the Gaussian mixture model parameters of each component in the Gaussian mixture model using the color value of each pixel in the initial frame. The Gaussian mixture model parameters include mean, covariance matrix and weight;

[0091] The EM algorithm is used to iteratively adjust the parameters of the Gaussian mixture model to obtain the background model.

[0092] A frame in a video sequence is selected as the basis of the background model. This frame should represent the background state as much as possible, that is, there is no interference from the foreground object, such as the state of a closed door and no people or objects in front. For the selected initial frame, the color value of each pixel is used to initialize the mean, covariance matrix, and weight parameters of each component in the GMM. These parameters reflect the characteristics of the color distribution at that position. As the video continues to play, each time a new frame is encountered, the EM (Expectation Maximization) algorithm is used to iteratively adjust the previously set GMM parameters so that the model can more accurately reflect the actual situation of the current background.

[0093] In the process of finding the expectation, it is necessary to calculate the posterior probability that each pixel belongs to each Gaussian component. The maximization process uses these posterior probabilities to update the mean, covariance matrix and weight of each Gaussian component.

[0094] Once a relatively stable background model is available, each pixel in the subsequent frames can be classified to determine whether it belongs to the background or foreground, and a threshold is set to compare the degree of match between the pixel and the nearest neighboring Gaussian models; if the match is lower than this threshold, the pixel is considered to be part of the moving target. When there are large changes in lighting and other factors in the scene, some internal parameters may need to be adjusted appropriately to ensure that the system can continue to work effectively. For example, 226 pictures are used for background modeling, and it is required that the door in the captured video is closed from a visual effect and there are no people walking or objects placed in front of the door, causing the door to be blocked. When there are many changes in light in the scene, the internal parameter rank needs to be adjusted upward.

[0095] For example, a suitable threshold is set to distinguish the open and closed state of the door. After the background modeling process is completed, the position of the door is determined according to the image, that is, the coordinate values ​​of the upper left corner and the lower right corner of the door, and the corresponding parameters are modified according to the position of the door. The background modeling file obtained is used to test on the collected video. When the door is in a closed state, the threshold value is determined according to the value displayed on the detection image. For example, the calculated value is 0.05. The threshold can be selected around this value according to the needs. For example, if the requirements for the state of the door are relatively strict at this time, it is considered that the door is open if it is opened a little. The threshold can be set to 0.05. If the requirements for the state of the door are relatively loose, the threshold can be set to 0.16, that is, it is considered that the door is open only when the wall behind the door appears.

[0096] In order to cope with the changes in lighting conditions in different time periods (such as morning, afternoon and night), it is recommended to establish independent background models for each time period. Set up multiple background models in the system background, corresponding to different time windows, and automatically or manually switch to the most appropriate background model according to the current actual time. Perform background modeling rank selection. The rank parameter adjusts the number or weight of Gaussian components in GMM and other properties to adapt to environmental changes. When the light in the scene changes dramatically, you may need to increase this value to enhance the flexibility of the model. For general situations, you can try to use one of the values ​​​​{1,3,5,7,9,10} as the rank. If you find that the performance is still poor even at the maximum recommended value, you should consider adopting a multi-time modeling strategy instead of continuing to increase the rank value. A larger rank value means a more complex model structure. Although it can theoretically handle more diverse backgrounds, it also increases the computational complexity and the risk of overfitting.

[0097] For scenes with large changes in light, it is recommended to establish a model for three time periods, namely morning, afternoon, and night. The background can be managed in the background and switched as needed to achieve good detection results.

[0098] The rank value is proportional to the light change. It is recommended to select a value set of {1, 3, 5, 7, 9, 10}. If it exceeds 10, it is recommended to use background modeling based on time periods. Table 1 shows the background modeling rank selection suggestions:

[0099] Table 1

[0100]

[0101]

[0102] Open the parameters for selecting the region on the door. select_region indicates which part of the door is selected, and cropping_ratio indicates how many parts the door is divided into. For example, if select_region is 0, cropping_ratio is 3, it means the upper third of the door is selected. If select_region is changed to 2, it means the lower third of the door is selected. The configuration files are all background_configs.yaml. Table 2 shows the parameter settings for the selected region:

[0103] Table 2

[0104]

[0105] Step S3: Based on the video image, the conversation room object risk identification is performed using the risk identification threshold parameter of the conversation room object to obtain the conversation room object risk identification result.

[0106] The risk identification threshold parameter of the conversation room object is set to a value of (0,1), and the default value is set to 0.60. The greater the deviation of the threshold from the default value, the lower the sensitivity of risk identification; the smaller the absolute value deviation of the threshold from the default value, the higher the sensitivity of risk identification;

[0107] Risk identification of objects in the conversation room can be based on actual business needs in the business scenario, by setting the target objects as abnormal items and conducting targeted risk identification and analysis.

[0108] Step S4: Obtain and display the total risk identification result based on the conversation room environment risk identification result and the conversation room object risk identification result.

[0109] Interview room environment risk identification mainly focuses on whether there are abnormal conditions in the interview room, such as whether the door is in an unsafe state (not closed properly, damaged, etc.), whether there are unauthorized people entering, etc. Interview room object risk identification focuses on detecting whether there are suspicious items or other objects that may cause safety hazards in the room.

[0110] If the risk score of a specific project exceeds the predetermined security threshold, the event is considered to have a high potential threat, and the relevant information will be sent to the front-end interface for relevant personnel to review and process; on the contrary, if the risk level of an identified item is lower than the set limit, it will not be presented, that is, it will be intercepted and filtered. Ultimately, all the risk points that need attention after screening will be combined into a comprehensive report or view to be presented to the user as a basis for further action.

[0111] In summary, video streams are collected from conversation environments, and the videos are decomposed into image frames and audio data for separate processing. The background model is initialized using the Gaussian mixture model (GMM), and the background information is dynamically updated through the expectation maximization (EM) algorithm to adapt to environmental changes. The IP-GoDec method is combined to finely separate foreground objects.

[0112] By setting different thresholds to control the recognition sensitivity, the system can flexibly respond to the needs of different application scenarios, reduce dependence on large amounts of labeled data, adopt more efficient training strategies, and allow limited data sets to achieve higher recognition performance. The recognition results are presented to the user after being properly formatted, facilitating quick response decisions, and providing a visual interface or report format to intuitively display information such as abnormal situations and their locations. In this application, GMM+IP-GoDec is used to optimize the risk identification process in video surveillance, which not only improves work efficiency but also ensures a high safety standard.

[0113] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0114] First, the use of more advanced algorithms and technologies will enable the system to automatically identify potential security threats or inappropriate content in videos more quickly, reduce the need for human intervention, and improve the degree of automation and processing speed.

[0115] Second, traditional methods of training machine learning models require a large amount of labeled data sets as input, which is not only time-consuming but also expensive. This application significantly reduces the amount of data required while ensuring the effect, thereby reducing the overall development and maintenance costs.

[0116] Third, we use a structured process to analyze individual videos and content shot from different angles, assign a comprehensive risk score to each video, and evaluate whether a video has problems from multiple aspects, thereby improving the accuracy and robustness of the judgment.

[0117] Fourth, it allows users to adjust the importance or sensitivity of a particular indicator according to specific needs. For example, in some cases, you may want to pay more attention to certain types of security threats; in other cases, you may pay more attention to issues such as false alarm rates. In this way, the entire solution is more versatile and can better serve various regulatory requirements.

[0118] Fifth, it significantly improves the efficiency and accuracy of security reviews of conversation videos, reduces human resource consumption, and enhances the ability to deal with complex situations.

[0119] A risk identification device based on conversation video is provided below, and the device comprises:

[0120] An acquisition module, used for acquiring video data of the conversation room and decomposing the video data into video images and audio data;

[0121] The first risk identification module is used to select the ROI area based on the video image and combine it with the background modeling, and then use the IP-GoDec method to compare the pixel values ​​in the foreground and the grayscale image, and judge the state of the conversation room environment according to the risk identification threshold parameter of the conversation room environment to obtain the conversation room environment risk identification result;

[0122] The second risk identification module is used to perform risk identification of objects in the conversation room based on the video image and using the risk identification threshold parameter of the objects in the conversation room to obtain a risk identification result of the objects in the conversation room;

[0123] The display module is used to obtain and display the total risk identification result based on the conversation room environment risk identification result and the conversation room object risk identification result.

[0124] This preferred embodiment provides a computer device, which can implement the steps in any embodiment of the risk identification method provided in the embodiments of the present application. Therefore, the beneficial effects of the risk identification method provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0125] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments can be completed by instructions, or by controlling related hardware through instructions, and the instructions can be stored in a computer-readable storage medium and loaded and executed by a processor. To this end, an embodiment of the present application provides a storage medium in which a plurality of instructions are stored, and the instructions can be loaded by a processor to execute the steps of any embodiment of the risk identification method provided in the embodiment of the present application.

[0126] The storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0127] Since the instructions stored in the storage medium can execute the steps in any risk identification method embodiment provided in the embodiments of the present application, the beneficial effects that can be achieved by any risk identification method provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0128] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A risk identification method based on conversation video, characterized in that: The method comprises: Acquire video data of the conversation room and decompose the video data into video images and audio data; Based on the video image, the ROI area is selected and combined with the background modeling. Then the IP-GoDec method is used to compare the pixel values ​​in the foreground and grayscale images. The state of the conversation room environment is judged according to the risk identification threshold parameters of the conversation room environment, and the conversation room environment risk identification results are obtained. Based on the video image, the risk identification threshold parameter of the conversation room object is used to perform the conversation room object risk identification to obtain the conversation room object risk identification result; Based on the conversation room environment risk identification results and the conversation room object risk identification results, the overall risk identification results are obtained and displayed.

2. The risk identification method according to claim 1, characterized in that: Based on the video image, the ROI area is selected and combined with the background modeling, and then the IP-GoDec method is used to compare the pixel values ​​in the foreground and the grayscale image, and the state of the conversation room environment is judged according to the risk identification threshold parameter of the conversation room environment, and the steps of obtaining the conversation room environment risk identification result include: Determine the ROI image area of ​​the conversation room environment based on the video image; Use Gaussian mixture model to perform background modeling on the ROI image area to obtain a background model; Use IP-GoDec to perform foreground segmentation to obtain the foreground area, and perform difference analysis between the pixel value of the foreground area and the background grayscale image in the background model to obtain the grayscale difference; The grayscale difference is compared with the risk identification threshold parameter of the conversation room environment to determine the state of the conversation room environment.

3. The risk identification method according to claim 2, characterized in that: The step of determining the ROI image area of ​​the conversation room environment based on the video image includes: Performing an initial frame selection on the video image to obtain an initial frame selection area; The secondary selection area is obtained by performing area segmentation and polygon selection on the primary selection area; The secondary framed area is manually annotated to determine the ROI image area of ​​the conversation room environment.

4. The risk identification method according to claim 2, characterized in that: The steps of using the Gaussian mixture model to perform background modeling on the ROI image region to obtain the background model include: Select a frame in the video image as an initial frame; Initialize the Gaussian mixture model parameters of each component in the Gaussian mixture model using the color value of each pixel in the initial frame. The Gaussian mixture model parameters include mean, covariance matrix and weight; The EM algorithm is used to iteratively adjust the parameters of the Gaussian mixture model to obtain the background model.

5. The risk identification method according to claim 1, characterized in that: The steps of decomposing video data into video images and audio data include: Import the video data into the video processing software or development environment, and decode it through the decoding library to obtain image frames and audio samples; Convert the audio samples into a processable format, index the image frames in time order, and then extract the image frames at a time point or within a time range to obtain the video image; Align audio samples and video images using timestamps.

6. The risk identification method according to claim 5, characterized in that: After the step of aligning the audio samples and video images using timestamps, also include: Performing format conversion, scaling, and cropping on the video image to obtain the processed video image; The audio samples are denoised, data enhanced, and format converted to obtain processed audio samples.

7. The risk identification method according to claim 1, characterized in that: The video data acquisition of the conversation room is automatically accessed through the GB28181 standard, or the video stream address of the camera is manually obtained and configured into the system.

8. A risk identification device based on conversation video, characterized in that: The device comprises: An acquisition module, used for acquiring video data of the conversation room and decomposing the video data into video images and audio data; The first risk identification module is used to select the ROI area based on the video image and combine it with the background modeling, and then use the IP-GoDec method to compare the pixel values ​​in the foreground and the grayscale image, and judge the state of the conversation room environment according to the risk identification threshold parameter of the conversation room environment to obtain the conversation room environment risk identification result; The second risk identification module is used to perform risk identification of objects in the conversation room based on the video image and using the risk identification threshold parameter of the objects in the conversation room to obtain a risk identification result of the objects in the conversation room; The display module is used to obtain and display the total risk identification result based on the conversation room environment risk identification result and the conversation room object risk identification result.

9. A computer device, characterized in that: The computer device includes a processor and a memory, wherein a computer program is stored in the memory, and the computer program is loaded and executed by the processor to implement the risk identification method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, which is loaded and executed by a processor to implement the risk identification method according to any one of claims 1 to 7.

Images