Method, system and device for protecting vehicle-mounted video privacy information in complex environments

Abstract

The present invention discloses a method, system, and device for protecting in-vehicle video privacy information in a complex environment, and relates to the technical field of video data transmission. The method comprises: traversing files under a preset storage target, performing classified reading, and establishing classified reading results; establishing a preset grid division result, and performing grid aggregation prediction to establish a multi-level feature prediction result; performing model inference prediction, generating inference prediction results, and performing absolute coordinate conversion; performing inference prediction result screening, and establishing category labels and confidence information based on the screening results; performing privacy identification, establishing a dynamic fuzzy scheme, and performing backtracking result fuzzification processing, and performing identification and management based on the fuzzy processing results, category labels, and confidence information. The method solves the technical problems of low efficiency in protecting in-vehicle video privacy information and insufficient privacy identification accuracy in the prior art, and achieves the technical effect of efficiently and accurately identifying and protecting privacy information in a complex environment.

Description

Technical Field

[0001] The present invention relates to the technical field of video data transmission, and in particular to a method, system and device for protecting vehicle-mounted video privacy information in a complex environment. Background Art

[0002] With the rapid development of intelligent transportation and autonomous driving technologies, the use of in-vehicle cameras is becoming increasingly widespread, enabling real-time recording of surrounding traffic conditions and pedestrian behavior. However, these videos often contain a large amount of personal privacy information, such as pedestrian facial features and vehicle license plates. Therefore, necessary privacy protection measures must be implemented when processing this type of data to ensure that personally identifiable information is not leaked. Traditional video privacy protection methods face challenges such as poor accuracy, low efficiency, and poor real-time performance under complex environmental conditions such as road conditions, lighting, and weather, making them unable to effectively address the risk of privacy leaks. Summary of the Invention

[0003] This application provides a method, system and device for protecting in-vehicle video privacy information in a complex environment, which solves the technical problems of low efficiency in protecting in-vehicle video privacy information and insufficient privacy identification accuracy in the prior art.

[0004] In view of the above problems, this application provides a method, system and device for protecting in-vehicle video privacy information in complex environments.

[0005] The first aspect of the present application provides a method for protecting privacy information of in-vehicle videos in a complex environment, the method comprising:

[0006] After loading the model, traverse the files under the preset storage target, and perform classified reading according to the file extension to establish the classified reading results; perform preset grid division on the classified reading results to establish the preset grid division results, and perform grid aggregation prediction of the preset grid division results to establish a multi-level feature prediction result; use the infer function to asynchronously execute model inference prediction on the multi-level feature prediction results through the CUDA stream to generate inference prediction results, and perform absolute coordinate transformation of the inference prediction results to the original image; use the absolute coordinate transformation results to filter the inference prediction results based on non-maximum suppression, and establish category labels and confidence information based on the filtering results; perform privacy identification based on the category labels, establish a dynamic fuzzy scheme, backtrack grid aggregation based on the filtering results, and use the dynamic fuzzy scheme to perform fuzzy processing on the backtracking results, and perform identification management based on the fuzzy processing results, category labels and confidence information.

[0007] A second aspect of the present application provides a vehicle-mounted video privacy information protection system in a complex environment, the system comprising:

[0008] Classification reading module: After loading the model, traverse the files under the preset storage target, and perform classification reading according to the file extension to establish the classification reading results; aggregation prediction module: perform preset grid division on the classification reading results, establish the preset grid division results, and perform grid aggregation prediction of the preset grid division results to establish multi-level feature prediction results; inference prediction module: use the infer function to asynchronously execute model inference prediction on the multi-level feature prediction results through the CUDA stream to generate inference prediction results, and perform absolute coordinate transformation of the inference prediction results to the original image; screening module: use the absolute coordinate transformation results to screen the inference prediction results based on non-maximum suppression, and establish category labels and confidence information according to the screening results; identification management module: perform privacy identification based on the category labels, establish a dynamic fuzzy scheme, backtrack grid aggregation based on the screening results, and use the dynamic fuzzy scheme to perform fuzzy processing on the backtracking results, and perform identification management according to the fuzzy processing results, category labels and confidence information.

[0009] The third aspect of the present application provides an electronic device, including: a memory for storing executable instructions; and a processor for executing the executable instructions stored in the memory to implement the method for protecting in-vehicle video privacy information in a complex environment provided by the present application.

[0010] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0011] After loading the model, the system traverses the files in the preset storage destination and classifies them according to their file extensions, generating a classification read result. The classification read result is then subjected to a preset grid division to generate the preset grid division result. A grid aggregation prediction is then performed on the preset grid division result to generate a multi-level feature prediction result. Furthermore, the multi-level feature prediction result is asynchronously executed using the infer function via CUDA streams to perform model inference prediction, generating an inference prediction result. The inference prediction result is then transformed to the absolute coordinates of the original image. The absolute coordinate transformation result is then used to filter the inference prediction results using non-maximum suppression, and category labels and confidence information are generated based on the filtered results. Finally, privacy identification is performed based on the category labels, and a dynamic fuzzification scheme is established. Backtracking grid aggregation is performed based on the filtered results, and the backtracking results are fuzzified using the dynamic fuzzification scheme. Identification and management are performed based on the fuzzification results, category labels, and confidence information. This approach addresses the technical issues of low privacy information protection efficiency and insufficient privacy identification accuracy in in-vehicle videos in the prior art, achieving the technical effect of efficiently and accurately identifying and protecting privacy information in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0013] Figure 1 A flowchart of a method for protecting in-vehicle video privacy information in a complex environment provided by an embodiment of the present application;

[0014] Figure 2 A schematic diagram of the structure of a vehicle-mounted video privacy information protection system in a complex environment provided by an embodiment of the present application;

[0015] Figure 3 This is a schematic diagram of the structure of an exemplary electronic device of this application.

[0016] Explanation of the accompanying symbols: classification reading module 11, aggregation prediction module 12, reasoning prediction module 13, screening module 14, identification management module 15, processor 21, memory 22, input device 23, output device 24. DETAILED DESCRIPTION

[0017] This application solves the technical problems of low efficiency of in-vehicle video privacy information protection and insufficient privacy identification accuracy in the prior art by providing a method, system and device for protecting in-vehicle video privacy information in a complex environment.

[0018] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only some of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0019] It should be noted that the terms "including" and "having" are intended to cover non-exclusive inclusions. For example, a process, method, system, product or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or modules that are not clearly listed or are inherent to these processes, methods, products or devices.

[0020] Example 1, as Figure 1 As shown, this application provides a method for protecting privacy information of in-vehicle videos in complex environments, wherein the method includes:

[0021] After the model is loaded, the files under the preset storage target are traversed and classified and read according to the file extension to establish the classified reading results.

[0022] The method for protecting in-vehicle video privacy information in a complex environment provided by this application can process privacy information in the in-vehicle terminal and also transmit it back to the platform for processing.

[0023] After loading the YOLOv5 model, it traverses all files in the preset storage destination and categorizes them by file extension, allowing for different file types to be processed differently. For example, image files are directly read using OpenCV's imread function; video files are opened using OpenCV's VideoCapture class and read frame by frame. This process creates a categorized reading result, preparing data for subsequent operations such as feature extraction and object detection.

[0024] The classified reading result is divided into a preset grid to establish a preset grid division result, and grid aggregation prediction of the preset grid division result is performed to establish a multi-level feature prediction result.

[0025] The read image or video frame is divided into a preset grid size. Usually, each grid represents an area in the image. For example, the image can be divided into multiple small grids, each grid containing a part of the image area. Aggregate prediction is performed on each divided grid, that is, target detection, feature extraction and prediction results are aggregated for each grid; based on the results of grid aggregation, multi-level feature prediction results are established. Multi-level feature prediction means that the model gradually identifies detailed information in the image through multiple levels of feature extraction. Multi-level feature prediction helps capture target information of different scales, such as the detection of small and large objects, or target recognition under different environmental conditions. Through multi-level features, the model can more comprehensively analyze various targets in the image and improve detection accuracy.

[0026] Furthermore, before performing the preset grid division, the classification reading result is copied to the GPU, and after the GPU completes the preset grid division and obtains the inference prediction result, the inference prediction result is copied from the GPU back to the CPU memory.

[0027] Before performing the preset grid division, the classification readout results (e.g., images or video frames) are first copied from the CPU memory to the GPU memory. The GPU has stronger parallel computing capabilities and can process large amounts of image data. By migrating data to the GPU, the subsequent image processing and inference speed can be significantly improved. Once the data is copied to the GPU, the GPU will grid the data according to the preset grid size. Grid division on the GPU can leverage the GPU's parallel processing capabilities, allowing each grid in the image to be processed in parallel, significantly improving computational efficiency. The data in each grid area is processed on the GPU to prepare for subsequent prediction operations. After grid division is completed, the YOLOv5 model will perform inference prediction on the GPU, generating inference prediction results for each grid area. After the GPU completes inference prediction, the prediction results are copied from the GPU memory back to the CPU memory. By copying the classification readout results to the GPU and performing grid division and inference prediction on the GPU, the entire privacy protection process can be significantly accelerated.

[0028] Furthermore, the grid aggregation prediction of the preset grid division results is performed to establish a multi-level feature prediction result, including:

[0029] Each grid of the preset grid division result is used as the center, center aggregation is performed to establish a multi-level center aggregation result; untrusted shapes are eliminated from the multi-level center aggregation result to establish a multi-level feature prediction result.

[0030] Specifically, the center of each preset grid division result is used as a reference point, and a center aggregation operation is performed on each grid. The center refers to the center point position of each grid. Based on the center position of each grid, the surrounding features are aggregated to more accurately analyze the target in the image. Based on the center aggregation, a multi-level center aggregation result is established through a multi-level processing hierarchy. Untrusted shape elimination is performed on the multi-level center aggregation result. Untrusted shape elimination is performed by filtering out those prediction results with low confidence or shapes that do not meet expectations to improve detection accuracy and reduce false positives. After removing untrusted shapes, the aggregation results will be further processed to generate the final multi-level feature prediction results. These results will include the target category, location (bounding box coordinates) and corresponding confidence. Through center aggregation and multi-level feature processing, the accuracy and efficiency of target detection are enhanced. In particular, the true shape and location of the target can be effectively identified in complex environments, thereby providing more accurate recognition results for privacy protection and other subsequent processing.

[0031] The multi-level feature prediction results are asynchronously executed through the CUDA stream using the infer function to perform model inference prediction, generate inference prediction results, and perform absolute coordinate conversion of the inference prediction results to the original image.

[0032] The infer function uses CUDA streams asynchronously to infer multi-level feature prediction results. The infer function performs inference predictions based on the loaded YOLOv5 model. Because it uses CUDA streams asynchronously, the inference process does not block the main thread, allowing other operations to proceed in parallel, thereby accelerating overall processing speed. After asynchronous inference, the generated inference prediction results include information such as the target category (such as person, car, etc.), location (bounding box coordinates), and confidence level.

[0033] The inference prediction results (such as bounding box coordinates) are given based on the local coordinate system of the grid, but ultimately need to be converted to the absolute coordinate system of the original image in order to align with the original image. By inferring the relative coordinates in the prediction results (for example, relative bounding box coordinates), combined with operations such as image scaling and cropping, an absolute coordinate conversion is performed. The conversion process usually includes scaling the predicted bounding box positions to ensure that they correspond to the correct positions in the original image. Through the above steps, the YOLOv5 model can efficiently extract features from images and perform accurate target detection. At the same time, with the help of asynchronous computing and absolute coordinate conversion of CUDA streams, the efficiency and accuracy of the processing process are achieved, ensuring that the final privacy information protection or target detection results can be accurately displayed on the original image.

[0034] The absolute coordinate transformation results are used to screen the inference prediction results based on non-maximum suppression, and category labels and confidence information are established based on the screening results.

[0035] After obtaining the inference prediction results after absolute coordinate transformation, non-maximum suppression (NMS) is used to select the optimal detection box. The main function of NMS is to remove overlapping and low-confidence boxes when multiple bounding boxes are predicted, retaining the most representative and high-confidence bounding boxes. After NMS screening, the remaining bounding boxes are used to generate category labels and confidence information.

[0036] Furthermore, the use of the absolute coordinate transformation results to screen the inference prediction results based on non-maximum suppression includes:

[0037] Obtain the maximum category confidence and category index of the inference prediction result corresponding to the absolute coordinate conversion result; configure the category confidence threshold based on the maximum category confidence and category index, filter the inference prediction results using the category confidence threshold, and establish a retained result; establish an output format, convert the format of the retained result, and then perform non-maximum suppression screening using the set intersection-over-union threshold.

[0038] Specifically, after the absolute coordinate transformation, each prediction box will contain the confidence of multiple categories. The maximum category confidence of each prediction box is identified to indicate the confidence level of the model in the target category predicted by the box. The category index corresponding to the maximum category confidence is obtained. The category index is a numerical value indicating the target category corresponding to the prediction box. According to the maximum category confidence and the category index, a threshold is set to determine whether the prediction box is credible. The higher the confidence value, the more reliable the prediction result. By setting the confidence threshold, those prediction boxes with lower confidence can be removed, thereby improving the accuracy of the final result. Based on the category confidence threshold, those inference prediction boxes with confidence values ​​higher than the threshold are screened out. Test results; convert the format of the retained results after confidence screening, for example, organize the information of each detection box (category, confidence, coordinates, etc.) into a unified output format; for all retained results, use the set intersection-over-union (IoU) threshold to perform non-maximum suppression (NMS), where IoU is an indicator of the degree of overlap between two prediction boxes. When the IoU value of two boxes is higher than the set threshold, it means that they are detecting the same target and there may be repeated detections. In this case, NMS will retain the box with higher confidence and suppress other boxes. In this way, redundant detection results can be removed, ensuring that each target is recognized only once and maintaining high-confidence prediction results.

[0039] Furthermore, before the inference prediction result screening is performed by the category confidence threshold, the following steps are included:

[0040] The center coordinates and width and height information of the inference prediction result are obtained, and the frame coordinates are converted according to the center coordinates and the width and height information. The category confidence threshold is screened according to the inference prediction result after the frame coordinate conversion.

[0041] For each predicted box, first extract its center coordinates, which is the geometric center point of the box, usually calculated by the coordinates of the upper left and lower right corners of the box; secondly, extract the width and height of each predicted box; perform box coordinate conversion based on the obtained center coordinates and width and height information. The goal of box coordinate conversion is to convert the box from a relative coordinate system (such as a normalized coordinate system or a grid coordinate system) to the absolute coordinate system of the original image to ensure that the box can correctly reflect the position of the target in the original image. Specifically, convert the center coordinates into absolute coordinates according to the size of the image, and adjust the width and height information to the image size to obtain the accurate box boundary position.

[0042] After completing the box coordinate conversion, each predicted box is filtered based on its category confidence. Category confidence is the probability value corresponding to each category predicted by the model, indicating the confidence that the target belongs to a certain category. By setting a category confidence threshold, boxes with confidence levels above the threshold are filtered out, and boxes with lower confidence levels are removed. For example, if the category confidence threshold is set to 0.5, all boxes with confidence levels below 0.5 are filtered out, and only boxes with confidence levels above 0.5 are retained. After completing the box coordinate conversion and category confidence filtering, qualified boxes are retained and provide qualified prediction results for the subsequent non-maximum suppression (NMS) operation.

[0043] Privacy identification is performed based on the category annotation, a dynamic fuzzy scheme is established, grid aggregation is backtracked based on the screening results, and the backtracking results are fuzzy processed using the dynamic fuzzy scheme, and identification management is performed based on the fuzzy processing results, category annotations, and confidence information.

[0044] Each filtered prediction box is accompanied by a category label, indicating the type of object it identifies (such as a person, car, license plate, etc.); based on these category labels, it is possible to determine which objects are private information. For example, objects related to personal identity (such as faces, license plates, etc.) may require special attention and will be marked as private data that needs to be protected. Once an object is identified as private information, protection measures such as blurring are applied to ensure that this sensitive information is not visible or recognizable in the final result.

[0045] Dynamic blurring is a strategy that automatically adjusts the blur intensity based on the target's category, size, position, and environmental conditions (such as lighting and occlusion). Different privacy information may require different levels of blurring. For example, for facial recognition, stronger blurring may be required to ensure that the original face cannot be recovered from the blurred image. For license plates, only the letters or numbers may need to be blurred, preserving the presence of the license plate but making it unreadable.

[0046] When identifying and obfuscating private information, grid aggregation can backtrack and reconstruct information from various grid regions. Backtracking grid aggregation ensures the consistency and integrity of the relevant target regions during obfuscation. For example, if the boundary of a target spans multiple grids, backtracking aggregation helps ensure the continuity and integrity of the target across the grids.

[0047] Using a dynamic blurring scheme, the actual blurring process is performed based on the back-aggregated target information. This can include Gaussian blurring, mosaic blurring, pixelation, and other methods, depending on the privacy protection requirements and target characteristics. Ultimately, a blurred image is output, demonstrating the privacy-preserving result, where sensitive information (such as faces and license plates) is blurred while non-private information remains visible.

[0048] Furthermore, the privacy identification based on the category labeling and the establishment of a dynamic fuzzy solution include:

[0049] Create a basic privacy level based on the privacy identification results; obtain the identified application scenario, match the fuzzy scheme library based on the application scenario and the basic privacy level, and generate a scheme matching result; perform frame-by-frame background analysis on the backtracking results to establish a background matching result; and establish a dynamic fuzzy scheme based on the scheme matching result and the background matching result.

[0050] Specifically, based on the identified privacy targets, a basic privacy level is created to represent the sensitivity of privacy information. For example, the low privacy level is some targets that are less related to personal privacy (for example, non-facial background characters, public objects, etc.); the medium privacy level involves targets with less sensitive information (for example, incomplete and recognizable facial or license plate information); and the high privacy level involves targets with highly sensitive information (for example, clear faces, complete license plates, etc.).

[0051] Obtain the application scenario of the current image or video. The application scenario can be different environments such as static images, dynamic videos, surveillance videos, traffic monitoring, and home videos. In traffic monitoring, license plates are private information, while faces may not be so sensitive. In home monitoring, facial information may be more important, while license plates may be less important. Based on the current application scenario and basic privacy level, select the corresponding blurring scheme from the pre-defined blurring scheme library. The blurring scheme library contains blurring strategies for different situations, which may include strong blurring schemes (suitable for high-privacy level targets, such as sensitive information such as faces and license plates) and light blurring schemes (suitable for medium or low-privacy level targets, such as blurred backgrounds and low-confidence targets). Based on the matching of application scenarios and privacy levels, generate the final scheme matching results, which will determine which private information needs to be strongly blurred, which needs to be lightly blurred, and which does not need to be blurred.

[0052] When processing dynamic video, the impact of background information on privacy protection cannot be ignored. After retracing the grid aggregation results, the background of each frame is analyzed to determine which background components may affect the recognition of private information. The background may contain additional information related to private information, such as inadvertently displayed facial features or license plates. When analyzing the background of each frame, it distinguishes which parts of the dynamic scene must be blurred and which do not. The background analysis results in a background match, indicating which background areas require blurring.

[0053] Ultimately, a dynamic blurring scheme is generated based on the combined results of the scheme matching and background matching. This process combines the type of privacy target, privacy level, application scenario, and background characteristics in dynamic scenes to determine how to handle the private information in each frame of the image or video. For example, for targets with a high privacy level, if there is no private content related to the target in the background, a lighter blurring may be applied; however, if the background also involves private information, the blurring strength may need to be increased. For targets with a low privacy level, even if there is relevant information in the background, only a light blurring may be used.

[0054] Furthermore, the identification management based on the fuzzy processing results, category labels and confidence information also includes:

[0055] Call the preset function, draw the prediction box on the original file through OpenCV, and partially replace the original file according to the blur processing result. Then, identify the original file after partial replacement based on the category label and confidence information, and store it in the preset directory.

[0056] On the processed image (whether it is a still picture or a video frame), the OpenCV library is used to draw the predicted box, which corresponds to the target previously identified by a model such as YOLOv5. For each detected target, OpenCV will mark a rectangular box on the image and indicate its category label and confidence value.

[0057] Based on the blurring results, privacy-sensitive areas in the image are partially replaced to ensure that the private information in these areas cannot be identified. Specific methods of partial replacement may include blurring and occlusion. Blurring: Using Gaussian blur, mosaic, or other blurring techniques, target areas containing private information (such as faces and license plates) are blurred and replaced with unrecognizable areas. Occlusion: For certain sensitive targets, occlusion techniques may be used, such as using black bars or mosaic blocks to cover the target area to ensure that private information is hidden.

[0058] OpenCV can achieve privacy protection by manipulating the pixel values ​​of an image to replace the area to be blurred with a corresponding blurred pattern. Specific replacement operations include: modifying the pixel color of the target area, applying a blur function, or replacing it with an occlusion image.

[0059] After partial replacement, the image is processed and its identification is updated. The image identification can include information about the privacy protection status of the image, such as a category identification indicating the type of privacy object detected in the image (such as a face, license plate, etc.) and its protection level (such as light blur or heavy blur); and a confidence identification indicating the confidence value of each object. The processed image file is stored according to the updated identification, ensuring effective protection of the image information and providing traceability. The processed image (i.e., partial replacement and identification update) is saved to the preset storage directory.

[0060] Furthermore, after storing in the preset directory, the preset destruction function is called to release the CUDA stream, GPU buffer and CPU memory resources, and destroy the engine, execution context and CUDA runtime instance.

[0061] After image processing is completed and the results are stored in the preset directory, the system will call the preset destruction function to release all occupied computing and storage resources. CUDA streams are processing units used to perform asynchronous operations and are used to manage the scheduling and execution of tasks during GPU computing. Each asynchronous operation (such as inference prediction, data transfer, etc.) is assigned a stream for execution, which needs to be released after the stream is executed. When the destruction function is called, the system will release all resources related to the CUDA stream to ensure that the GPU memory is no longer occupied.

[0062] After completing image processing and file storage, the system will call the preset destruction function to release the occupied computing and storage resources to ensure that memory and computing resources are managed efficiently. First, the system will release resources related to the CUDA stream, and destroy the stream to avoid GPU memory occupation. Next, the system will release all data in the GPU buffer, including input images and prediction results, to ensure that GPU memory is reclaimed. Then, the system will release all resources allocated in the CPU memory, clear image data and other computing data, and avoid memory overflow or performance degradation. At the same time, the system will destroy the inference engine and execution context, and release the memory and computing resources related to the execution of the inference task. Finally, the CUDA runtime instance will also be destroyed, and all resources related to GPU computing will be cleaned up by calling cudaDeviceReset to ensure that the system is in a clean state when the next computing task starts.

[0063] In summary, the embodiments of the present application have at least the following technical effects:

[0064] After loading the model, the system traverses the files in the preset storage destination and classifies them according to their file extensions, generating a classification read result. The classification read result is then subjected to a preset grid division to generate the preset grid division result. A grid aggregation prediction is then performed on the preset grid division result to generate a multi-level feature prediction result. Furthermore, the multi-level feature prediction result is asynchronously executed using the infer function via CUDA streams to perform model inference prediction, generating an inference prediction result. The inference prediction result is then transformed to the absolute coordinates of the original image. The absolute coordinate transformation result is then used to filter the inference prediction results using non-maximum suppression, and category labels and confidence information are generated based on the filtered results. Finally, privacy identification is performed based on the category labels, and a dynamic fuzzification scheme is established. Backtracking grid aggregation is performed based on the filtered results, and the backtracking results are fuzzified using the dynamic fuzzification scheme. Identification and management are performed based on the fuzzification results, category labels, and confidence information. This approach addresses the technical issues of low privacy information protection efficiency and insufficient privacy identification accuracy in in-vehicle videos in the prior art, achieving the technical effect of efficiently and accurately identifying and protecting privacy information in complex environments.

[0065] Embodiment 2 is based on the same inventive concept as the method for protecting privacy information of in-vehicle video in a complex environment in the above embodiment. Figure 2 As shown, the present application provides a vehicle-mounted video privacy information protection system in a complex environment, wherein the system includes:

[0066] Classification reading module 11: After loading the model, traverse the files under the preset storage target, and perform classification reading according to the file extension to establish the classification reading result; aggregation prediction module 12: perform preset grid division on the classification reading result, establish the preset grid division result, and perform grid aggregation prediction of the preset grid division result to establish a multi-level feature prediction result; inference prediction module 13: use the infer function to asynchronously execute model inference prediction on the multi-level feature prediction result through the CUDA stream to generate an inference prediction result, and perform the absolute coordinate transformation of the inference prediction result to the original image; screening module 14: use the absolute coordinate transformation result to filter the inference prediction result based on non-maximum suppression, and establish category labeling and confidence information according to the screening result; identification management module 15: perform privacy identification based on the category labeling, establish a dynamic fuzzy scheme, backtrack grid aggregation based on the screening result, and use the dynamic fuzzy scheme to perform fuzzy processing on the backtracking result, and perform identification management according to the fuzzy processing result, category labeling and confidence information.

[0067] Furthermore, the screening module 14 is configured to perform the following method:

[0068] Obtain the maximum category confidence and category index of the inference prediction result corresponding to the absolute coordinate conversion result; configure the category confidence threshold based on the maximum category confidence and category index, filter the inference prediction results using the category confidence threshold, and establish a retained result; establish an output format, convert the format of the retained result, and then perform non-maximum suppression screening using the set intersection-over-union threshold.

[0069] Furthermore, the screening module 14 is configured to perform the following method:

[0070] The center coordinates and width and height information of the inference prediction result are obtained, and the frame coordinates are converted according to the center coordinates and the width and height information. The category confidence threshold is screened according to the inference prediction result after the frame coordinate conversion.

[0071] Furthermore, the identification management module 15 is configured to execute the following method:

[0072] Call the preset function, draw the prediction box on the original file through OpenCV, and partially replace the original file according to the blur processing result. Then, identify the original file after partial replacement based on the category label and confidence information, and store it in the preset directory.

[0073] Furthermore, the identification management module 15 is configured to execute the following method:

[0074] After storing in the preset directory, the preset destruction function is called to release the CUDA stream, GPU buffer and CPU memory resources, and destroy the engine, execution context and CUDA runtime instance.

[0075] Furthermore, the aggregation prediction module 12 is configured to perform the following method:

[0076] Before performing the preset grid division, the classification reading results are copied to the GPU. After the GPU completes the preset grid division and obtains the inference prediction results, the inference prediction results are copied from the GPU back to the CPU memory.

[0077] Furthermore, the identification management module 15 is configured to execute the following method:

[0078] Create a basic privacy level based on the privacy identification results; obtain the identified application scenario, match the fuzzy scheme library based on the application scenario and the basic privacy level, and generate a scheme matching result; perform frame-by-frame background analysis on the backtracking results to establish a background matching result; and establish a dynamic fuzzy scheme based on the scheme matching result and the background matching result.

[0079] Furthermore, the aggregation prediction module 12 is configured to perform the following method:

[0080] Each grid of the preset grid division result is used as the center, center aggregation is performed to establish a multi-level center aggregation result; untrusted shapes are eliminated from the multi-level center aggregation result to establish a multi-level feature prediction result.

[0081] Example 3, Figure 3 This is a structural diagram of an electronic device provided in accordance with a third embodiment of the present invention, showing a block diagram of an exemplary electronic device suitable for implementing the embodiments of the present invention. Figure 3 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present invention. Figure 3 As shown, the electronic device includes a processor 21, a memory 22, an input device 23 and an output device 24; the number of processors 21 in the electronic device can be one or more. Figure 3 Taking a processor 21 as an example, the processor 21, memory 22, input device 23 and output device 24 in the electronic device can be connected through a bus or other means. Figure 3 The bus connection is taken as an example.

[0082] It should be noted that the order in which the embodiments of the present application are presented is for illustrative purposes only and does not necessarily represent the superiority or inferiority of the embodiments. Furthermore, the foregoing descriptions of specific embodiments of this specification are provided. The processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0083] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

[0084] This specification and drawings are merely illustrative of the present application and are intended to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Obviously, those skilled in the art may make various modifications and variations to this application without departing from the scope of this application. Thus, this application is intended to include such modifications and variations as fall within the scope of this application and its equivalents.

Claims

1. A method for protecting privacy information of in-vehicle videos in complex environments, characterized by: The method comprises: After loading the model, traverse the files under the preset storage target, and read them by category according to the file extension, and establish the classification reading results; Performing preset grid division on the classification reading result to establish a preset grid division result, and performing grid aggregation prediction on the preset grid division result to establish a multi-level feature prediction result; The multi-level feature prediction results are asynchronously executed through the CUDA stream using the infer function to perform model inference prediction, generate inference prediction results, and perform absolute coordinate conversion of the inference prediction results to the original image; The absolute coordinate transformation results are used to screen the inference prediction results based on non-maximum suppression, and the category labeling and confidence information are established based on the screening results; Based on the category labeling, privacy identification is performed, a dynamic fuzzy scheme is established, grid aggregation is backtracked based on the screening results, and the backtracking results are fuzzified using the dynamic fuzzy scheme. Identification management is performed based on the fuzzy processing results, category labels, and confidence information. The backtracking grid aggregation can backtrack and reorganize the information of each grid area. By backtracking the grid aggregation, the consistency and integrity of the relevant target areas are ensured during the fuzzy processing. Among them, establishing a dynamic blur solution includes: Create a basic privacy level based on the privacy identification results; Obtaining an identified application scenario, and generating a solution matching result based on a fuzzy solution library matching the application scenario and the basic privacy level; Perform frame-by-frame background analysis on the backtracking results to establish background matching results; A dynamic blurring scheme is established according to the scheme matching result and the background matching result.

2. The method for protecting privacy information of in-vehicle video in a complex environment according to claim 1, characterized in that: The method of using the absolute coordinate transformation result to screen the inference prediction results based on non-maximum suppression includes: Get the maximum category confidence and category index of the inference prediction result corresponding to the absolute coordinate transformation result; Configuring a category confidence threshold based on the maximum category confidence and the category index, screening the inference prediction results using the category confidence threshold, and establishing a retained result; An output format is established, and after the retained results are format-converted, non-maximum suppression screening is performed using a set intersection-over-union ratio threshold.

3. The method for protecting privacy information of in-vehicle video in a complex environment as claimed in claim 2, characterized in that: The screening of the inference prediction results by the category confidence threshold includes: The center coordinates and width and height information of the inference prediction result are obtained, and the frame coordinates are converted according to the center coordinates and the width and height information. The category confidence threshold is screened according to the inference prediction result after the frame coordinate conversion.

4. The method for protecting privacy information of in-vehicle video in a complex environment as claimed in claim 1, characterized in that: The identification management based on the fuzzy processing results, category labels and confidence information also includes: Call the preset function, draw the prediction box on the original file through OpenCV, and partially replace the original file according to the blur processing result. Then, identify the original file after partial replacement based on the category label and confidence information, and store it in the preset directory.

5. The method for protecting privacy information of in-vehicle video in a complex environment as claimed in claim 4, characterized in that: After storing in the preset directory, the preset destruction function is called to release the CUDA stream, GPU buffer and CPU memory resources, and destroy the engine, execution context and CUDA runtime instance.

6. The method for protecting privacy information of in-vehicle video in a complex environment as claimed in claim 1, characterized in that: Before performing the preset grid division, the classification reading results are copied to the GPU. After the GPU completes the preset grid division and obtains the inference prediction results, the inference prediction results are copied from the GPU back to the CPU memory.

7. The method for protecting privacy information of in-vehicle video in a complex environment as claimed in claim 1, characterized in that: The method further comprises: performing grid aggregation prediction of preset grid division results to establish multi-level feature prediction results, including: Taking each grid of the preset grid division result as the center, performing center aggregation to establish a multi-level center aggregation result; Untrusted shapes are eliminated from the multi-level center aggregation results to establish a multi-level feature prediction result.

8. The vehicle-mounted video privacy information protection system in complex environments is characterized by: The system is used to implement the method for protecting privacy information of in-vehicle videos in a complex environment as described in any one of claims 1 to 7, comprising: Classification reading module: After loading the model, it traverses the files under the preset storage target, and performs classification reading according to the file extension to establish the classification reading results; Aggregation prediction module: performs preset grid division on the classification reading result, establishes preset grid division result, and performs grid aggregation prediction of the preset grid division result to establish multi-level feature prediction result; Inference prediction module: uses the infer function to perform model inference prediction on the multi-level feature prediction results through CUDA stream asynchronous execution to generate inference prediction results, and performs absolute coordinate conversion of the inference prediction results to the original image; Screening module: Use the absolute coordinate transformation results to screen the inference prediction results based on non-maximum suppression, and establish category labels and confidence information based on the screening results; Identification management module: performs privacy identification based on the category labeling, establishes a dynamic fuzzy scheme, backtracks grid aggregation based on the screening results, and uses the dynamic fuzzy scheme to fuzzify the backtracking results, and performs identification management based on the fuzzy processing results, category labeling and confidence information.

9. An electronic device, characterized in that: The electronic device comprises: a memory for storing executable instructions; The processor is configured to implement the method for protecting privacy information of in-vehicle videos in a complex environment according to any one of claims 1 to 7 when executing the executable instructions stored in the memory.

Images