Vehicle-mounted video privacy information protection method, system and equipment in complex environment

Abstract

The invention discloses a vehicle-mounted video privacy information protection method, system and device in a complex environment, and relates to the technical field of video data transmission. The method comprises the steps of traversing files under a preset storage target, performing classified reading, and establishing a classified reading result; establishing a preset grid division result, executing grid aggregation prediction, and establishing a multi-level feature prediction result; model reasoning prediction is executed, a reasoning prediction result is generated, and absolute coordinate conversion is executed; reasoning prediction results are screened, and category labels and confidence degree information are established according to the screening results; and carrying out privacy identification, establishing a dynamic fuzzy scheme, carrying out backtracking result fuzzy processing, and carrying out identification management according to a fuzzy processing result, category labels and confidence degree information. The technical problems that in the prior art, vehicle-mounted video privacy information protection efficiency is low, and privacy recognition accuracy is insufficient are solved, and the technical effect of efficiently and accurately recognizing and protecting privacy information in a complex environment is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of video data transmission, and in particular to a method, system and device for protecting vehicle-mounted video privacy information in a complex environment. Background Art

[0002] With the rapid development of intelligent transportation and autonomous driving technologies, vehicle-mounted cameras are increasingly being used to record surrounding traffic conditions and pedestrian behavior in real time. However, these videos often contain a large amount of personal privacy information, such as facial features of pedestrians and license plate numbers of vehicles. Therefore, necessary privacy protection measures must be taken when processing such data to ensure that personal identity information is not leaked. Under complex environmental conditions such as road conditions, lighting, and weather, traditional video privacy protection methods face problems such as poor accuracy, low efficiency, and poor real-time performance, and cannot effectively deal with the risk of privacy leakage. Summary of the invention

[0003] The present application provides a method, system and device for protecting the privacy information of in-vehicle videos in a complex environment, which solves the technical problems of low efficiency in protecting the privacy information of in-vehicle videos and insufficient accuracy in privacy identification in the prior art.

[0004] In view of the above problems, the present application provides a method, system and device for protecting vehicle-mounted video privacy information in a complex environment.

[0005] The first aspect of the present application provides a method for protecting privacy information of in-vehicle videos in a complex environment, the method comprising:

[0006] After loading the model, traverse the files under the preset storage target, and perform classified reading according to the file extension to establish the classified reading result; perform preset grid division on the classified reading result to establish the preset grid division result, and perform grid aggregation prediction of the preset grid division result to establish a multi-level feature prediction result; use the infer function to asynchronously perform model inference prediction on the multi-level feature prediction result through the CUDA stream to generate an inference prediction result, and perform an absolute coordinate transformation of the inference prediction result to the original image; use the absolute coordinate transformation result to filter the inference prediction result based on non-maximum suppression, and establish category annotation and confidence information according to the filtering result; perform privacy identification based on the category annotation, establish a dynamic fuzzy scheme, backtrack grid aggregation based on the filtering result, and use the dynamic fuzzy scheme to fuzzy the backtracked result, and perform identification management according to the fuzzy processing result, category annotation and confidence information.

[0007] The second aspect of the present application provides a vehicle-mounted video privacy information protection system in a complex environment, the system comprising:

[0008] Classification reading module: after loading the model, traverse the files under the preset storage target, and perform classification reading according to the file extension to establish the classification reading result; aggregation prediction module: perform preset grid division on the classification reading result, establish the preset grid division result, and perform grid aggregation prediction of the preset grid division result to establish a multi-level feature prediction result; reasoning prediction module: use the infer function to asynchronously execute model reasoning prediction on the multi-level feature prediction result through the CUDA stream to generate the reasoning prediction result, and perform the absolute coordinate transformation of the reasoning prediction result to the original image; screening module: use the absolute coordinate transformation result to screen the reasoning prediction result based on non-maximum suppression, and establish category labeling and confidence information according to the screening result; identification management module: perform privacy identification based on the category labeling, establish a dynamic fuzzy scheme, backtrack grid aggregation based on the screening result, and use the dynamic fuzzy scheme to perform fuzzy processing on the backtracking result, and perform identification management according to the fuzzy processing result, category labeling and confidence information.

[0009] The third aspect of the present application provides an electronic device, including: a memory for storing executable instructions; a processor for executing the executable instructions stored in the memory to implement the method for protecting vehicle-mounted video privacy information in a complex environment provided by the present application.

[0010] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0011] After loading the model, the files under the preset storage target are traversed, and classified reading is performed according to the file extension to establish the classified reading result. Then, the classified reading result is divided into preset grids, the preset grid division result is established, and the grid aggregation prediction of the preset grid division result is performed to establish the multi-level feature prediction result. Further, the multi-level feature prediction result is asynchronously executed through the CUDA stream using the infer function to perform model inference prediction, generate the inference prediction result, and perform the absolute coordinate transformation of the original image on the inference prediction result. Then, the inference prediction result based on non-maximum suppression is screened using the absolute coordinate transformation result, and the category annotation and confidence information are established according to the screening result. Finally, privacy identification is performed based on the category annotation, a dynamic fuzzy scheme is established, the grid aggregation is traced back based on the screening result, and the fuzzy processing of the traced result is performed using the dynamic fuzzy scheme, and the identification management is performed according to the fuzzy processing result, the category annotation and the confidence information. The technical problems of low efficiency of privacy information protection and insufficient accuracy of privacy identification in vehicle-mounted videos in the prior art are solved, and the technical effect of efficiently and accurately identifying and protecting privacy information in a complex environment is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0013] Figure 1 A flow chart of a method for protecting privacy information of in-vehicle video in a complex environment provided by an embodiment of the present application;

[0014] Figure 2 A schematic diagram of the structure of a vehicle-mounted video privacy information protection system in a complex environment provided by an embodiment of the present application;

[0015] Figure 3 This is a schematic diagram of the structure of an exemplary electronic device of the present application.

[0016] Explanation of the reference numerals: classification reading module 11, aggregation prediction module 12, reasoning prediction module 13, screening module 14, identification management module 15, processor 21, memory 22, input device 23, output device 24. DETAILED DESCRIPTION

[0017] The present application solves the technical problems of low efficiency of in-vehicle video privacy information protection and insufficient privacy identification accuracy in the prior art by providing a method, system and device for protecting in-vehicle video privacy information in a complex environment.

[0018] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0019] It should be noted that the terms "including" and "having" are intended to cover non-exclusive inclusions. For example, a process, method, system, product or server that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or modules that are not explicitly listed or inherent to these processes, methods, products or devices.

[0020] Embodiment 1, as Figure 1 As shown, the present application provides a method for protecting privacy information of in-vehicle videos in a complex environment, wherein the method includes:

[0021] After the model is loaded, the files under the preset storage target are traversed, and classified reading is performed according to the file extension to establish the classified reading results.

[0022] The method for protecting in-vehicle video privacy information in a complex environment provided by this application can process privacy information in the in-vehicle terminal and can also be transmitted back to the platform for processing.

[0023] After the YOLOv5 model is loaded, all files under the preset storage target are traversed, and the files are classified and read according to the file extensions, so that different file types can be processed differently. For example, image files are directly read using the imread function of OpenCV; video files use the VideoCapture class of OpenCV to open the video stream and read frame by frame. Through this process, the classification reading results can be established to prepare data for subsequent feature extraction, target detection and other operations.

[0024] The classified reading result is divided into preset grids to establish a preset grid division result, and grid aggregation prediction of the preset grid division result is performed to establish a multi-level feature prediction result.

[0025] The read image or video frame is divided into a preset grid size. Usually, each grid represents an area in the image. For example, the image can be divided into multiple small grids, each of which contains a part of the image. Aggregate prediction is performed on each divided grid, that is, target detection, feature extraction and prediction results are aggregated for each grid; multi-level feature prediction results are established based on the results of grid aggregation. Multi-level feature prediction refers to the model gradually identifying detailed information in the image through multiple levels of feature extraction. Multi-level feature prediction helps capture target information of different scales, such as the detection of small and large objects, or target recognition under different environmental conditions. Through multi-level features, the model can more comprehensively analyze various targets in the image and improve detection accuracy.

[0026] Furthermore, before performing the preset grid division, the classification reading result is copied to the GPU, and after the GPU completes the preset grid division and obtains the inference prediction result, the inference prediction result is copied from the GPU back to the CPU memory.

[0027] Before performing the preset grid division, the classification read results (for example, images or video frames) are first copied from the CPU memory to the GPU memory. The GPU has stronger parallel computing capabilities and can process a large amount of image data. By migrating the data to the GPU, the subsequent image processing and reasoning speed can be significantly improved. Once the data is copied to the GPU, the GPU will grid the data according to the preset grid size. Grid division on the GPU can take advantage of the parallel processing capabilities of the GPU, so that each grid of the image can be processed in parallel, thereby greatly improving the computing efficiency. The data in each grid area will be processed on the GPU to prepare for subsequent prediction operations. After the grid division is completed, the YOLOv5 model will perform inference prediction on the GPU to generate inference prediction results for each grid area; after the GPU completes the inference prediction, the prediction results are copied from the GPU memory back to the CPU memory. By copying the classification read results to the GPU and completing grid division and inference prediction on the GPU, the entire privacy protection process can be significantly accelerated.

[0028] Furthermore, the grid aggregation prediction of the preset grid division results is performed to establish a multi-level feature prediction result, including:

[0029] Each grid of the preset grid division result is taken as the center, center aggregation is performed, and a multi-level center aggregation result is established; untrusted shapes are eliminated on the multi-level center aggregation result, and a multi-level feature prediction result is established.

[0030] Specifically, the center of each preset grid division result is used as a reference point, and a center aggregation operation is performed on each grid, where the center refers to the center point position of each grid. Based on the center position of each grid, the surrounding features are aggregated to more accurately analyze the target in the image; on the basis of center aggregation, a multi-level center aggregation result is established through a multi-level processing hierarchy; the multi-level center aggregation result is subjected to untrusted shape elimination, which is to filter out those prediction results with low confidence or shapes that do not meet expectations, so as to improve detection accuracy and reduce false positives; the aggregation results after removing untrusted shapes will be further processed and generate the final multi-level feature prediction results, which will include the target category, location (bounding box coordinates) and corresponding confidence. Through center aggregation and multi-level feature processing, the accuracy and efficiency of target detection are enhanced, especially in complex environments, the true shape and location of the target can be effectively identified, thereby providing more accurate recognition results for privacy protection and other subsequent processing.

[0031] The multi-level feature prediction results are asynchronously executed through the CUDA stream using the infer function to perform model inference prediction, generate inference prediction results, and perform absolute coordinate conversion of the inference prediction results to the original image.

[0032] The infer function combines CUDA stream asynchronously to infer the multi-level feature prediction results. The infer function will perform inference prediction based on the loaded YOLOv5 model, and because CUDA stream asynchronous execution is used, the inference process will not block the main thread, allowing other operations to be performed in parallel, thereby speeding up the overall processing speed. After asynchronous inference, the generated inference prediction results contain information such as the target category (such as people, cars, etc.), location (coordinates of the bounding box), and confidence.

[0033] The inference prediction results (such as bounding box coordinates) are given based on the local coordinate system of the grid, but ultimately need to be converted to the absolute coordinate system of the original image in order to align with the original image. By inferring the relative coordinates in the prediction results (for example, relative bounding box coordinates), combined with image scaling, cropping and other operations, absolute coordinate conversion is performed. The conversion process usually includes scaling the predicted bounding box positions to ensure that they correspond to the correct positions in the original image. Through the above steps, the YOLOv5 model can efficiently extract features from images and perform accurate target detection. At the same time, with the help of asynchronous computing and absolute coordinate conversion of CUDA streams, the efficiency and accuracy of the processing process are achieved, ensuring that the final privacy information protection or target detection results can be accurately displayed on the original image.

[0034] The absolute coordinate transformation results are used to screen the inference prediction results based on non-maximum suppression, and category annotations and confidence information are established according to the screening results.

[0035] After obtaining the inference prediction results after absolute coordinate transformation, it is necessary to filter out the best detection box through non-maximum suppression (NMS). The main function of NMS is to remove those boxes with overlapping positions and low confidence when multiple bounding boxes are predicted, and retain the most representative and high-confidence bounding boxes. After NMS screening, the bounding boxes that are finally retained will be used to generate category annotations and confidence information.

[0036] Furthermore, the use of the absolute coordinate transformation result to screen the inference prediction result based on non-maximum suppression includes:

[0037] The maximum category confidence and category index of the inference prediction result corresponding to the absolute coordinate conversion result are obtained; the category confidence threshold is configured based on the maximum category confidence and the category index, and the inference prediction result is screened by the category confidence threshold to establish a retained result; an output format is established, and after the retained result is format-converted, non-maximum suppression screening is performed by the set intersection-over-union ratio threshold.

[0038] Specifically, after the absolute coordinate transformation, each prediction box will contain the confidence of multiple categories. The maximum category confidence of each prediction box is identified, indicating the confidence of the model in the target category predicted by the box. The category index corresponding to the maximum category confidence is obtained. The category index is a numerical value indicating the target category corresponding to the prediction box. According to the maximum category confidence and the category index, a threshold is set to determine whether the prediction box is credible. The higher the confidence value, the more reliable the prediction result. By setting the confidence threshold, those prediction boxes with low confidence can be removed, thereby improving the accuracy of the final result. Based on the category confidence threshold, those inference prediction boxes with confidence values ​​higher than the threshold are screened out. The method is to convert the format of the retained results after confidence screening, for example, organize the information of each detection box (category, confidence, coordinates, etc.) into a unified output format; for all retained results, use the set intersection over union (IoU) threshold to perform non-maximum suppression (NMS), where IoU is an indicator of the degree of overlap between two prediction boxes. When the IoU value of two boxes is higher than the set threshold, it means that they are detecting the same target and there may be repeated detections. In this case, NMS will retain the box with higher confidence and suppress other boxes. In this way, redundant detection results can be removed, ensuring that each target is recognized only once and maintaining high-confidence prediction results.

[0039] Furthermore, before the inference prediction result screening is performed by the category confidence threshold, the following steps are included:

[0040] The center coordinates and width and height information are obtained for the inference prediction result, and the frame coordinates are converted according to the center coordinates and the width and height information. The category confidence threshold is screened according to the inference prediction result after the frame coordinate conversion.

[0041] For each predicted box, first extract its center coordinates, which is the geometric center point of the box, usually calculated by the coordinates of the upper left and lower right corners of the box; secondly, extract the width and height of each predicted box; perform box coordinate transformation through the obtained center coordinates and width and height information. The goal of box coordinate transformation is to transform the box from a relative coordinate system (such as a normalized coordinate system or a grid coordinate system) to the absolute coordinate system of the original image to ensure that the box can correctly reflect the position of the target in the original image. Specifically, convert the center coordinates into absolute coordinates according to the size of the image, and adjust the width and height information to the image size in ratio to obtain the accurate box boundary position.

[0042] After completing the box coordinate conversion, the prediction box is filtered according to the category confidence of each prediction box. The category confidence is the probability value corresponding to each category predicted by the model, indicating the confidence that the target belongs to a certain category. By setting a category confidence threshold, the boxes with confidence higher than the threshold are filtered out, and the boxes with lower confidence are removed. For example, if the category confidence threshold is set to 0.5, all boxes with confidence lower than 0.5 will be filtered out, and only boxes with confidence higher than 0.5 will be retained. After completing the box coordinate conversion and category confidence screening, the qualified boxes will be retained and provide qualified prediction results for the subsequent non-maximum suppression (NMS) operation.

[0043] Based on the category annotation, privacy identification is performed, a dynamic fuzzy scheme is established, grid aggregation is traced back based on the screening results, and the dynamic fuzzy scheme is used to fuzzify the traced results, and identification management is performed based on the fuzzy processing results, category annotations, and confidence information.

[0044] Each filtered prediction box has been accompanied by a category annotation, indicating the type of target identified by the box (such as people, cars, license plates, etc.); based on these category annotations, it can be determined which targets belong to private information. For example, special attention may be paid to targets related to personal identity (such as faces, license plates, etc.), and these targets will be marked as private data that needs to be protected. Once the target is identified as private information, protection measures such as blurring will be applied to ensure that this sensitive information is not visible or recognizable in the final result.

[0045] The dynamic blurring scheme is a strategy that automatically adjusts the blurring intensity according to the target’s category, size, position, and environmental conditions (such as lighting, occlusion, etc.). Different privacy information may require different degrees of blurring. For example, for facial recognition, stronger blurring may be required to ensure that the original face cannot be restored from the blurred image. For license plate numbers, only the letters or numbers may need to be blurred to keep the license plate visible but unreadable.

[0046] When performing private information identification and obfuscation, grid aggregation can trace back and reorganize the information of each grid area. By tracing back to grid aggregation, the consistency and integrity of the relevant target area can be ensured when performing obfuscation. For example, if the boundary of the target spans multiple grids, tracing back to aggregation will help ensure the continuity and integrity of the target across grids.

[0047] Using a dynamic blurring scheme, actual blurring is performed based on the target information after back-aggregation, including Gaussian blurring, mosaic blurring, pixelation, etc., depending on the privacy protection requirements and target characteristics. Finally, the blurred image will be used as output, which will show the privacy-preserving result, where sensitive information (such as faces, license plates, etc.) has been blurred, while non-private information remains visible.

[0048] Furthermore, the privacy identification based on the category labeling and establishing a dynamic blurring solution include:

[0049] Create a basic privacy level based on the privacy identification results; obtain the identified application scenario, match the fuzzy scheme library based on the application scenario and the basic privacy level, and generate a scheme matching result; perform frame-by-frame background analysis on the backtracking results to establish a background matching result; and establish a dynamic fuzzy scheme based on the scheme matching result and the background matching result.

[0050] Specifically, based on the identified privacy targets, a basic privacy level is created to indicate the sensitivity of the privacy information. For example, the low privacy level refers to targets that are less related to personal privacy (e.g., non-facial background characters, public objects, etc.); the medium privacy level refers to targets that are less sensitive (e.g., incomplete and recognizable facial or license plate information); and the high privacy level refers to targets that are highly sensitive (e.g., clear faces, complete license plates, etc.).

[0051] Obtain the application scenario of the current image or video. The application scenario can be different environments such as static images, dynamic videos, surveillance videos, traffic surveillance, and home videos. In traffic surveillance, license plates are private information, while faces may not be so sensitive. In home surveillance, the protection of facial information may be more important, while license plate information may be secondary. According to the current application scenario and basic privacy level, select the corresponding fuzzy processing scheme from the pre-defined fuzzy scheme library. The fuzzy scheme library contains fuzzy processing strategies in different situations, which may include strong fuzzy processing schemes (applicable to high privacy level targets, such as sensitive information such as faces and license plates) and light fuzzy processing schemes (applicable to medium or low privacy level targets, such as blurred backgrounds, low confidence targets, etc.); according to the matching of application scenarios and privacy levels, generate the final scheme matching results, which will determine which private information needs to be strongly blurred, which needs to be lightly blurred, and which does not need to be blurred.

[0052] When processing dynamic videos, the impact of background information on privacy protection cannot be ignored. After tracing back the grid aggregation results, the background of each frame will be analyzed to check which background parts may affect the recognition of private information. The background may contain additional information related to private information, such as inadvertently displayed facial and license plate information. When analyzing the background of each frame, it will distinguish which parts of the dynamic scene must be blurred and which do not need to be blurred; the results of the background analysis will generate background matching results, indicating which background areas need to be blurred.

[0053] Ultimately, the dynamic blur solution will be generated based on the solution matching results and the background matching results. This process combines the type of privacy target, privacy level, application scenario, and background features in dynamic scenes to determine how to handle the privacy information in each frame of the image or video. For example, for targets with high privacy levels, if there is no privacy content related to the target in the background, a lighter blur may be applied; but if the background also involves privacy information, the blur intensity may need to be increased; for targets with low privacy levels, even if there is relevant information in the background, only a light blur may be used.

[0054] Furthermore, the identification management according to the fuzzy processing result, the category label and the confidence information also includes:

[0055] Call the preset function, draw the prediction box on the original file through OpenCV, and replace the original file partially according to the blur processing result. Then, identify the original file after partial replacement based on the category label and confidence information, and store it in the preset directory.

[0056] On the processed image (whether it is a static picture or a video frame), the OpenCV library is used to draw the prediction box, which corresponds to the target previously identified by the YOLOv5 model. For each detected target, OpenCV will mark a rectangular box on the image and indicate its category label and confidence value.

[0057] Based on the blurring results, the privacy-sensitive areas in the image are partially replaced to ensure that the privacy information in these areas cannot be identified. Specific methods of partial replacement may include blurring and occlusion processing; blurring: using Gaussian blur, mosaic or other blurring techniques to blur the target area containing privacy information (such as face, license plate, etc.) and replace it with an unrecognizable area; occlusion processing: for some sensitive targets, occlusion technology may be used, such as using black bars or mosaic blocks to cover the target area to ensure that the privacy information is hidden.

[0058] OpenCV can achieve privacy protection by manipulating the pixel values ​​of an image and replacing the area to be blurred with the corresponding blurred pattern. The specific replacement operations include: modifying the pixel color of the target area, applying a blur function, or replacing it with an occluded image.

[0059] After partial replacement, the image will be processed and its identification will be updated. The identification of the image can contain information about the privacy protection status of the image, such as category identification: showing the type of privacy targets detected in the image (such as faces, license plates, etc.), and their protection level (such as light blur or heavy blur); confidence identification: indicating the confidence value of each target. The processed image file will be stored according to the updated identification to ensure that the image information is effectively protected and provide traceability. The processed image (i.e., partial replacement and identification update) is saved to the preset storage directory.

[0060] Furthermore, after storing in the preset directory, a preset destruction function is called to release CUDA streams, GPU buffers and CPU memory resources, and destroy the engine, execution context and CUDA runtime instance.

[0061] After the image processing is completed and the processing results are stored in the preset directory, the system will call the preset destruction function to release all occupied computing and storage resources. CUDA streams are processing units used to perform asynchronous operations, which are used to manage the scheduling and execution of tasks during GPU computing. Each asynchronous operation (such as inference prediction, data transmission, etc.) will be assigned a stream for execution, and the stream needs to be released after execution. When the destruction function is called, the system will release all resources related to the CUDA stream to ensure that the GPU's memory is no longer occupied.

[0062] After completing image processing and file storage, the system will call the preset destruction function to release the occupied computing and storage resources to ensure that memory and computing resources are managed efficiently. First, the system will release the resources related to the CUDA stream, and avoid GPU memory occupation by destroying the stream. Next, the system will release all data in the GPU buffer, including input images and prediction results, to ensure that GPU memory is reclaimed. Then, the system will release all resources allocated in the CPU memory, clear image data and other computing data, and avoid memory overflow or performance degradation. At the same time, the system will destroy the inference engine and execution context, and release the memory and computing resources related to the execution of the inference task. Finally, the CUDA runtime instance will also be destroyed, and all resources related to GPU computing will be cleaned up by calling cudaDeviceReset to ensure that the system is in a clean state when the next computing task starts.

[0063] In summary, the embodiments of the present application have at least the following technical effects:

[0064] After loading the model, the files under the preset storage target are traversed, and classified reading is performed according to the file extension to establish the classified reading result. Then, the classified reading result is divided into preset grids, the preset grid division result is established, and the grid aggregation prediction of the preset grid division result is performed to establish the multi-level feature prediction result. Further, the multi-level feature prediction result is asynchronously executed through the CUDA stream using the infer function to perform model inference prediction, generate the inference prediction result, and perform the absolute coordinate transformation of the original image on the inference prediction result. Then, the inference prediction result based on non-maximum suppression is screened using the absolute coordinate transformation result, and the category annotation and confidence information are established according to the screening result. Finally, privacy identification is performed based on the category annotation, a dynamic fuzzy scheme is established, the grid aggregation is traced back based on the screening result, and the fuzzy processing of the traced result is performed using the dynamic fuzzy scheme, and the identification management is performed according to the fuzzy processing result, the category annotation and the confidence information. The technical problems of low efficiency of privacy information protection and insufficient accuracy of privacy identification in vehicle-mounted videos in the prior art are solved, and the technical effect of efficiently and accurately identifying and protecting privacy information in a complex environment is achieved.

[0065] Embodiment 2 is based on the same inventive concept as the method for protecting privacy information of in-vehicle video in a complex environment in the above embodiment. Figure 2 As shown, the present application provides a vehicle-mounted video privacy information protection system in a complex environment, wherein the system includes:

[0066] Classification reading module 11: after loading the model, traverse the files under the preset storage target, and perform classification reading according to the file extension, and establish the classification reading result; aggregation prediction module 12: perform preset grid division on the classification reading result, establish the preset grid division result, and perform grid aggregation prediction of the preset grid division result to establish a multi-level feature prediction result; reasoning prediction module 13: use the infer function to asynchronously execute model reasoning prediction on the multi-level feature prediction result through the CUDA stream to generate the reasoning prediction result, and perform the absolute coordinate transformation of the reasoning prediction result to the original image; screening module 14: use the absolute coordinate transformation result to filter the reasoning prediction result based on non-maximum suppression, and establish category labeling and confidence information according to the screening result; identification management module 15: perform privacy identification based on the category labeling, establish a dynamic fuzzy scheme, backtrack grid aggregation based on the screening result, and use the dynamic fuzzy scheme to perform fuzzy processing on the backtracking result, and perform identification management according to the fuzzy processing result, category labeling and confidence information.

[0067] Furthermore, the screening module 14 is used to perform the following method:

[0068] The maximum category confidence and category index of the inference prediction result corresponding to the absolute coordinate conversion result are obtained; the category confidence threshold is configured based on the maximum category confidence and the category index, and the inference prediction result is screened by the category confidence threshold to establish a retained result; an output format is established, and after the retained result is format-converted, non-maximum suppression screening is performed by the set intersection-over-union ratio threshold.

[0069] Furthermore, the screening module 14 is used to perform the following method:

[0070] The center coordinates and width and height information are obtained for the inference prediction result, and the frame coordinates are converted according to the center coordinates and the width and height information. The category confidence threshold is screened according to the inference prediction result after the frame coordinate conversion.

[0071] Furthermore, the identification management module 15 is used to execute the following method:

[0072] Call the preset function, draw the prediction box on the original file through OpenCV, and replace the original file partially according to the blur processing result. Then, identify the original file after partial replacement based on the category label and confidence information, and store it in the preset directory.

[0073] Furthermore, the identification management module 15 is used to execute the following method:

[0074] After storing in the preset directory, the preset destruction function is called to release the CUDA stream, GPU buffer and CPU memory resources, and destroy the engine, execution context and CUDA runtime instance.

[0075] Furthermore, the aggregation prediction module 12 is used to perform the following method:

[0076] Before performing the preset grid division, the classification reading results are copied to the GPU. After the GPU completes the preset grid division and obtains the inference prediction results, the inference prediction results are copied from the GPU back to the CPU memory.

[0077] Furthermore, the identification management module 15 is used to execute the following method:

[0078] Create a basic privacy level based on the privacy identification results; obtain the identified application scenario, match the fuzzy scheme library based on the application scenario and the basic privacy level, and generate a scheme matching result; perform frame-by-frame background analysis on the backtracking results to establish a background matching result; and establish a dynamic fuzzy scheme based on the scheme matching result and the background matching result.

[0079] Furthermore, the aggregation prediction module 12 is used to perform the following method:

[0080] Each grid of the preset grid division result is taken as the center, center aggregation is performed, and a multi-level center aggregation result is established; untrusted shapes are eliminated on the multi-level center aggregation result, and a multi-level feature prediction result is established.

[0081] Embodiment 3, Figure 3 The schematic diagram of the structure of the electronic device provided in the third embodiment of the present invention shows a block diagram of an exemplary electronic device suitable for implementing the implementation mode of the present invention. Figure 3 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present invention. Figure 3 As shown, the electronic device includes a processor 21, a memory 22, an input device 23 and an output device 24; the number of processors 21 in the electronic device can be one or more. Figure 3 Taking a processor 21 as an example, the processor 21, memory 22, input device 23 and output device 24 in the electronic device can be connected through a bus or other means. Figure 3 The example of connecting through bus is taken in the following.

[0082] It should be noted that the above-mentioned sequence of the embodiments of the present application is only for description and does not represent the advantages and disadvantages of the embodiments. And the above-mentioned specific embodiments of this specification are described. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0083] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

[0084] This specification and drawings are merely exemplary illustrations of the present application and are deemed to cover any and all modifications, variations, combinations or equivalents within the scope of the present application. Obviously, a person skilled in the art may make various modifications and variations to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present application and its equivalents, the present application intends to include these modifications and variations.

Claims

1. A method for protecting privacy information of vehicle-mounted video in complex environments, characterized in that: The method comprises: After loading the model, traverse the files under the preset storage target, and classify and read them according to the file extensions to establish the classification reading results; Performing preset grid division on the classified reading result, establishing a preset grid division result, and performing grid aggregation prediction of the preset grid division result to establish a multi-level feature prediction result; The multi-level feature prediction results are asynchronously performed using the infer function through the CUDA stream to perform model inference prediction, generate inference prediction results, and perform absolute coordinate conversion of the inference prediction results to the original image; The absolute coordinate transformation results are used to screen the inference prediction results based on non-maximum suppression, and the category labeling and confidence information are established according to the screening results; Based on the category annotation, privacy identification is performed, a dynamic fuzzy scheme is established, grid aggregation is traced back based on the screening results, and the dynamic fuzzy scheme is used to fuzzify the traced results, and identification management is performed based on the fuzzy processing results, category annotations, and confidence information.

2. The method for protecting privacy information of in-vehicle video in a complex environment as claimed in claim 1, characterized in that: The method of using the absolute coordinate transformation result to screen the inference prediction result based on non-maximum suppression includes: Get the maximum category confidence and category index of the inference prediction result corresponding to the absolute coordinate transformation result; A category confidence threshold is configured based on a maximum category confidence and a category index, and the inference prediction results are screened by the category confidence threshold to establish a retention result; An output format is established, and after the retained results are format converted, non-maximum suppression screening is performed using a set intersection-over-union ratio threshold.

3. The method for protecting the privacy information of in-vehicle video in a complex environment as claimed in claim 2, characterized in that: Before the inference prediction result screening is performed by the category confidence threshold, the method includes: The center coordinates and width and height information are obtained for the inference prediction result, and the frame coordinates are converted according to the center coordinates and the width and height information. The category confidence threshold is screened according to the inference prediction result after the frame coordinate conversion.

4. The method for protecting privacy information of in-vehicle video in a complex environment as claimed in claim 1, characterized in that: The identification management according to the fuzzy processing result, the category label and the confidence information also includes: Call the preset function, draw the prediction box on the original file through OpenCV, and replace the original file partially according to the blur processing result. Then, identify the original file after partial replacement based on the category label and confidence information, and store it in the preset directory.

5. The method for protecting privacy information of in-vehicle video in a complex environment as claimed in claim 4, characterized in that: After storing in the preset directory, the preset destruction function is called to release the CUDA stream, GPU buffer and CPU memory resources, and destroy the engine, execution context and CUDA runtime instance.

6. The method for protecting privacy information of in-vehicle video in a complex environment as claimed in claim 1, characterized in that: Before performing the preset grid division, the classification reading results are copied to the GPU. After the GPU completes the preset grid division and obtains the inference prediction results, the inference prediction results are copied from the GPU back to the CPU memory.

7. The method for protecting privacy information of in-vehicle video in a complex environment as claimed in claim 1, characterized in that: The privacy identification based on the category labeling and establishing a dynamic fuzzy solution include: Create a basic privacy level based on the privacy identification results; Acquire the identified application scenario, and generate a solution matching result based on the application scenario and the basic privacy level matching fuzzy solution library; Perform frame-by-frame background analysis on the traceback results to establish background matching results; A dynamic blurring scheme is established according to the scheme matching result and the background matching result.

8. The method for protecting privacy information of in-vehicle video in a complex environment as claimed in claim 1, characterized in that: The method further comprises: performing a grid aggregation prediction of a preset grid division result and establishing a multi-level feature prediction result, including: Taking each grid of the preset grid division result as the center, performing center aggregation to establish a multi-level center aggregation result; Untrusted shapes are eliminated from the multi-level center aggregation results to establish a multi-level feature prediction result.

9. The vehicle-mounted video privacy information protection system in complex environments is characterized by: The system is used to implement the method for protecting privacy information of in-vehicle video in a complex environment as described in any one of claims 1 to 8, and comprises: Classification reading module: After loading the model, it traverses the files under the preset storage target, and performs classification reading according to the file extension to establish the classification reading results; Aggregation prediction module: performs preset grid division on the classification reading result, establishes preset grid division result, and performs grid aggregation prediction of the preset grid division result to establish multi-level feature prediction result; Inference prediction module: using the infer function to asynchronously perform model inference prediction on the multi-level feature prediction results through the CUDA stream to generate inference prediction results, and performing absolute coordinate conversion of the inference prediction results to the original image; Screening module: Use the absolute coordinate transformation results to screen the inference prediction results based on non-maximum suppression, and establish category annotations and confidence information based on the screening results; Identification management module: perform privacy identification based on the category annotation, establish a dynamic fuzzy scheme, trace back grid aggregation based on the screening results, and use the dynamic fuzzy scheme to fuzzify the traced results, and perform identification management based on the fuzzy processing results, category annotations and confidence information.

10. An electronic device, characterized in that: The electronic device comprises: A memory for storing executable instructions; The processor is used to implement the vehicle-mounted video privacy information protection method in a complex environment according to any one of claims 1 to 8 when executing the executable instructions stored in the memory.

Images