Method for desensitizing target object in image in vehicle driving process

Abstract

The embodiment of the invention discloses a method for desensitizing a target object in an image in a vehicle driving process, and the method comprises the steps: determining a position region of the target object in a current environment image in the vehicle driving process; based on the position area, determining a preset type area in the target object in the current environment image; determining a detection result in the preset type region; and based on the detection result, performing desensitization processing on the preset type region in the current environment image to obtain a desensitized image corresponding to the current environment image. According to the embodiment of the invention, the accuracy of the to-be-desensitized area can be effectively improved, and the problem of missing detection of the to-be-desensitized area is avoided.

Description

Technical Field

[0001] The present disclosure relates to computer vision technology, and more particularly to a method for desensitizing a target object in an image during vehicle driving. Background Art

[0002] In the field of intelligent driving, vehicles are usually equipped with cameras, which will collect environmental images around the vehicle (which can be called current environmental images) during driving. However, the collected images may contain some sensitive information, such as license plate information, face information and other sensitive information. For the security of information, the collected images need to be desensitized. In related technologies, sensitive areas are usually screened from candidate areas that may belong to sensitive areas by pre-setting confidence thresholds. This method based on pre-set confidence thresholds is easily affected by the pre-set confidence thresholds and has the problem of missed detection of sensitive areas. Summary of the invention

[0003] The embodiments of the present disclosure provide a method for desensitizing a target object in an image during vehicle driving, which can improve the accuracy of the desensitized area and avoid missed detection or false detection of sensitive areas.

[0004] A first aspect of an embodiment of the present disclosure provides a method for desensitizing a target object in an image during vehicle driving, comprising: during vehicle driving, determining a position area of ​​a target object in a current environment image; based on the position area, determining a preset type area in the target object in the current environment image; determining a detection result in the preset type area; based on the detection result, desensitizing the preset type area in the current environment image to obtain a desensitized image corresponding to the current environment image.

[0005] A second aspect of an embodiment of the present disclosure provides a device for desensitizing a target object in an image during vehicle driving, comprising: a first determination module, used to determine a position area of ​​a target object in a current environment image during vehicle driving; a first processing module, used to determine a preset type area of ​​the target object in the current environment image based on the position area; a second processing module, used to determine a detection result in the preset type area; and a third processing module, used to desensitize the preset type area in the current environment image based on the detection result to obtain a desensitized image corresponding to the current environment image.

[0006] According to a third aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, wherein the storage medium stores a computer program, and the computer program is used to execute the method for desensitizing a target object in an image during vehicle driving as described in any of the above embodiments of the present disclosure.

[0007] According to a fourth aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a processor; a memory for storing executable instructions of the processor; the processor is configured to read the executable instructions from the memory and execute the instructions to implement the method for desensitizing a target object in an image during vehicle driving as described in any one of the above embodiments of the present disclosure.

[0008] A fifth aspect of the embodiments of the present disclosure provides a computer program product. When the instructions in the computer program product are executed by a processor, the method for desensitizing a target object in an image during vehicle driving provided in any of the above embodiments of the present disclosure is performed.

[0009] Based on the method for desensitizing the target object in the image during the vehicle driving process provided by the above-mentioned embodiment of the present disclosure, the detection result of the sensitive area (such as the license plate area in the target vehicle) is determined by combining the position area of ​​the target object (such as the target vehicle) in the current environment image and the preset type area in the target object (i.e., the candidate area that may belong to the sensitive area), which can avoid using the preset confidence threshold to screen the sensitive area from multiple candidate areas, and the sensitive area belongs to the partial area of ​​the target object. The area occupied by the target object in the current environment image is often larger than the sensitive area on the target object. Therefore, the detection of the target object has a higher detection accuracy. The sensitive area information is constrained by the position area information of the target object, which can effectively improve the accuracy of the sensitive area and avoid the problem of missed detection of the sensitive area. For example, the target object takes the target vehicle (which can also be a target pedestrian, a target sign, etc. in actual application) as an example, and the license plate candidate area with the highest confidence built in the position area of ​​each target vehicle in the current environment image is used as the license plate area of ​​the target vehicle, and the license plate area in the current environment image is desensitized. For another example, based on the number N (N is an integer) of target vehicles in the current environment image, target candidate areas with confidence TOPN (i.e., the highest N confidences) are selected from the license plate candidate areas in the current environment image as license plate areas, and the license plate areas in the current environment image are desensitized to ensure that the number of license plate areas is consistent with the number of target vehicles, thereby effectively avoiding missed license plate detections. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 This is an exemplary application scenario of the method provided by the present disclosure for desensitizing a target object in an image during vehicle driving;

[0011] Figure 2 is a flow chart of a method for desensitizing a target object in an image during vehicle driving provided by an exemplary embodiment of the present disclosure;

[0012] Figure 3 is a flow chart of a method for desensitizing a target object in an image during vehicle driving provided by another exemplary embodiment of the present disclosure;

[0013] Figure 4 is a flow chart of a method for desensitizing a target object in an image during vehicle driving provided by yet another exemplary embodiment of the present disclosure;

[0014] Figure 5 is a flowchart of a method for desensitizing a target object in an image during vehicle driving provided by another exemplary embodiment of the present disclosure;

[0015] Figure 6 is a flow chart of a method for desensitizing a target object in an image during vehicle driving provided by yet another exemplary embodiment of the present disclosure;

[0016] Figure 7 is a flowchart of a method for desensitizing a target object in an image during vehicle driving provided by another exemplary embodiment of the present disclosure;

[0017] Figure 8 is a flowchart of a method for desensitizing a target object in an image during vehicle driving provided by an exemplary embodiment of the present disclosure;

[0018] Fig. 9 is a schematic structural diagram of a device for desensitizing a target object in an image during vehicle driving provided by an exemplary embodiment of the present disclosure;

[0019] Fig.10 is a schematic structural diagram of a device for desensitizing a target object in an image during vehicle driving provided by another exemplary embodiment of the present disclosure;

[0020] Fig.11 is a schematic structural diagram of a device for desensitizing a target object in an image during vehicle driving provided by yet another exemplary embodiment of the present disclosure;

[0021] Fig.12 is a schematic structural diagram of a device for desensitizing a target object in an image during vehicle driving provided by another exemplary embodiment of the present disclosure;

[0022] Fig.13 is a schematic structural diagram of a device for desensitizing a target object in an image during vehicle driving provided by yet another exemplary embodiment of the present disclosure;

[0023] Fig.14 is a schematic structural diagram of a device for desensitizing a target object in an image during vehicle driving provided by another exemplary embodiment of the present disclosure;

[0024] Fig.15 It is a structural diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0025] To explain the present disclosure, example embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. It should be understood that the present disclosure is not limited to the example embodiments.

[0026] It should be noted that the relative arrangement of components and steps, the numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.

[0027] SUMMARY OF THE DISCLOSURE

[0028] In the process of realizing the present disclosure, the inventors found that in the field of intelligent driving, a camera is usually installed on the vehicle, and the environment image around the vehicle (which can be called the current environment image) will be collected during the driving process of the vehicle. However, the collected image may contain some sensitive information, such as license plate information, face information and other sensitive information. For the security of information, the collected image needs to be desensitized. In the related art, sensitive areas are usually screened from candidate areas that may belong to sensitive areas by pre-setting confidence thresholds. For example, multiple license plate candidate areas are detected from the image, and the confidence of each license plate candidate area is compared with the preset confidence threshold. The license plate candidate area greater than the preset confidence threshold is used as the license plate area. This method based on the preset confidence threshold is easily affected by the preset confidence threshold and has the problem of missed detection of sensitive areas. For example, the actual image contains 6 license plates, and only 5 license plate areas are obtained according to the preset confidence threshold, resulting in the problem of missed detection of license plate areas.

[0029] Exemplary Overview

[0030] Figure 1 This is an exemplary application scenario of the method provided by the present disclosure for desensitizing a target object in an image during vehicle driving. Figure 1 As shown, a camera 12 is provided on the vehicle 11. When the vehicle 11 is traveling on a road (a two-lane road formed by three lane lines 13 is taken as an example in the figure), the camera 12 collects an environmental image (i.e., a current environmental image) around the vehicle 11. By using the method for desensitizing a target object in an image during vehicle driving (hereinafter referred to as the method) according to an embodiment of the present disclosure, the target object (e.g., Figure 1The position area of ​​the target vehicle 14 in front of the self-vehicle 11 in the current environment image; based on the position area, determine the preset type area in the target object in the current environment image; determine the detection result in the preset type area; based on the detection result, desensitize the preset type area in the current environment image to obtain the desensitized image corresponding to the current environment image. The preset type area in the target object in the current environment image is a candidate area that may belong to the sensitive area in the target object in the current environment image. Combining the position area of ​​the target object in the current environment image and the preset type area in the target object, the detection result of the sensitive area (such as the license plate area in the target vehicle) is determined, which can avoid using a preset confidence threshold to screen the sensitive area from multiple candidate areas, and for a target object, in the current environment image, the sensitive area in the target object belongs to a partial area of ​​the target object, and the area occupied by the target object in the current environment image is often larger than the sensitive area on the target object, so the detection of the target object has a higher detection accuracy, and the sensitive area information is constrained by the position area information of the target object, which can effectively improve the accuracy of the sensitive area and avoid the problem of missed detection of the sensitive area. For example, the target object is the target vehicle 14 in the figure (the target object in actual application is not limited to the target vehicle, but can also be a target pedestrian, a target signboard, etc.), and the license plate candidate area with the highest confidence built in the position area of ​​each target vehicle 14 in the current environment image is used as the license plate area of ​​the target vehicle 14, and the license plate area in the current environment image is desensitized. For another example, according to the number N of target vehicles 14 in the current environment image, the target candidate area with confidence TOPN is selected from the license plate candidate area in the current environment image as the license plate area, and the license plate area in the current environment image is desensitized to ensure that the number of license plate areas is consistent with the number of target vehicles, effectively avoiding license plate leakage. The target object can include any object that may contain sensitive information, such as vehicles (which can be called target vehicles), pedestrians (which can be called target pedestrians), and signboards (which can be called target signboards). Sensitive information includes license plates, faces, road names on signboards, city names, etc.

[0031] In practical applications, the number of cameras set on the vehicle 11 can be one or more, not limited to the above-mentioned camera 12. For example, a front view camera, a rear view camera, a left view camera, a right view camera, etc. can be set on the vehicle 11, and there is no specific limitation. In the case of multiple cameras, the image captured by each camera can be used as the current environment image, and desensitization processing can be performed according to the method of the embodiment of the present disclosure to obtain the desensitized image corresponding to each image.

[0032] Exemplary Methods

[0033] Figure 2is a flow chart of a method for desensitizing a target object in an image during vehicle driving provided by an exemplary embodiment of the present disclosure. This embodiment can be applied to electronic devices, such as a vehicle-mounted computing platform, such as Figure 2 As shown, the method of the embodiment of the present disclosure may include the following steps:

[0034] Step 210 : during the driving process of the vehicle, determine the location area of ​​the target object in the current environment image.

[0035] Among them, during the driving process of the vehicle (i.e., the vehicle itself), the environmental image around the vehicle can be collected by the camera on the vehicle as the current environmental image. The target objects may include other vehicles (referred to as target vehicles), pedestrians, signboards and other objects that may contain sensitive information (or information to be desensitized). Sensitive information may include, for example, license plate information on the target vehicle, facial information of pedestrians, city names and road name information on signboards, etc. The type of specific target object is not limited.

[0036] In some optional embodiments, the position region of the target object in the current environment image may be a region of a preset shape occupied by the target object in the current environment image. The position region may be represented in any manner, for example, the position region may be represented as the pixel coordinates of the center point of the region in the current environment image and the region size information, or the position region may be represented as the coordinates of the corner points of the region. Specifically, for example, the position region may be represented as a rectangular frame region.

[0037] In some optional embodiments, the location area of ​​the target object in the current environment image can be obtained by detecting the current environment image using a preconfigured target detection model, or by other means. The target detection model can use any detection model obtained by pre-training, which is not limited in the embodiments of the present disclosure.

[0038] In some optional embodiments, the number of target objects in the current environment image may be one or more. In the case where the number of target objects is more than one, the location area of ​​each target object in the current environment image may be determined.

[0039] Step 220: Determine a preset type area in the target object in the current environment image based on the location area.

[0040] Among them, the preset type area in the target object in the current environment image refers to a candidate area that may belong to the area to be desensitized (or sensitive area). The preset type area may include the area where the license plate is located, the area where the face is located, the area where the sensitive information on the sign is located, etc., without specific limitation. The types of preset type areas corresponding to different target objects are different. For example, the preset type area in the target vehicle is the area where the license plate is located, and the preset type area in the target pedestrian is the area where the face is located.

[0041] In some optional embodiments, the preset type area in the target object in the current environment image can be determined in different ways based on the position area. For example, the different ways can include directly performing preset type area detection on the current environment image to obtain the preset type area, performing preset type area detection on the target object area corresponding to the position area in the current environment image to obtain the preset type area, etc.

[0042] Step 230, determining the detection result in the preset type area.

[0043] The detection result refers to the result of whether the preset type area is the area to be desensitized. For example, the detection result includes the result of whether each preset type area that may be a license plate area in the target vehicle is a license plate.

[0044] In some optional embodiments, it can be determined whether a preset type area is an area to be desensitized based on the confidence level of the preset type area.

[0045] Step 240, based on the detection result, desensitize the preset type area in the current environment image to obtain a desensitized image corresponding to the current environment image.

[0046] Among them, according to the detection results, the preset type area belonging to the area to be desensitized in the current environment image can be desensitized to obtain the desensitized image corresponding to the current environment image. For example, according to the detection results, the area determined as a license plate in the current environment image can be desensitized, the area determined as a face in the current environment image can be desensitized, and so on.

[0047] In some optional embodiments, the desensitization processing method for the relevant area in the current environment image may be any method. For example, the desensitization processing method may include: changing the brightness, contrast, and color of the pixels in the area in the current environment image to blur the sensitive information in the area.

[0048] The method for desensitizing a target object in an image during vehicle driving provided in the present embodiment combines the position area of ​​the target object in the current environment image and a preset type area in the target object (i.e., a candidate area that may belong to the area to be desensitized) to determine the detection result of the area to be desensitized, thereby avoiding the use of a preset confidence threshold to screen the area to be desensitized from multiple candidate areas. The area to be desensitized belongs to a partial area of ​​the target object, and the area occupied by the target object in the current environment image is often larger than the area to be desensitized on the target object. Therefore, the detection of the target object has a higher detection accuracy. The information of the area to be desensitized is constrained by the position area information of the target object, which can effectively improve the accuracy of the area to be desensitized and avoid the problem of missed detection of the area to be desensitized.

[0049] Figure 3 It is a flowchart of a method for desensitizing a target object in an image during vehicle driving provided by another exemplary embodiment of the present disclosure.

[0050] In some optional embodiments, in the above Figure 2 Based on the embodiment shown, Figure 3 As shown, step 220 of determining a preset type area in the target object in the current environment image based on the position area may include:

[0051] Step 2210: perform detection processing on the current environment image to obtain a candidate area in the current environment image.

[0052] Among them, the candidate area in the current environment image is a candidate area detected from the current environment image that may belong to the area to be desensitized.

[0053] In some optional embodiments, the current environment image may be detected and processed by a preconfigured target detection model to obtain candidate regions in the current environment image. The target detection model may include, for example, a single-task detection model such as a license plate detection model and a face detection model, as well as a multi-task detection model. A multi-task detection model may detect and obtain multiple types of target objects at the same time by one model. For example, the current environment image may be detected and processed by a multi-task detection model to obtain candidate regions that may belong to license plates, candidate regions that may belong to faces, etc. in the current environment image.

[0054] Step 2220: Based on the location area, determine a preset type area in the target object from the candidate areas.

[0055] Among them, there is a certain relative position relationship between the candidate area detected directly from the current environment image and the position area of ​​each target object. According to the relative position relationship between the candidate area and the position area of ​​each target object, the preset type area in each target object can be determined from the candidate area, that is, the candidate area that may belong to the area to be desensitized in each target object is determined. For example, the license plate candidate area in the current environment image should be located within the position area range of the corresponding target vehicle. Based on the relative position relationship between the license plate candidate area and the position area of ​​each target vehicle, the license plate candidate area in each target vehicle (that is, the preset type area in the target vehicle) can be determined from the license plate candidate area.

[0056] In some optional embodiments, the number of preset type areas in each target object may be one or more. If the number of preset type areas in the target object is one, it can be determined that the detection result of the preset type area in the target object is that the preset type area is an area to be desensitized, and the preset type area is used as the target preset type area in the target object. If the number of preset type areas in the target object is multiple, the confidence of the multiple preset type areas can be determined, and a preset type area with the largest confidence is selected from the multiple preset type areas as the target preset type area in the target object. The target preset type area in the current environment image is desensitized to obtain a desensitized image.

[0057] In the embodiments of the present disclosure, since the candidate area detected from the current environment image that may belong to the area to be desensitized has a certain relative position relationship with the position area of ​​the target object, and according to the method that the area to be desensitized in the target object should be within the position area of ​​the target object, the preset type area in each target object can be accurately obtained from the candidate area, and then the preset type area of ​​the target to be desensitized in the target object can be determined from the preset type area in each target object, ensuring that the area to be desensitized corresponds to the target object one-to-one, avoiding the problem of missed detection of the area to be desensitized.

[0058] Figure 4 It is a flowchart of a method for desensitizing a target object in an image during vehicle driving provided by yet another exemplary embodiment of the present disclosure.

[0059] In some optional embodiments, based on any of the above embodiments, Figure 4 As shown, step 230 of determining the detection result in the preset type area may include:

[0060] Step 2310: Detect and process the current environment image using a preconfigured detection model to obtain confidences corresponding to the candidate areas of the preset type in the current environment image.

[0061] In some optional embodiments, the preconfigured detection model may be any confidence detection model, such as a convolutional neural network-based model, a Transformer-based model, a residual network-based model, etc. The confidence may be expressed as the probability that the candidate region belongs to the region to be desensitized.

[0062] In some optional embodiments, the preconfigured detection model may be a target detection model, through which candidate regions belonging to preset types of regions in the current environment image and the corresponding confidence of each candidate region may be detected simultaneously. For example, the detection model is a license plate detection model, through which the current environment image is detected and processed to obtain candidate regions that may belong to the license plate region in the current environment image and the corresponding probability of each candidate region.

[0063] Step 2320 , based on the confidences corresponding to the candidate regions, determine the confidence corresponding to each preset type region in the target object.

[0064] According to the relative position relationship between the candidate region and the position region of the target object, the candidate region in each target object can be determined from the candidate regions as the preset type region in the target object, and the confidence corresponding to each candidate region in the target object is used as the confidence corresponding to the corresponding preset type region in the target object.

[0065] Step 2330 , determining the attribute of each preset type region in the target object according to the confidence level corresponding to each preset type region.

[0066] The attribute of each preset type area refers to whether the preset type area is a to-be-desensitized area. For example, whether the preset type area is a license plate area, a face area, etc. The attribute may include a first preset attribute and a second preset attribute.

[0067] In some optional embodiments, the attribute of each preset type area can be expressed as 1 (i.e., the first preset attribute) or 0 (i.e., the second preset attribute), 1 indicates that the preset type area belongs to the area to be desensitized, and 0 indicates that the preset type area does not belong to the area to be desensitized.

[0068] In some optional embodiments, for each target object, if there is only one preset type area in the target object, the attribute of the preset type area can be determined as the first preset attribute; if there are multiple preset type areas in the target object, the attribute of the preset type area with the highest confidence among the multiple preset type areas can be determined as the first preset attribute based on the confidence levels corresponding to each of the multiple preset type areas, and the attributes of the other preset type areas among the multiple preset type areas except the preset type area with the highest confidence can be determined as the second preset attribute.

[0069] Step 2340, determining the detection result according to the attributes of each preset type area.

[0070] The attribute of each preset type area may be used as the detection result, or the target preset type area belonging to the first preset type may be used as the detection result, and the specific content of the detection result is not limited.

[0071] In the embodiments of the present disclosure, since the confidence corresponding to each preset type area in each target object can be obtained based on the confidence corresponding to each candidate area belonging to the preset type area in the current environment image, it is possible to obtain, for each target object, a target preset type area with an attribute of the first preset attribute in the target object as the area to be desensitized of the target object, thereby ensuring a one-to-one correspondence between the area to be desensitized and the target object, thereby avoiding the problem of missed detection of the area to be desensitized.

[0072] Figure 5 It is a flowchart of a method for desensitizing a target object in an image during vehicle driving, provided by yet another exemplary embodiment of the present disclosure.

[0073] In some optional embodiments, in the above Figure 2 Based on the embodiment shown, Figure 5 As shown, step 220 of determining a preset type area in the target object in the current environment image based on the position area may include:

[0074] Step 22a0, determining the target object area from the current environment image according to the position area.

[0075] The target object region is the region occupied by the target object in the current environment image, that is, the target object region can be cut out from the current environment image according to the position region.

[0076] Step 22b0, determining a preset type area in the target object in the current environment image according to the target object area.

[0077] The target object area can be detected by a preconfigured target detection model to obtain a preset type area of ​​the target object in the current environment. The target detection model may include, for example, a license plate detection model, a face detection model, a multi-task detection model, and the like.

[0078] In some optional embodiments, each target object region in the current environment image may be detected and processed to obtain a preset type region in the target object corresponding to each target object region. Then, the confidence of the preset type region in each target object region may be determined, and the region to be desensitized in each target object may be determined according to the confidence of each preset type region in each target object region.

[0079] In the disclosed embodiment, since the preset type area in the target object can be determined for each target object area, each target object can obtain a preset type area belonging to the area to be desensitized, ensuring the consistency between the number of areas to be desensitized and the number of target objects, avoiding the problem of missed detection of areas to be desensitized.

[0080] In some optional embodiments, such as Figure 5 As shown, step 230 of determining the detection result in the preset type area may include:

[0081] Step 23a0, determining the confidence level corresponding to each preset type of area in the target object area.

[0082] In some optional embodiments, the confidence corresponding to each preset type of region in the target object region can be obtained based on the confidence detection model detection. For example, the corresponding region portion can be cut out from the current environment image based on each preset type of region, and the confidence detection is performed on the region portion corresponding to each preset type of region based on the pre-configured trained confidence detection model to obtain the confidence corresponding to each preset type of region.

[0083] In some optional embodiments, the confidence level corresponding to each preset type region in the target object region can be obtained by performing detection processing on the target object region using the target detection model described above. That is, the target detection model can perform detection processing on each target object region to obtain each preset type region in the target object and the confidence level corresponding to each preset type region.

[0084] Step 23b0, determining whether each preset type area belongs to the area to be desensitized according to the confidence level corresponding to each preset type area in the target object.

[0085] Among them, the state of belonging to the area to be desensitized includes two states: the preset type area belongs to the area to be desensitized and does not belong to the area to be desensitized. For each target object, it can be determined that a preset type area with the largest confidence in the target object belongs to the area to be desensitized, and preset type areas with other confidences except the maximum confidence do not belong to the area to be desensitized.

[0086] Step 23c0, determining the detection result according to the status of each preset type area belonging to the area to be desensitized.

[0087] Among them, the state of each preset type area belonging to the area to be desensitized can be used as the detection result, or the state of the preset type area belonging to the area to be desensitized can be used as the detection result.

[0088] According to the disclosed embodiments, the area to be desensitized corresponding to each target object can be obtained, thereby avoiding the problem of missed detection of the area to be desensitized and improving the accuracy of the area to be desensitized.

[0089] Figure 6 It is a flowchart of a method for desensitizing a target object in an image during vehicle driving provided by yet another exemplary embodiment of the present disclosure.

[0090] In some optional embodiments, in the above Figure 2 Based on the embodiment shown, Figure 6 As shown, step 220 of determining a preset type area in the target object in the current environment image based on the position area may include:

[0091] Step 2201, determining a preset type area from the current environment image.

[0092] Wherein, the preset type area can be determined from the current environment image based on the preconfigured target detection model. For details, please refer to the above embodiment.

[0093] Step 2202, determining the confidence level corresponding to each preset type of area.

[0094] The confidence level corresponding to each preset type of area may be determined based on a confidence detection model or the above-mentioned target detection model.

[0095] Step 2203: Determine the preset type area in the target object in the current environment image based on the number of position areas and the confidence level corresponding to each preset type area.

[0096] Among them, based on the number of position areas N, the top-N preset type areas with high confidence rankings can be determined from the current environment image as the preset type areas in the target object in the current environment image. In this case, it can be determined that the preset type areas in each target object in the current environment image are all areas to be desensitized.

[0097] In the disclosed embodiment, since the number of location areas of the target objects represents the number of target objects, the preset type areas in the target objects are constrained by the number of target objects as areas to be desensitized. This can make the number of areas to be desensitized consistent with the number of target objects, thereby avoiding the problem of missed detection of areas to be desensitized and improving the accuracy of the desensitization processing.

[0098] In some optional embodiments, determining the position area of ​​the target object in the current environment image in step 210 may include: obtaining the position area of ​​the target object in the current environment image from a driving assistance system on the vehicle; the driving assistance system determines the position area of ​​the target object in the current environment image based on the current environment image and sensor data collected by at least one preset sensor; the type of the preset sensor is different from the type of sensor for collecting the current environment image.

[0099] Among them, the preset sensors may include, for example, radar, lidar, millimeter wave radar, ultrasonic radar and other sensors.

[0100] In some optional embodiments, the driving assistance system may be, for example, an Advanced Driving Assistance System (ADAS). The driving assistance system may sense the vehicle's surroundings based on multimodal sensor data such as image data, radar data, lidar data, millimeter-wave radar data, and ultrasonic radar data, and obtain a perception result of the surrounding environment for downstream planning and control. The perception result may include the location area of ​​the target object in the current environment image. Therefore, the location area of ​​the target object in the current environment image may be obtained from the driving assistance system.

[0101] In the disclosed embodiment, when desensitizing the target object in the current environment image, the perception result of the target object by the driving assistance system on the vehicle can be reused, thereby avoiding the deployment of an additional detection model for detecting the target object on the vehicle side, reducing the occupation of vehicle-side resources, and saving the processing task of performing target object detection separately, thereby improving the efficiency of the desensitization process. In addition, since the perception result of the target object by the driving assistance system is obtained based on the perception of multiple types of sensor data, the detection result precision and accuracy of the multi-sensor data are higher than that of detecting the target object by a single image, thereby improving the accuracy of the location area of ​​the target object in the current environment image.

[0102] Figure 7 It is a flowchart of a method for desensitizing a target object in an image during vehicle driving, provided by yet another exemplary embodiment of the present disclosure.

[0103] In some optional embodiments, based on any of the above embodiments, step 240 performs desensitization processing on the preset type area in the current environment image based on the detection result to obtain a desensitized image corresponding to the current environment image, which may include:

[0104] Step 2410: Based on the detection result, determine a target preset type area whose attribute is a first preset attribute from the preset type areas.

[0105] The first preset attribute refers to the preset type area belonging to the area to be desensitized. The target preset type area belonging to the first preset attribute may be, for example, a license plate area, a face area, etc. The detection result includes the attribute corresponding to each preset type area, and according to the attribute corresponding to each preset type area, a target preset type area whose attribute is the first preset attribute is determined from the preset type areas.

[0106] Step 2420, desensitizing the target preset type area in the current environment image to obtain a desensitized image corresponding to the current environment image.

[0107] After determining the target preset type area with the first preset attribute, the target preset type area in the current environment image may be desensitized. For example, the target preset type area in the current environment image may be blurred by changing pixel color, mosaic, etc., so as to blur the sensitive information in the target preset type area and ensure information security.

[0108] In the embodiments of the present disclosure, since the area to be desensitized (i.e., the target preset type area) is determined based on the detection results with higher accuracy, the accuracy of the desensitization processing results can be improved to avoid missed detection of sensitive information in the desensitized image.

[0109] In some optional embodiments, Figure 8 FIG. 1 is a flowchart of a method for desensitizing a target object in an image during vehicle driving provided by an exemplary embodiment of the present disclosure. Figure 8 As shown, the target object is a target vehicle as an example, and the area to be desensitized is the license plate area. The process of the method may include the following steps:

[0110] Step 310, obtaining the target vehicle area and the number N of target vehicles in the current environment image from the driving assistance system 111 on the vehicle. The driving assistance system 111 obtains the target vehicle area and the number of target vehicles in the current environment image based on multimodal data perception. The multimodal data includes the current environment image and sensor data collected by at least one preset sensor.

[0111] Step 320: Perform license plate detection on each target vehicle area through the license plate detection model to obtain a list of candidate areas (i.e., preset type areas) that may belong to the license plate area. The candidate area list includes each candidate area and the corresponding confidence level. The number of candidate areas included in the candidate area list is greater than or equal to N.

[0112] Step 330 , according to the number N of target vehicles and the confidence of each candidate area, N target candidate areas (ie, target preset type areas) are determined from the candidate areas as license plate areas.

[0113] Step 340, desensitizing the N license plate areas in the current environment image to obtain a desensitized image corresponding to the current environment image.

[0114] Figure 8 This is only an exemplary flow chart and is not intended to limit the method of the embodiment of the present disclosure. For example, the target object may also be a target pedestrian, a target sign, or other types of objects.

[0115] The embodiments of the present disclosure can avoid deploying additional models for detecting target objects on the vehicle side by reusing the overall perception results of the target object by the driving assistance system 111. On the one hand, the occupation of vehicle-side resources (such as storage resources and processor resources) can be reduced. On the other hand, since the driving assistance system 111 obtains the overall perception results of the target object based on multimodal data, the accuracy of the target object area and the number of target objects can be improved. Based on the number of target objects, target candidate areas with the same number of target objects can be screened out from the list of candidate areas that may belong to the areas to be desensitized as the areas to be desensitized, thereby improving the accuracy of the areas to be desensitized and avoiding missed detection of license plate areas and face areas waiting for desensitization.

[0116] In the related art, the area to be desensitized is determined based on a preset confidence threshold (or hyperparameter) set manually. For example, M candidate areas Pi (i=1, 2, ..., M) that may belong to sensitive areas are detected from the image. The confidence representation pi corresponding to each candidate area Pi is greater than 0 and less than 1, and the candidate area list is represented as P1:p1, P2:p2, ..., PM:pM. The preset confidence threshold is, for example, 0.95, then the confidence representation pi corresponding to each candidate area Pi is compared with 0.95, and only the candidate areas greater than 0.95 will be determined as the area to be desensitized. Since the preset confidence threshold is set manually, it lacks the universality of the scene, and there is a certain deviation in the target recall rate for different scenes. It is easily affected by the environment, resulting in the problem of missed detection of the target. For example, if the number of sensitive areas actually contained in the image is N, only Nj candidate areas among the M candidate areas may have a confidence greater than the preset confidence threshold, which will cause j sensitive areas to be missed. M, N, and j are all integers. N is greater than j.

[0117] In response to this missed detection problem in the related art, the method of the embodiment of the present invention determines the sensitive area based on the position area of ​​the entire target object associated with the sensitive area in the current environment image or the number of target objects, so that the number of sensitive areas is consistent with the number of target objects. For example, if N target vehicles are detected in the current environment image, N license plate areas are determined from the M detected candidate areas that may belong to the license plate area, ensuring that the number of license plate areas is consistent with the number of target vehicles, avoiding determining the sensitive area based on a preset confidence threshold, and thus avoiding the problem of missed detection of sensitive areas.

[0118] The above-mentioned embodiments of the present disclosure may be implemented separately or in any combination without conflict. The specific configuration may be based on actual needs and the present disclosure does not limit this.

[0119] Any method for desensitizing a target object in an image during vehicle travel provided by the embodiments of the present disclosure may be performed by any appropriate electronic device with data processing capabilities, including but not limited to: electronic devices such as terminal devices and servers. Alternatively, any method for desensitizing a target object in an image during vehicle travel provided by the embodiments of the present disclosure may be performed by a processor, such as the processor executing any method for desensitizing a target object in an image during vehicle travel mentioned in the embodiments of the present disclosure by calling corresponding instructions stored in a memory. This will not be described in detail below.

[0120] Exemplary Devices

[0121] Fig. 9 is a schematic diagram of a structure of a device for desensitizing a target object in an image during vehicle driving provided by an exemplary embodiment of the present disclosure. The device of this embodiment can be used to implement the corresponding method embodiments of the present disclosure, such as Fig. 9 The device shown may include: a first determining module 51 , a first processing module 52 , a second processing module 53 and a third processing module 54 .

[0122] The first determination module 51 is used to determine the position area of ​​the target object in the current environment image during the vehicle driving process.

[0123] The first processing module 52 is used to determine a preset type area in the target object in the current environment image based on the position area.

[0124] The second processing module 53 is used to determine the detection result in the preset type area.

[0125] The third processing module 54 is used to perform desensitization processing on the preset type area in the current environment image based on the detection result, so as to obtain a desensitized image corresponding to the current environment image.

[0126] Fig.10 It is a structural schematic diagram of an apparatus for desensitizing a target object in an image during vehicle driving, provided by another exemplary embodiment of the present disclosure.

[0127] In some optional embodiments, in the above Fig. 9 Based on the embodiment shown, Fig.10 As shown, the first processing module 52 may include: a first processing unit 521 and a second processing unit 522 .

[0128] The first processing unit 521 is used to perform detection processing on the current environment image to obtain a candidate area in the current environment image.

[0129] The second processing unit 522 is configured to determine a preset type area in the target object from the candidate areas based on the location area.

[0130] Fig.11 It is a schematic structural diagram of a device for desensitizing a target object in an image during vehicle driving, provided by yet another exemplary embodiment of the present disclosure.

[0131] In some optional embodiments, based on any of the above embodiments, Fig.11 As shown, the second processing module 53 may include:

[0132] The third processing unit 531 is used to perform detection processing on the current environment image through a preconfigured detection model to obtain the confidence corresponding to each candidate area belonging to a preset type of area in the current environment image.

[0133] The fourth processing unit 532 is used to determine the confidence level corresponding to each preset type of region in the target object based on the confidence levels corresponding to the candidate regions.

[0134] The fifth processing unit 533 is used to determine the attribute of each preset type of region in the target object according to the confidence level corresponding to each preset type of region in the target object.

[0135] The sixth processing unit 534 is used to determine the detection result according to the attribute of each preset type area.

[0136] Fig.12 It is a structural schematic diagram of a device for desensitizing a target object in an image during vehicle driving, provided by yet another exemplary embodiment of the present disclosure.

[0137] In some optional embodiments, in the above Fig. 9 Based on the embodiment shown, Fig.12 As shown, the first processing module 52 may include: a first determining unit 52a and a second determining unit 52b.

[0138] The first determining unit 52a is used to determine the target object area from the current environment image according to the position area.

[0139] The second determining unit 52b is used to determine a preset type area in the target object in the current environment image according to the target object area.

[0140] In some optional embodiments, such as Fig.12 As shown, the second processing module 53 may include: a third determining unit 53a, a fourth determining unit 53b and a fifth determining unit 53c.

[0141] The third determining unit 53a is used to determine the confidence level corresponding to each preset type of area in the target object area.

[0142] The fourth determining unit 53b is used to determine the state of each preset type area belonging to the to-be-desensitized area according to the confidence level corresponding to each preset type area in the target object.

[0143] The fifth determination unit 53c is used to determine the detection result according to the state of each preset type area belonging to the area to be desensitized.

[0144] Fig.13 It is a schematic structural diagram of a device for desensitizing a target object in an image during vehicle driving, provided by yet another exemplary embodiment of the present disclosure.

[0145] In some optional embodiments, in the above Fig. 9 Based on the embodiment shown, Fig.13 As shown, the first processing module 52 may include: a sixth determining unit 52c, a seventh determining unit 52d, and an eighth determining unit 52e.

[0146] The sixth determining unit 52c is used to determine a preset type area from the current environment image.

[0147] The seventh determining unit 52d is used to determine the confidence level corresponding to each preset type of area.

[0148] The eighth determining unit 52e is used to determine the preset type area in the target object in the current environment image based on the number of position areas and the confidence level corresponding to each preset type area.

[0149] In some optional embodiments, the first determination module 51 is specifically used to: obtain a position area of ​​the target object in the current environment image from a driving assistance system on the vehicle; the driving assistance system determines the position area of ​​the target object in the current environment image based on the current environment image and sensor data collected by at least one preset sensor; the type of the preset sensor is different from the type of sensor for collecting the current environment image.

[0150] Fig.14 It is a structural schematic diagram of a device for desensitizing a target object in an image during vehicle driving, provided by yet another exemplary embodiment of the present disclosure.

[0151] In some optional embodiments, based on any of the above embodiments, the third processing module 54 may include: a ninth determining unit 541 and a seventh processing unit 542 .

[0152] The ninth determining unit 541 is configured to determine, based on the detection result, a target preset type region whose attribute is a first preset attribute from the preset type regions.

[0153] The seventh processing unit 542 is used to perform desensitization processing on the target preset type area in the current environment image to obtain a desensitized image corresponding to the current environment image.

[0154] The beneficial technical effects corresponding to the exemplary embodiment of the present device can be found in the corresponding beneficial technical effects of the above exemplary method section, which will not be repeated here.

[0155] Exemplary Electronic Devices

[0156] Fig.15 It is a structural diagram of an electronic device provided by an embodiment of the present disclosure, including at least one processor 91 and a memory 92.

[0157] The processor 91 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 90 to perform desired functions.

[0158] The memory 92 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 91 may execute one or more computer program instructions to implement the methods and / or other desired functions of the various embodiments of the present disclosure described above.

[0159] In one example, the electronic device 90 may further include: an input device 93 and an output device 94 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0160] The input device 93 may also include, for example, a touch screen, a microphone, various sensors, and the like. Sensors may include, for example, image sensors (such as cameras, cameras, etc.), laser radars, millimeter-wave radars, ultrasonic radars, positioning sensors, pressure sensors, air quality sensors, temperature sensors, and the like. Image sensors, laser radars, millimeter-wave radars, ultrasonic radars, and the like may be used to sense the surrounding environment, that is, to detect dynamic and static objects in the surrounding environment. Dynamic and static objects may include, for example, static objects such as lane lines, curbs, arrows, signs, trees, and buildings, as well as dynamic objects such as surrounding vehicles, pedestrians, and cyclists. Positioning sensors are used to realize the positioning of movable devices (such as self-vehicles, robots, etc.) where electronic devices are located. Positioning sensors may include, for example, inertial measurement units (Inertial Measurement Unit, referred to as: IMU), global positioning systems (Global Positioning System, referred to as: GPS), and the like. Pressure sensors may be used to detect seat pressure. Temperature sensors may be used to detect the temperature in the vehicle cabin. Air quality sensors may be used to detect the air quality in the vehicle cabin.

[0161] The output device 94 can output various information to the outside, and may include, for example, a display, a speaker, a communication network and a remote output device connected thereto, and the like.

[0162] Of course, to simplify, Fig.15 Only some of the components related to the present disclosure in the electronic device 90 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device 90 may also include any other appropriate components.

[0163] Exemplary computer program products and computer-readable storage media

[0164] In addition to the above methods and devices, embodiments of the present disclosure may also provide a computer program product, including computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the methods of various embodiments of the present disclosure described in the above “Exemplary Methods” section.

[0165] The computer program product may be written in any combination of one or more programming languages ​​to write program code for performing the operations of the disclosed embodiments, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0166] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium having computer program instructions stored thereon. When the computer program instructions are executed by a processor, the processor executes the steps in the method of various embodiments of the present disclosure described in the above “Exemplary Method” section.

[0167] Computer readable storage media can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium is, for example, but not limited to, a system, device or device including electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0168] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and cannot be considered as necessary for each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, rather than limitation, and the above details do not limit the present disclosure to being implemented by adopting the above specific details.

[0169] Those skilled in the art may make various changes and modifications to the present disclosure without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present disclosure claims and their equivalents, the present disclosure is also intended to include these modifications and variations.

Claims

1. A method for desensitizing a target object in an image during vehicle driving, comprising: During the driving process of the vehicle, determining the location area of ​​the target object in the current environment image; Based on the location area, determining a preset type area in the target object in the current environment image; Determine the detection result in the preset type area; Based on the detection result, the preset type area in the current environment image is desensitized to obtain a desensitized image corresponding to the current environment image.

2. The method according to claim 1, wherein: The determining, based on the position area, a preset type area in the target object in the current environment image includes: Performing detection processing on the current environment image to obtain a candidate area in the current environment image; Based on the location area, a preset type area in the target object is determined from the candidate areas.

3. The method according to claim 1, wherein: Determining the detection result in the preset type area includes: Performing detection processing on the current environment image through a preconfigured detection model to obtain confidences corresponding to respective candidate areas of a preset type in the current environment image; Determining the confidence level corresponding to each of the preset type areas in the target object based on the confidence levels corresponding to the candidate areas; Determining the attributes of each of the preset types of regions in the target object according to the confidence corresponding to each of the preset types of regions; The detection result is determined according to the attribute of each of the preset type areas.

4. The method according to claim 1, wherein: The determining, based on the position area, a preset type area in the target object in the current environment image includes: Determining a target object area from the current environment image according to the position area; According to the target object area, a preset type area in the target object in the current environment image is determined.

5. The method according to claim 1, wherein: Determining the detection result in the preset type area includes: Determine the confidence level corresponding to each preset type of area in the target object area; Determining, according to the confidence level corresponding to each preset type area in the target object, whether each preset type area belongs to the area to be desensitized; The detection result is determined according to the status of each of the preset type areas belonging to the area to be desensitized.

6. The method according to claim 1, wherein: The determining, based on the position area, a preset type area in the target object in the current environment image includes: Determine a preset type area from the current environment image; Determine the confidence level corresponding to each preset type of area; Based on the number of the position areas and the confidence level corresponding to each preset type area, a preset type area in the target object in the current environment image is determined.

7. The method according to any one of claims 1 to 6, wherein: The determining of a location area of ​​a target object in a current environment image includes: The location area of ​​the target object in the current environment image is obtained from the driving assistance system on the vehicle; the driving assistance system determines the location area of ​​the target object in the current environment image based on the current environment image and sensor data collected by at least one preset sensor; the type of the preset sensor is different from the type of sensor for collecting the current environment image.

8. The method according to any one of claims 1 to 6, wherein: The method of performing desensitization processing on the preset type area in the current environment image based on the detection result to obtain a desensitized image corresponding to the current environment image includes: Based on the detection result, determining a target preset type area whose attribute is a first preset attribute from the preset type areas; Perform desensitization processing on the target preset type area in the current environment image to obtain a desensitized image corresponding to the current environment image.

9. A device for desensitizing a target object in an image while a vehicle is traveling, comprising: A first determination module is used to determine a location area of ​​a target object in a current environment image during vehicle driving; A first processing module, configured to determine a preset type area in the target object in the current environment image based on the position area; A second processing module, used to determine the detection result in the preset type area; The third processing module is used to perform desensitization processing on the preset type area in the current environment image based on the detection result to obtain a desensitized image corresponding to the current environment image.

10. A computer-readable storage medium storing a computer program, wherein the computer program is used to execute the method according to any one of claims 1 to 8.

11. An electronic device, comprising: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the instructions to implement the method described in any one of claims 1-8.

Images