Image Security Processing Method and Apparatus, Electronic Device, and Storage Medium

Abstract

Embodiments of the present disclosure disclose an image security processing method and apparatus, an electronic device, and a storage medium. The method includes: determining a geographical location when an image sensor acquires an image to be processed; determining a geographical security level corresponding to the geographical location; determining a target object in the image to be processed based on the geographical security level; and performing privacy protection processing on an image area where the target object is located. Embodiments of the present disclosure can perform privacy protection processing on corresponding target objects based on security protection requirements of different geographical locations to achieve corresponding security protection; at the same time, it can avoid consuming computing resources by performing privacy protection processing on unnecessary objects.

Description

Technical Field

[0001] The present disclosure relates to image processing technologies, and in particular, to an image security processing method and apparatus, an electronic device, and a storage medium. Background Art

[0002] Image security is one of the important fields of information security. Image acquisition using image acquisition devices is widely applied in all aspects of society, such as driving, mobile phones, monitoring, the Internet of Things, and so on. Since the acquired images may involve personal privacy and national security, it is often necessary to perform image security processing on the acquired images, such as blurring faces, blurring license plates, and blurring sensitive objects related to national security, and so on.

[0003] Existing image security processing methods usually perform unified security processing on all acquired images. Summary of the Invention

[0004] To solve the above technical problems, the present disclosure is proposed. Embodiments of the present disclosure provide an image security processing method and apparatus, an electronic device, and a storage medium.

[0005] According to one aspect of the embodiments of the present disclosure, an image security processing method is provided, including:

[0006] Determining the geographical location when an image sensor acquires an image to be processed;

[0007] Determining the geographical security level corresponding to the geographical location;

[0008] Based on the geographical security level, determining the target object in the image to be processed;

[0009] Performing privacy security protection processing on the image area where the target object is located.

[0010] According to another aspect of the embodiments of the present disclosure, an image security processing apparatus is provided, including

[0011] A first determination module, configured to determine the geographical location when an image sensor acquires an image to be processed;

[0012] A second determination module, configured to determine the geographical security level corresponding to the geographical location;

[0013] A third determination module, configured to determine the target object in the image to be processed based on the geographical security level;

[0014] A processing module, configured to perform privacy protection processing on the image area where the target object is located.

[0015] According to another aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium storing a computer program for executing the image security processing method according to any one of the above embodiments of the present disclosure.

[0016] According to still another aspect of the embodiments of the present disclosure, there is provided an electronic device, which includes:

[0017] A processor;

[0018] A memory for storing executable instructions of the processor;

[0019] The processor is configured to execute the image security processing method according to any one of the above embodiments of the present disclosure.

[0020] Based on the image security processing method, device, electronic device, and storage medium provided in the above embodiments of the present disclosure, when performing security processing on the image to be processed, by determining the target object in the image to be processed based on the geographical security level corresponding to the geographical location when the image sensor collects the image to be processed, and performing privacy protection processing on the image area where the target object is located, it realizes privacy protection processing on the corresponding target object based on the security protection requirements of different geographical locations, and achieves the corresponding security protection; at the same time, it can avoid consuming computing resources by performing privacy protection processing on unnecessary objects.

[0021] The technical solution of the present disclosure will be further described in detail below with reference to the drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] By describing the embodiments of the present disclosure in more detail in conjunction with the drawings, the above and other objects, features, and advantages of the present disclosure will become more obvious. The drawings are used to provide a further understanding of the embodiments of the present disclosure, and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure, and do not constitute a limitation to the present disclosure. In the drawings, the same reference numerals generally represent the same components or steps.

[0023] Figure 1 It is a flowchart of an image security processing method provided by an exemplary embodiment of the present disclosure.

[0024] Figure 2 It is a flowchart of an image security processing method provided by another exemplary embodiment of the present disclosure.

[0025] Figure 3 It is a flowchart of an image security processing method provided by still another exemplary embodiment of the present disclosure.

[0026] Figure 4 It is a flowchart of an image security processing method provided by still another exemplary embodiment of the present disclosure.

[0027] Figure 5 It is a schematic flowchart of an image security processing method provided by another exemplary embodiment of the present disclosure.

[0028] Figure 6 It is a schematic structural diagram of an embodiment of an image security processing device of the present disclosure.

[0029] Figure 7 It is a schematic structural diagram of another embodiment of an image security processing device of the present disclosure.

[0030] Figure 8 It is a schematic structural diagram of yet another embodiment of an image security processing device of the present disclosure

[0031] Figure 9 It is a structural diagram of an electronic device provided by an exemplary embodiment of the present disclosure. Detailed implementation manners

[0032] Next, exemplary embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all of the embodiments of the present disclosure. It should be understood that the present disclosure is not limited by the exemplary embodiments described herein.

[0033] It should be noted that: Unless otherwise specifically stated, the relative arrangements, numerical expressions, and numerical values of the components and steps set forth in these embodiments do not limit the scope of the present disclosure.

[0034] Those skilled in the art can understand that terms such as "first", "second", etc. in the embodiments of the present disclosure are only used to distinguish different steps, devices, or modules, etc., and neither represent any specific technical meaning nor indicate an inevitable logical order between them.

[0035] It should also be understood that in the embodiments of the present disclosure, "a plurality" may refer to two or more, and "at least one" may refer to one, two, or more.

[0036] It should also be understood that for any component, data, or structure mentioned in the embodiments of the present disclosure, unless otherwise clearly defined or given a contrary indication in the context, it is generally understood to be one or more.

[0037] In addition, the term "and / or" in the present disclosure is only a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in the present disclosure generally represents an "or" relationship between the associated objects before and after.

[0038] It should also be understood that the descriptions of the various embodiments in the present disclosure emphasize the differences between the various embodiments, and the same or similar parts can be referred to each other. For the sake of brevity, they will not be elaborated one by one.

[0039] Meanwhile, it should be understood that, for the sake of description convenience, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0040] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way a limitation on the present disclosure, its application, or its use.

[0041] Technologies, methods, and devices known to those of ordinary skill in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and devices should be regarded as part of the specification.

[0042] It should be noted that like reference numerals and letters indicate like items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further discussed in subsequent drawings.

[0043] The embodiments of the present disclosure can be applied to any first electronic device with a camera function, such as an imaging device like a surveillance camera, or can also be applied to a second electronic device such as a terminal device, a computer system, a server, etc. that is connected and communicates with the first electronic device with a camera function. The first electronic device with a camera function sends the original image data collected by the image sensor and the geographical location when the original image data is collected to the second electronic device. After the second electronic device performs security processing, it returns to the first electronic device. The second electronic device can operate with many other general-purpose or special-purpose computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments, and / or configurations suitable for use with second electronic devices such as terminal devices, computer systems, servers, etc. include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers, small computer systems, large computer systems, and distributed cloud computing technology environments including any of the above systems, and so on.

[0044] A first electronic device and a second electronic device such as a terminal device, a computer system, a server, etc. may be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. Generally, program modules may include routines, programs, target programs, components, logics, data structures, etc., which perform specific tasks or implement specific abstract data types. The computer system / server may be implemented in a distributed cloud computing environment where tasks are executed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules may be located on local or remote computing system storage media including storage devices.

[0045] Exemplary System

[0046] Embodiments of the present disclosure may be applicable to a vehicle monitoring system, which may include a camera device and a server. The camera device communicates with the server and transmits the captured monitoring images to the server for storage and other processing. The camera device is disposed at a preset position outside the vehicle, and the preset position is determined according to the scene information capable of capturing the geographical location where the vehicle is located.

[0047] In the embodiments of the present disclosure, after the image sensor of the camera device captures an image to be processed, it may first determine the geographical location when the image sensor captures the image to be processed, then determine the geographical security level corresponding to the geographical location, and based on the geographical security level, determine the target object in the image to be processed, and further perform privacy security protection processing on the image area where the target object is located.

[0048] In practical applications, the image security processing of the image to be processed may be implemented by adding an image security processing unit between the image sensor and the Image Signal Processing (ISP). The image security processing unit may be implemented by using a GPU (graphics processing unit) or an AI (Artificial Intelligence) chip; in addition, the image security processing of the image to be processed may also be implemented by the ISP, which may be specifically set according to actual requirements, and the present disclosure does not make any limitations. In some applications, the image security processing of the image to be processed may also be implemented by a second electronic device other than the camera device. By communicating with the camera device, the image to be processed captured by the image sensor of the camera device is transmitted to the second electronic device, and after the second electronic device performs image security processing, it is returned to the camera device.

[0049] Exemplary Method

[0050] Figure 1It is a schematic flowchart of an image security processing method provided by an exemplary embodiment of the present disclosure. This embodiment can be applied to an electronic device, such as Figure 1 as shown, and includes the following steps:

[0051] Step 101, determine the geographical location when the image sensor collects the image to be processed.

[0052] In the embodiment of the present disclosure, the image to be processed can be a scene image of the current geographical location collected by the image sensor. Exemplarily, the image to be processed can be a scene image of the geographical location where the vehicle is located collected by an external camera during vehicle assisted driving or autonomous driving. The scene image of the geographical location where the vehicle is located can include any object captured by the external camera, for example, license plates, faces, buildings, road facilities, etc., which are not limited in the embodiment of the present disclosure.

[0053] In the embodiment of the present disclosure, the image to be processed can be an unprocessed raw image (also called raw data) collected by the image sensor. The unprocessed raw image is specifically an unprocessed image in which the image sensor converts the captured light source signal into a digital signal. Exemplarily, the image to be processed can be an unprocessed raw scene image of the geographical location where the vehicle is located collected by an external camera during vehicle assisted driving or autonomous driving.

[0054] In the embodiment of the present disclosure, the geographical location when the image sensor collects the image to be processed can be determined by a positioning device and can be represented by longitude and latitude coordinates. The positioning device can be set in the device where the image sensor is located, or can also be set near the device where the image sensor is located to ensure that the geographical location determined by the positioning device is consistent with the geographical location when the image sensor collects the image to be processed.

[0055] In an optional example, the positioning device can be any one of positioning devices such as a Beidou positioning device, a GPS positioning device, etc., which are not limited in the embodiment of the present disclosure.

[0056] Step 102, determine the geographical security level corresponding to the geographical location.

[0057] In the embodiment of the present disclosure, the geographical security level can be used to identify the level of geographical information security requirements for the geographical location, and the geographical security level of the geographical location can be determined according to the geographical information security requirements of the geographical location. Specifically, for geographical locations with higher geographical information security requirements, a higher geographical security level can be determined; for geographical locations with lower geographical information security requirements, a lower geographical security level can be determined. It should be noted that geographical information security can specifically refer to the protection of data in hardware, software and their systems involved in links such as collection, processing, storage, processing, transmission, service and application of geographical information.

[0058] Exemplarily, during vehicle assisted driving or autonomous driving, if the geographical location of the image to be processed collected by the external camera of the vehicle is an undisclosed location, and the geographical information security requirements for the undisclosed location are high, the geographical security level of the geographical location can be determined to be a high level; if the geographical location of the image to be processed collected by the external camera of the vehicle is a semi-public location, and the geographical information security requirements for the semi-public location are medium, the geographical security level of the geographical location can be determined to be a medium level; if the geographical location of the image to be processed collected by the external camera of the vehicle is a public location, and the geographical information security requirements for the public location are low, the geographical security level of the geographical location can be determined to be a low level.

[0059] In an alternative example, the geographical security level may include more than two levels. For example, more than two levels may include: Geographical Security Level 1, Geographical Security Level 2, and Geographical Security Level 3. For another example, more than two levels may also include: Geographical Security Level A, Geographical Security Level B, and Geographical Security Level C. The embodiments of the present disclosure do not make any limitations.

[0060] In an alternative example, any one or more of symbols such as Roman numerals, lowercase Arabic numerals, uppercase Arabic numerals, Chinese numerals, lowercase English letters, and uppercase English letters can be used to name different geographical security levels to distinguish different geographical security levels. The embodiments of the present disclosure do not make any limitations on the naming method of geographical security levels.

[0061] Step 103, determine the target object in the image to be processed based on the geographical security level.

[0062] In the embodiments of the present disclosure, when the image sensor collects the image to be processed, it is easy to collect the image of an object that the public does not want to know about the current geographical location (which can also be referred to as a sensitive object). The target object in the image to be processed is the sensitive object that needs to be protected by privacy processing in the image to be processed.

[0063] In an alternative example, the sensitive object may include, but is not limited to, license plates, faces, buildings, and road facilities.

[0064] In the embodiments of the present disclosure, the types of sensitive objects corresponding to each geographical security level can be preset, and the types of sensitive objects corresponding to each geographical security level can be partially the same or completely different.

[0065] As an example, the type of sensitive object corresponding to Geographical Security Level 1 can be set as a building, the type of sensitive object corresponding to Geographical Security Level 2 can be set as a license plate, and the type of sensitive object corresponding to Geographical Security Level 3 can be set as a face.

[0066] As another example, the sensitive object types corresponding to the first level of geographical security can be set as buildings and road facilities, the sensitive object types corresponding to the second level of geographical security can be license plates and buildings, and the sensitive types corresponding to the third level of geographical security can be license plates and faces.

[0067] In the embodiments of the present disclosure, the levels of each geographical security level can be distinguished according to a preset rule. In an optional example, the preset rule can be that the smaller the Arabic numeral included in the name of the geographical security level, the higher the geographical security level; or the larger the Arabic numeral included in the name of the geographical security level, the higher the geographical security level; or the later the English letter included in the name of the geographical security level, the higher the geographical security level; or the earlier the English letter included in the name of the geographical security level, the higher the geographical security level. It should be noted that using the size of the Arabic numerals included in the name of the geographical security level, or the sorting of English letters can simply and effectively distinguish different geographical security levels, but it is not limited to this. Other methods, such as using the size of the Roman numerals included in the name of the geographical security level, the size of Chinese numerals, etc. are also applicable.

[0068] In an optional example, among each geographical security level, the higher the geographical security level, the more the number of sensitive object types it corresponds to, and the lower the geographical security level, the fewer the number of sensitive object types it corresponds to. It should be noted that the embodiments of the present disclosure do not limit the specific number of sensitive object types corresponding to each geographical security level.

[0069] As an example, each geographical security level can include three levels: Geographical Security Level A, Geographical Security Level B, and Geographical Security Level C. The above preset rule can be that the later the English letter included in the name of the geographical security level, the higher the geographical security level. Thus, among each geographical security level, Geographical Security Level A has the lowest level, and Geographical Security Level C has the highest level. It can be set that the sensitive object types corresponding to Geographical Security Level C can be license plates, faces, buildings, and road facilities, the sensitive object types corresponding to Geographical Security Level B are license plates, faces, and buildings, and the sensitive types corresponding to Geographical Security Level A are license plates and faces.

[0070] In an optional example, a higher geographical security level can correspond to more sensitive object types, and a lower geographical security level can correspond to fewer sensitive object types. Specifically, the higher and lower geographical security levels can be determined by means of a preset threshold. For example, a level higher than or equal to the preset threshold can be regarded as the higher geographical security level in the present disclosure, and a level lower than the preset threshold can be regarded as the lower geographical security level. It should be noted that the embodiments of the present disclosure do not limit the specific number of sensitive object types corresponding to each geographical security level.

[0071] As an example, each geographical security level may include four levels: Geographical Security Level A, Geographical Security Level B, Geographical Security Level C, and Geographical Security Level D. Among them, Geographical Security Level A has the lowest level, and Geographical Security Level D has the highest level. A preset threshold can be set as Level C, so that Geographical Security Level A and Geographical Security Level B can be determined as lower geographical security levels, and Geographical Security Level C and Geographical Security Level D can be determined as higher geographical security levels. Furthermore, the types of sensitive objects corresponding to the higher geographical security levels can be set as license plates, faces, buildings, and road facilities, and the types of sensitive objects corresponding to the lower geographical security levels can be set as license plates and faces.

[0072] In practical applications, according to the pre-set corresponding relationship between each geographical security level and the type of sensitive object, the type of sensitive object corresponding to the geographical security level determined in step 102 can be obtained, and the image object whose type identified from the image to be processed is the same as the type of sensitive object can be used as the target object in the image to be processed.

[0073] Step 104, perform privacy protection processing on the image area where the target object is located.

[0074] Specifically, since the target object that needs to be subjected to privacy protection processing in the image to be processed has been obtained in step 103, privacy protection processing can be performed on the image data within the image area where the target object is located. For example, privacy protection processing can be performed by means of image occlusion or image blurring to ensure the security of sensitive data in the image to be processed.

[0075] In the embodiment of the present disclosure, when performing security processing on the image to be processed, the target object in the image to be processed is determined based on the geographical security level corresponding to the geographical location when the image to be processed is collected by the image sensor, and privacy protection processing is performed on the image area where the target object is located, thereby realizing privacy protection processing on the corresponding target object based on the security protection requirements of different geographical locations and realizing the corresponding security protection; at the same time, it is possible to avoid consuming computing resources by performing privacy protection processing on unnecessary objects.

[0076] Figure 2 It is a schematic flowchart of an image security processing method provided by another exemplary embodiment of the present disclosure. As Figure 2 shown, on the basis of the embodiment shown above Figure 1 shown, step 102 may include the following steps:

[0077] Step 102-1, obtain the target geographical location that matches the geographical location from the preset geographical location matching information.

[0078] In the embodiments of the present disclosure, the preset geographical location matching information can be stored in the form of a database or a data table, etc. The preset geographical location matching information can include at least one preset geographical location and the location area range corresponding to each preset geographical location in the at least one preset geographical location.

[0079] In the embodiments of the present disclosure, the geographical location when the image sensor collects the image to be processed determined in step 101 above can be compared with each preset geographical location and the location area range corresponding to each preset geographical location in the preset geographical location matching information, so as to obtain the target geographical location matched with this geographical location.

[0080] In an optional example, the target geographical location matched with this geographical location can be any preset geographical location identical to this geographical location, or can also be the preset geographical location corresponding to any location area range containing this geographical location.

[0081] Step 102-2, obtain the target geographical security level corresponding to the target geographical location and the target object type corresponding to the target geographical security level from the preset geographical security matching information.

[0082] In the embodiments of the present disclosure, the preset geographical security matching information can be stored in the form of a database or a data table, etc. The preset geographical security matching information can include: the geographical security level corresponding to each preset geographical location and the object type corresponding to each geographical security level.

[0083] In an optional example, the object type in the preset geographical security matching information can include any one or more of the following: license plate, face, building, road facility.

[0084] In the embodiments of the present disclosure, the target geographical location determined in step 102-2 can be compared with each preset geographical location in the preset geographical security matching information, and the geographical security level corresponding to the preset geographical location identical to this target geographical location is obtained as the target geographical security level corresponding to this target geographical location.

[0085] The target geographical security level corresponding to the target geographical location is compared with each geographical security level in the preset geographical security matching information, and the object type corresponding to the geographical security level identical to this target geographical security level is obtained as the target object type corresponding to this target geographical security level.

[0086] In the embodiments of the present disclosure, when determining the geographical security level corresponding to a geographical location, by comparing the geographical location with the information in the preset geographical location matching information and the preset geographical security matching information set in advance, the geographical security level corresponding to the geographical location and the target object type corresponding to the geographical security level can be quickly found, which helps to reduce the time consumption of image security processing and improve the efficiency of image security processing.

[0087] Figure 3 It is a schematic flowchart of an image security processing method provided by another exemplary embodiment of the present disclosure. As Figure 3 shown, based on the embodiment shown above Figure 2 shown, step 103 may include the following steps:

[0088] Step 103-1a, perform semantic segmentation on the image to be processed to obtain at least one region of interest in the image to be processed.

[0089] Step 103-2a, perform semantic recognition on at least one region of interest to obtain at least one object of interest in the image to be processed.

[0090] In the embodiments of the present disclosure, a first pre-trained image semantic analysis model can be used to perform semantic segmentation and semantic recognition on the image to be processed. Semantic segmentation can obtain at least one region of interest in the image to be processed, and this region of interest is the image region where there may be an object of interest in the image to be processed. Semantic recognition can determine the type to which each pixel in the region of interest belongs. Based on each pixel in the region of interest and the type to which each pixel belongs, the object of interest (including the pixels belonging to the object of interest) and the type of the object of interest in the region of interest can be determined. The type of the object of interest may include, but is not limited to, license plates, faces, buildings, and road facilities.

[0091] In the embodiments of the present disclosure, the first pre-trained image semantic analysis model can be obtained through the following method:

[0092] Construct a first initial image semantic analysis model; obtain a plurality of image samples from the training sample set; use the plurality of image samples as inputs and provide them to the first initial image semantic analysis model. The first initial image semantic analysis model performs semantic segmentation and semantic recognition on each input image sample respectively. According to the output of the first initial image semantic analysis model, at least one object of interest of each predicted image sample is obtained; according to at least one object of interest of each predicted image sample and the object of interest annotation information of each image sample, the model parameters of the first initial image semantic analysis model are adjusted.

[0093] In an optional example, the first initial image semantic analysis model may be a convolutional neural network model or a fully convolutional neural network model.

[0094] Step 103-3a: Determine a target object from at least one object of interest based on the object type corresponding to the geographical security level.

[0095] After determining at least one object of interest in the image to be processed via Step 103-2a, compare the type of each object of interest with the object type corresponding to the geographical security level, and select the object of interest with the same type as the object type corresponding to the geographical security level from at least one object of interest as the target object.

[0096] In the embodiments of the present disclosure, through a pre-trained image semantic analysis model, at least one object of interest in the image to be processed can be quickly and accurately identified, so that the target object that needs privacy protection processing in the image to be processed can be quickly and accurately determined, which helps to improve the efficiency and accuracy of image security processing. At the same time, performing hierarchical security processing on the image to be processed according to the geographical security level helps to improve the flexibility of image security processing compared with the existing method of performing unified security processing on all collected images.

[0097] Figure 4 It is a schematic flowchart of an image security processing method provided by another exemplary embodiment of the present disclosure. As Figure 4 shown, on the basis of the embodiment shown above Figure 2 shown, Step 102-2 may include the following steps:

[0098] Step 102-2a: Obtain the target geographical security level corresponding to the target geographical location from the preset geographical security matching information.

[0099] In the embodiments of the present disclosure, the preset geographical security matching information may be stored in the form of a database or a data table, etc. The preset geographical security matching information may include: the geographical security levels corresponding to each preset geographical location, the object types corresponding to each geographical security level, and each object category included in the corresponding object type.

[0100] In an optional example, the object type in the geographical security matching information includes any one or more of the following: license plate, face, building, road facility.

[0101] In the embodiments of the present disclosure, an object category is a further subdivision of an object type, and an object type can be further subdivided into more than one object category. Exemplarily, the object type "license plate" can include three object categories: "military license plate", "civilian license plate", and "police license plate"; the object type "human face" can include two object categories: "eyes" and "face"; the object type "building" can include one object category: "building"; and the object type "road facility" can include two object categories: "traffic light" and "road sign". It should be noted that the subdivision method and the number of subdivisions of the object types in the embodiments of the present disclosure are not limited, and those skilled in the art can determine the subdivision method and the number of subdivisions of the object types according to actual needs.

[0102] In the embodiments of the present disclosure, the target geographical location determined in step 102-2 can be compared with each preset geographical location in the preset geographical security matching information, and the geographical security level corresponding to the preset geographical location identical to the target geographical location can be obtained as the target geographical security level corresponding to the target geographical location.

[0103] Step 102-2b: Obtain the target object type corresponding to the target geographical security level and each object category included in the target object type from the preset geographical security matching information.

[0104] Compare the target geographical security level corresponding to the target geographical location determined in step 102-2a with each geographical security level in the preset geographical security matching information, obtain the object type corresponding to the geographical security level identical to the target geographical security level as the target object type, and obtain each object category included in the target object type.

[0105] In the embodiments of the present disclosure, by further dividing the object type corresponding to the geographical security level into each object category, the discrimination accuracy of the target object that needs privacy protection processing in the image to be processed can be improved, which helps to improve the accuracy of image security processing.

[0106] Figure 5 is a schematic flowchart of an image security processing method provided by another exemplary embodiment of the present disclosure. As Figure 5 shown, on the basis of the embodiment shown above Figure 4 shown, step 103 may include the following steps:

[0107] Step 103-1b: Perform semantic segmentation on the image to be processed to obtain at least one region of interest in the image to be processed.

[0108] Step 103-2b: Perform semantic recognition on at least one region of interest to obtain at least one interesting object in the image to be processed.

[0109] In the embodiments of the present disclosure, a second pre-trained image semantic analysis model can be used to perform semantic segmentation and semantic recognition on the image to be processed. Semantic segmentation can obtain at least one region of interest in the image to be processed, where the region of interest is an image region in the image to be processed where an object of interest may exist. Semantic recognition can determine the category to which each pixel in the region of interest belongs. Based on each pixel in the region of interest and the category to which each pixel belongs, the object of interest (including the pixels belonging to the object of interest) and the category of the object of interest in the region of interest can be determined.

[0110] In the embodiments of the present disclosure, the category of the object of interest is a subdivision of the type of the above-mentioned object of interest, and the subdivision method of the type of the object in the region of interest is the same as the subdivision method of the above-mentioned object type.

[0111] In the embodiments of the present disclosure, the second pre-trained image semantic analysis model can be obtained in the following manner: constructing a second initial image semantic analysis model; obtaining a plurality of image samples from a training sample set; using the plurality of image samples as inputs respectively and providing them to the second initial image semantic analysis model. The second initial image semantic analysis model performs semantic segmentation and semantic recognition on each input image sample respectively, and based on the output of the second initial image semantic analysis model, at least one object of interest of each predicted image sample is obtained; the model parameters of the second initial image semantic analysis model are adjusted according to at least one object of interest of each predicted image sample and the object of interest annotation information of each image sample.

[0112] In an optional example, the second initial image semantic analysis model can be a convolutional neural network model or a fully convolutional neural network model.

[0113] Step 103-3b: Determine a target object from at least one object of interest based on the object type corresponding to the geographical security level and each object category included in the corresponding object type.

[0114] After determining at least one object of interest in the image to be processed via step 103-2b, compare the category of each object of interest with each object category included in the object type corresponding to the geographical security level, and select the object of interest whose category is the same as any one of the object categories from at least one object of interest as the target object.

[0115] In the embodiments of the present disclosure, by further dividing the object type corresponding to the geographical security level into each object category, the discrimination accuracy of the target object that needs privacy protection processing in the image to be processed can be improved, which helps to improve the accuracy of image security processing.

[0116] In an optional example, step 104 above may include: performing an occlusion process on the pixel points in the image area where the target object is located in the image to be processed according to the mask image corresponding to the image to be processed, so as to obtain target image data.

[0117] In the embodiments of the present disclosure, a mask image having the same size as the image to be processed is provided, and the image area that needs to be subjected to privacy protection processing is marked by the mask image.

[0118] In the embodiments of the present disclosure, in the mask image, a mark (for example, the value 1) is set for the image area where the target object that needs to be subjected to privacy protection processing in the corresponding image to be processed is located. In this way, when the subsequent steps perform privacy protection processing, it can be determined whether to perform an occlusion process on the pixel points in the corresponding image block of the image to be processed according to whether there is a mark in the image block of the mask image.

[0119] In this embodiment, the image area where the target object that needs to be subjected to privacy protection processing in the image to be processed can be marked by the mask image, and further, it is convenient for the subsequent steps to accurately perform privacy protection processing on the image to be processed, and the privacy data security in the image to be processed can be effectively protected.

[0120] Any one of the image security processing methods provided in the embodiments of the present disclosure can be executed by any suitable device with data processing capabilities, including but not limited to: terminal devices, servers, etc. Alternatively, any one of the image security processing methods provided in the embodiments of the present disclosure can be executed by a processor. For example, the processor executes any one of the image security processing methods mentioned in the embodiments of the present disclosure by calling the corresponding instructions stored in the memory. Details will not be described hereinafter.

[0121] Exemplary Device

[0122] Figure 6 It is a schematic structural diagram of an embodiment of an image security processing device of the present disclosure. The device in this embodiment can be used to implement the corresponding method embodiments of the present disclosure. As Figure 6 shown, the device includes: a first determination module 201, a second determination module 202, a third determination module 203, and a processing module 204.

[0123] The first determination module 201 is used to determine the geographical location when the image sensor collects the image to be processed.

[0124] The second determination module 202 is used to determine the geographical security level corresponding to the geographical location.

[0125] The third determination module 203 is used to determine the target object in the image to be processed based on the geographical security level.

[0126] The processing module 204 is used to perform privacy protection processing on the image area where the target object is located.

[0127] Figure 7 It is a schematic structural diagram of another embodiment of the image security processing device of the present disclosure. As Figure 7 shown, the second determination module 202 may include a first acquisition unit 202-1 and a second acquisition unit 202-2.

[0128] The first acquisition unit 202-1 is used to acquire the target geographical location that matches the geographical location from the preset geographical location matching information.

[0129] In an optional example, the preset geographical location matching information includes at least one preset geographical location and the location area range corresponding to each preset geographical location in the at least one preset geographical location.

[0130] The second acquisition unit 202-2 is used to acquire the target geographical security level corresponding to the target geographical location and the target object type corresponding to the target geographical security level from the preset geographical security matching information.

[0131] In an optional example, the preset geographical security matching information includes: the geographical security level corresponding to each preset geographical location and the object type corresponding to each geographical security level.

[0132] In an optional example, the object type in the geographical security matching information includes any one or more of the following: license plate, face, building, preset equipment.

[0133] Figure 8 It is a schematic structural diagram of still another embodiment of the image security processing device of the present disclosure. As Figure 8 shown, the third determination module 203 may include a first recognition unit 203-1, a second recognition unit 203-2, and a determination unit 203-3.

[0134] The first recognition unit 203-1 is used to perform semantic segmentation on the image to be processed to obtain at least one region of interest in the image to be processed.

[0135] The second recognition unit 203-2 is used to perform semantic recognition on at least one region of interest to obtain at least one interesting object in the image to be processed.

[0136] The determination unit 203-3 is used to determine the target object from at least one interesting object based on the object type corresponding to the geographical security level.

[0137] In an alternative example, the second acquisition unit 202-2 is further configured to obtain the target geographical security level corresponding to the target geographical location from the preset geographical security matching information; obtain the target object type corresponding to the target geographical security level and each object category included in the target object type from the preset geographical security matching information.

[0138] In an alternative example, the preset geographical security matching information includes: the geographical security levels corresponding to the preset geographical locations, the object types corresponding to the geographical security levels, and each object category included in the corresponding object types.

[0139] In an alternative example, the determination unit 203-3 is further configured to determine a target object from at least one object of interest based on the object type corresponding to the geographical security level and each object category included in the corresponding object type.

[0140] In an alternative example, the processing module 204 is specifically configured to perform occlusion processing on the pixel points at the corresponding positions in the image area of the target object in the image to be processed according to the mask image corresponding to the image to be processed, so as to obtain target image data.

[0141] Exemplary Electronic Device

[0142] Next, with reference to Figure 9 Let's describe the electronic device according to the embodiments of the present disclosure. As Figure 9 shown, the electronic device includes one or more processors 901 and a memory 902.

[0143] The processor 901 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

[0144] The memory 902 may include one or more computer program products, and the computer program products may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory, etc. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 901 may run the program instructions to implement the image security processing methods of the various embodiments of the present disclosure described above and / or other desired functions. Various contents such as input signals, signal components, and noise components may also be stored in the computer-readable storage medium.

[0145] In one example, the electronic device may further include: an input device 903 and an output device 904, and these components are interconnected through a bus system and / or other forms of connection mechanisms (not shown). The input device 903 may include, for example, a keyboard, a mouse, and so on. The output device 904 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, and so on.

[0146] Of course, for simplicity, Figure 9 only some of the components related to the present disclosure in the electronic device are shown, and components such as a bus, an input / output interface, and so on are omitted. In addition, according to specific application scenarios, the electronic device may further include any other appropriate components.

[0147] Exemplary Computer Program Product and Computer Readable Storage Medium

[0148] In addition to the above methods and devices, an embodiment of the present disclosure may also be a computer program product, which includes computer program instructions that, when run by a processor, cause the processor to execute the steps in the image security processing method according to various embodiments of the present disclosure described in the "Exemplary Method" section above of this specification.

[0149] The computer program product may be written in any combination of one or more programming languages for programming code to perform the operations of the embodiments of the present disclosure. The programming languages include object-oriented programming languages such as Java, C++, etc., and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's device, executed as an independent software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0150] Furthermore, an embodiment of the present disclosure may also be a computer-readable storage medium, on which computer program instructions are stored, and the computer program instructions, when run by a processor, cause the processor to execute the steps in the image security processing method according to various embodiments of the present disclosure described in the "Exemplary Method" section above of this specification.

[0151] The computer-readable storage medium may adopt any combination of one or more readable media. The readable media may be a readable signal medium or a readable storage medium. The readable storage medium may include, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0152] The basic principles of the present disclosure have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, benefits, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, benefits, effects, etc. are essential for each embodiment of the present disclosure. In addition, the above-mentioned specific details are only for illustrative and easy-to-understand purposes and not for limitation, and the above details do not limit the present disclosure to necessarily adopt the above specific details for implementation.

[0153] Each embodiment in this specification is described in a progressive manner. The key point of each embodiment is to illustrate the differences from other embodiments. For the same or similar parts among the embodiments, reference can be made to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and for the relevant parts, reference can be made to the description of the method embodiment.

[0154] The block diagrams of the devices, apparatuses, equipment, and systems involved in the present disclosure are only illustrative examples and do not intend to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open-ended words, meaning "including but not limited to", and can be used interchangeably with each other. The word "or" and "and" used herein refer to the word "and / or", and can be used interchangeably with each other, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to", and can be used interchangeably with each other.

[0155] The methods and apparatuses of the present disclosure may be implemented in many ways. For example, the methods and apparatuses of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of the steps for the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above unless otherwise specifically stated. In addition, in some embodiments, the present disclosure may also be implemented as a program recorded in a recording medium, and these programs include machine-readable instructions for implementing the method according to the present disclosure. Therefore, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.

[0156] It should also be noted that in the apparatuses, devices, and methods of the present disclosure, each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent solutions of the present disclosure.

[0157] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0158] The above description has been presented for purposes of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the form disclosed herein. Although several example aspects and embodiments have been discussed above, those skilled in the art will recognize some of their variations, modifications, alterations, additions, and subcombinations.

Claims

1. An image security processing method, comprising: Determining the geographical location when an image sensor acquires an image to be processed; Determining the geographical security level corresponding to the geographical location; Determining the target object in the image to be processed based on the geographical security level; Performing privacy protection processing on the image area where the target object is located; Wherein, determining the geographical security level corresponding to the geographical location includes: determining the geographical security level corresponding to the geographical location and the type of target object corresponding to the geographical security level; Determining the target object in the image to be processed based on the geographical security level includes: determining, as the target object, the object of interest with the same type as the target object type among at least one object of interest identified from the image to be processed.

2. The method according to claim 1, wherein Determining the geographical security level corresponding to the geographical location includes: Obtaining the target geographical location matched with the geographical location from the preset geographical location matching information; wherein, the preset geographical location matching information includes at least one preset geographical location and the position area range corresponding to each preset geographical location in the at least one preset geographical location; Obtaining the target geographical security level corresponding to the target geographical location and the type of target object corresponding to the target geographical security level from the preset geographical security matching information; wherein, the preset geographical security matching information includes: the geographical security level corresponding to each preset geographical location, and the object type corresponding to each geographical security level.

3. The method according to claim 2, wherein The object type in the geographical security matching information includes any one or more of the following: license plate, face, building, and road facilities.

4. The method according to claim 2, wherein, Determining the target object in the image to be processed based on the geographical security level includes: Performing semantic segmentation on the image to be processed to obtain at least one region of interest in the image to be processed; Performing semantic recognition on the at least one region of interest to obtain at least one object of interest in the image to be processed; Determining the target object from the at least one object of interest based on the object type corresponding to the geographical security level.

5. The method according to claim 2, wherein Obtaining the target geographical security level corresponding to the target geographical location and the type of target object corresponding to the target geographical security level from the preset geographical security matching information includes: Obtaining the target geographical security level corresponding to the target geographical location from the preset geographical security matching information; Obtaining the type of target object corresponding to the target geographical security level and each object category included in the type of target object from the preset geographical security matching information; wherein, the preset geographical security matching information includes: the geographical security level corresponding to each preset geographical location, the object type corresponding to each geographical security level, and each object category included in the corresponding object type.

6. The method according to claim 5, wherein, Determining the target object in the image to be processed based on the geographical security level includes: Performing semantic segmentation on the image to be processed to obtain at least one region of interest in the image to be processed; Performing semantic recognition on the at least one region of interest to obtain at least one object of interest in the image to be processed; Determine a target object from the at least one object of interest based on the object type corresponding to the geographic security level and each object category included in the corresponding object type.

7. The method according to any one of claims 1-6, wherein, Performing privacy protection processing on the image area where the target object is located includes: According to the mask image corresponding to the image to be processed, perform occlusion processing on the pixel points in the image area where the target object is located in the image to be processed to obtain target image data.

8. An image security processing device, comprising: A first determination module, configured to determine the geographical location when an image sensor acquires an image to be processed; A second determination module, configured to determine the geographical security level corresponding to the geographical location; A third determination module, configured to determine a target object in the image to be processed based on the geographical security level; A processing module, configured to perform privacy protection processing on the image area where the target object is located; Wherein, determining the geographical security level corresponding to the geographical location is further configured to: determine the geographical security level corresponding to the geographical location and the target object type corresponding to the geographical security level; Based on the geographical security level to determine the target object in the image to be processed, is further configured to: determine the object of interest with the same type as the target object type from at least one object of interest identified from the image to be processed as the target object.

9. A computer-readable storage medium, the storage medium stores a computer program, and the computer program is used to execute the image security processing method according to any one of the above claims 1-7.

10. An electronic device, the electronic device includes: A processor; A memory for storing executable instructions of the processor; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the image security processing method according to any one of the above claims 1-7.

Images