METHOD AND DEVICE FOR PROCESSING IMAGE DATA

Abstract

A method, a device, and a computer-readable medium for processing image data are provided. The method includes converting raw image data into visible images, identifying specific areas of the visible images containing sensitive information, identifying specific raw image data within the raw image data corresponding to these areas, and masking this specific raw image data within the raw image data. This increases processing speed while simultaneously reducing data loss.

Description

FIELD OF THE INVENTION

[0001] The present invention relates to the field of data security and in particular to the processing of image data. STATE OF THE ART

[0002] With the development of big data technology, the integration and sharing of large amounts of data are being promoted, and the value of this data is being enhanced through its exploitation. However, this inevitably raises concerns about data security. Large amounts of data contain sensitive data such as private user information, and the theft of this sensitive data can have negative consequences for users. Data security has therefore become a critical factor in the development of the big data industry.

[0003] Data masking technology is currently being used to redesign sensitive data in large data sets to prevent negative consequences from data theft. However, there is still room for improvement in current masking technology when it comes to image data. DISCLOSURE OF THE INVENTION

[0004] It is desirable to provide a method and apparatus for processing image data that can not only increase the processing speed but also reduce data loss.

[0005] According to one aspect, a method for processing image data is provided, comprising: converting raw image data into visible images, determining specific regions of the visible images that contain sensitive information, determining specific raw image data in the raw image data that correspond to the specific regions, and masking the specific raw image data in the raw image data.

[0006] According to another aspect, there is provided an apparatus for processing image data, comprising: a conversion unit that converts raw image data into visible images; a positioning unit that determines specific areas of the visible images that contain sensitive information; and a processing unit that determines specific raw image data in the raw image data corresponding to the specific areas and masks the specific raw image data in the raw image data.

[0007] According to another aspect, an electronic device is provided, comprising: a memory having at least one computer-executable instruction stored thereon; and one or more processors that, upon execution of the at least one computer-executable instruction, cause the following: converting raw image data into visible images; determining specific regions of the visible images that contain sensitive information; determining specific raw image data in the raw image data that correspond to the specific regions; and masking the specific raw image data in the raw image data.

[0008] According to a further aspect, a computer-readable medium is provided on which a computer program is stored, wherein the execution of the computer program by a computer or processor causes the computer or processor to carry out a method according to the individual embodiments of the present invention.

[0009] According to the embodiments of the individual aspects of the present invention, the raw image data itself is masked directly, rather than the visible images, thus eliminating the need for the step of converting the masked visible images into raw image data. This reduces the likelihood of data loss. Data loss is minimized as much as possible while simultaneously increasing the processing speed of the masking. DESCRIPTION OF THE CHARACTERS

[0010] In the figures, the embodiments are explained only by way of example and not by way of limitation, and in the figures, like reference numerals refer to like elements. Fig. 1 shows a current art method for processing image data; Fig. 2 shows a flowchart of a method for processing image data according to an embodiment of the present invention; Fig. 3 shows a scenario of processing raw image data using a method for processing image data according to an embodiment of the present invention; Fig. 4 shows a block diagram of an apparatus for processing image data according to an embodiment of the present invention; Fig. 5 shows a block diagram of an electronic device that executes a method for processing image data according to an embodiment of the present invention.

[0011] The individual aspects and features of the individual embodiments of the present invention are described with reference to the figures mentioned. These figures are merely schematic and not limiting. Furthermore, not all individual parts of the devices of the embodiments of the invention are identified by reference numerals in the figures mentioned, and in some figures, only relevant components are identified, which does not limit the individual components to the appearances shown in the figures for description. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0012] For example, in the field of intelligent transportation, large amounts of image data must be collected. This image data may contain sensitive data such as vehicle license plates and users' faces, and this sensitive data must be processed to prevent misuse. Although intelligent transportation was chosen as an example, it is not limited to this, and masking image data can be widely applied in various fields, including medicine.

[0013] Fig. 1 shows a flowchart of a method 100 according to current technology for processing image data. According to the flow of this method, sensitive data in the raw image data is masked, thereby removing the sensitive data from the raw image data. Masking means modifying data for specific sensitive information based on masking rules, thus ensuring reliable protection of sensitive private data. When user security data or commercially sensitive data is involved, the actual data must be modified for later use without violating the system's rules. For example, data masking of personal information such as ID numbers, mobile phone numbers, bank card numbers, customer numbers, user faces, vehicle license plates, etc. must be performed.Masking can be carried out using a variety of data processing methods, although the methods for implementing masking are not restricted here.

[0014] According to this method 100, raw image data is acquired in step 110. Here, raw image data refers to raw data acquired by image capture devices such as cameras, scanners, etc., that has not yet been processed to generate visible images, particularly raw data that has not yet been processed in any way. Raw image data is typically binary data. For example, the data structure of raw video image data typically includes file headers, camera sensor metadata, image metadata, thumbnail images, image sequences, sensor image data, and JPEG (Joint Photographic Experts Group) conversions of any images. After said raw image data has been acquired by image capture devices such as cameras, this raw image data can be processed to generate visible images.

[0015] In step 120, the acquired raw image data is processed to generate visible images. In step 130, the sensitive areas in the visible images are determined, i.e., specific areas in the visible images that contain sensitive information, for example, areas showing vehicle license plates, are identified. In step 140, the sensitive areas determined in the visible images are masked. In step 150, the masked visible images are converted back into raw image data, i.e., the masked visible images are converted back into the binary format of the raw data as acquired by the image acquisition device. In step 160 following step 150, masked raw image data can be obtained for transmission and / or storage.

[0016] In the Fig. In the raw image data masking process shown in Figure 1, the raw image data must first be converted into visible images, then sensitive areas are identified and processed based on these visible images to achieve the masking goal. After masking the visible images, the masked visible images must also be converted back into raw image data. The raw image data is converted into visible images, and then the masked visible images are converted back into the raw image data format. Multiple conversions will inevitably result in data loss and affect the masking processing speed.

[0017] According to the individual embodiments of the present invention, the raw image data itself is masked directly, rather than the visible images, thus eliminating the need for the step of converting the masked visible images into raw image data. This reduces the probability of data loss, minimizes data loss, and simultaneously increases the processing speed of the masking.

[0018] Fig. 2 shows a flowchart of a method 200 for processing image data according to an embodiment of the present invention. As in Fig. 2, according to this method 200, raw image data is acquired in step 210, which corresponds to step 110 in the Fig. 1. The acquired raw image data includes two-dimensional image data or three-dimensional image data.

[0019] In step 220, the acquired raw image data is processed to generate visible images. This means that the raw image data is converted into visible images. Visible images of any format are conceivable. There are no restrictions regarding the type of visible images generated.

[0020] In step 230, specific areas of the visible images are determined that contain sensitive information. Sensitive information can be any type of information that concerns personal privacy or is commercially sensitive, such as faces, bank cards, ID numbers, etc.

[0021] Determining specific areas of the visible images that contain sensitive information can be done using any currently available technology. These include, but are not limited to, image processing technology, methods based on user input, and machine learning techniques. In one embodiment, this step identifies and determines the coordinate positions of the specific areas of the visible images that contain sensitive information. Various types of coordinates can be used to indicate the coordinate positions of these specific areas, but are not limited to this. In another embodiment, this step can display the visible images to the user, and then, based on the user input, the coordinate positions of the specific areas of the visible images that contain sensitive information can be determined.Of course, it is desirable that the specific areas of the visible images that contain sensitive information can be determined automatically, without requiring manual intervention. For example, based on machine learning techniques, the visible images can be input into a trained machine learning model to output the coordinate positions of the specific areas of these visible images that contain sensitive information.

[0022] After the specific areas of the visible images containing sensitive information have been determined, in step 240, specific raw image data in the raw image data are determined that correspond to the specific areas containing sensitive information, and this specific raw image data in the raw image data is immediately masked.

[0023] By mapping the coordinate positions corresponding to the specific areas of the visible images and the raw image data, the specific raw image data in the raw image data that has been determined to contain sensitive information can be determined. Specifically, in the steps described above, the coordinate positions of the specific areas of the visible images that contain sensitive information can be acquired first. Then, based on the mapping relationships between the coordinate positions among the visible images and the raw image data, the coordinate positions of these specific areas in the visible images can be mapped to the raw image data, thereby obtaining specific raw image data corresponding to the specific areas that contain sensitive information.These mapping relationships may be determined in advance and may vary depending on the type of raw image data and / or the type of image acquisition device used to acquire the raw image data.

[0024] After the specific raw image data containing sensitive information has been determined, it is sufficient to mask this specific raw image data to prevent disclosure of this sensitive information. After masking in step 240, the masked raw image data is obtained in step 250 for storage and / or output.

[0025] It is evident that, in contrast to the Fig. 1, in the method 200 of this embodiment according to the present invention, after determining the specific areas of the visible images that contain sensitive information, in step 240 these specific areas are again assigned to the raw image data in order to find the corresponding specific raw image data, and then the specific raw image data and not the sensitive areas in the visible images are immediately masked. Fig. 1, in which the masked visible images are converted back into raw image data, thus reducing the probability of data loss and at the same time saving processing steps, thereby increasing the processing speed of the masking.

[0026] Fig. 3 shows a scenario of processing raw image data using a method for processing image data according to an embodiment of the present invention. As in Fig. As shown in Figure 3, in step 210, the raw image data a is acquired, in step 220, the raw image data a is converted into visible images b, and in step 230, based on the visible images b, the specific regions contained therein that contain sensitive information are determined. Here, image c illustrates sensitive information such as people's faces and a vehicle license plate. Subsequently, in step 240, the specific regions in the visible images are mapped to the specific raw image data, and the specific raw image data are masked, so that masked raw image data d is obtained in step 250.

[0027] Although with reference to Fig. 2 and Fig. 3 a method of an embodiment according to the present invention has been described, it is understood that one or more steps may vary depending on the different scenario.

[0028] In one embodiment, the raw image data comprises video image data. In such a case, in step 220, the raw video image data may be converted into visible multiple images. It is desirable that the specific regions containing sensitive information be identified in parallel in all visible images among the visible multiple images, the specific raw image data corresponding to the specific regions of all visible images be determined in parallel in the raw image data, and finally, all specific raw image data of the individual visible images among the visible multiple images in the raw image data be masked together. Positioning and assigning sensitive regions in parallel can save processing time and increase processing speed.

[0029] Of course, this does not preclude the serial execution of one or more steps of the individual embodiments of the present invention. For example, after converting the raw video image data into multiple visible images in step 220, the operations of steps 230 to 240 can be executed individually in sequence for the visible images, and finally, after the partial specific raw image data has been masked with sensitive information corresponding to the respective visible images, the final masked raw image data can be obtained in step 250. In practice, the individual steps of the method of the embodiments of the present invention can be executed serially or in parallel for different types of raw image data, as needed.

[0030] Although it was described above that processing is performed per image as a unit, this is also not limiting. The raw image data can also be arbitrarily divided into different data blocks according to other requirements, whereupon the method according to the embodiments of the present invention is carried out with data blocks as a unit.

[0031] Furthermore, to increase processing speed, if steps 210 to 250 are considered a processing chain, multiple such processing chains can be established, each processing different raw image data (or data blocks). The number of processing chains is limited by the processing capacity of the device.

[0032] In the field of Big Data, data security has always been an important focus point. In one embodiment, with regard to the security of the image data, it is desirable to mask the raw image data after the raw image data has been acquired by the image acquisition device, but before the raw image data has been stored. In such a case, a component with processing capabilities can be provided locally at the image acquisition device, for example, a graphics processing unit (GPU) or a central processing unit (CPU), to mask the raw image data before sending or storing the raw image data. In such a case, the method according to the present invention is carried out immediately after the raw image data has been acquired by the image acquisition device, i.e.The raw image data is converted into visible images, specific areas of the visible images that contain sensitive information are determined, specific raw image data within the raw image data corresponding to the specific areas is determined, and the specific raw image data is masked within the raw image data. The ultimately obtained masked raw image data can be sent and / or stored locally.

[0033] In another case, the image capture device may not be locally equipped with a component with processing capabilities, nor with a component capable of storing raw image data. The raw image data must be sent to a remote location for processing and storage. In such a case, the image capture device captures the raw image data, and then the method according to the present invention is carried out, i.e., converting the raw image data into visible images, determining specific areas of the visible images that contain sensitive information, determining specific raw image data in the raw image data that correspond to the specific areas, and masking the specific raw image data in the raw image data. The masked raw image data finally obtained can be sent and / or stored remotely.

[0034] Fig. 4 shows a block diagram of an apparatus 10 for processing image data according to an embodiment of the present invention. As in Fig. 4, the device 10 comprises at least a conversion unit 11, a positioning unit 12 and a processing unit 13. It can be expected that the device 10 may further comprise a retrieval unit 14, a transmission unit 15 and / or a storage unit 16.

[0035] Retrieval unit 14 retrieves the raw image data. In one embodiment, raw image data is acquired by an image capture device, and retrieval unit 14 can serve as an interface unit for receiving the raw image data from the image capture device. In another embodiment, retrieval unit 14 itself comprises an image capture device.

[0036] The conversion unit 11 converts the raw image data into visible images; the positioning unit 12 determines the specific areas of the visible images that contain sensitive information; and the processing unit 13 determines the specific raw image data in the raw image data that correspond to the specific areas and masks the specific raw image data in the raw image data. In one embodiment, the processing unit 13 comprises a first processing unit 131 and a second processing unit 132. The first processing unit 131 determines the specific raw image data that correspond to the specific areas in the raw image data, which is determined in particular by assigning the coordinate positions determined between the visible images and the raw image data. The second processing unit 132 masks the specific raw image data in the raw image data.

[0037] When the raw image data includes video image data, the visible images obtained by the conversion unit 11 may include multiple visible images. It is desirable to process these multiple visible images in parallel. The positioning unit 12 determines, for each visible image of the multiple visible images, the specific areas containing sensitive information in the visible image in parallel, and the first processing unit 131 determines, in parallel, the specific raw image data corresponding to the specific areas in the visible image. The second processing unit 132 masks the respective specific raw image data in the raw image data corresponding to the respective visible image among the multiple visible images.

[0038] The masked raw image data can be transmitted by the transmitting unit 15 and / or stored by the storage unit 16. This can be selected according to actual needs. For example, if the masked raw image data needs to be transmitted to a remote storage location for storage, the device according to the present invention includes the transmitting unit 15 that transmits the masked raw image data; or the device according to the present invention includes the storage unit 16, and if the masked raw image data only needs to be stored locally, the transmitting unit 15 need not be included, but the masked raw image data is stored directly in the local storage unit 16. It can also be expected that the device according to the present invention includes both the transmitting unit 15 and the storage unit 16.

[0039] Although in Fig. While Figure 4 shows a device 10 for processing image data according to the present invention, this is not limiting, and one or more components may be omitted, combined, separated, and / or modified as long as a technical solution within the meaning of the present invention can be realized. For example, the processing unit may be combined with the positioning unit, or the functions of the first processing unit in the processing unit may be integrated into the positioning unit. As another example, the retrieval unit 14 and the transmission unit 15 may be implemented by the same interface unit if they serve as interface units for external devices.Any configuration and arrangement method of the individual unit components that can realize the idea of ​​the present invention fall within the equivalent scope of the technical solutions defined by the claims of the present invention.

[0040] In one embodiment, device 10 may also include an image capture device. Therefore, the so-called device for processing image data should be understood as a device that has the capability of processing image data, and not just as a device for processing image data.

[0041] In a specific scenario, the device according to the present invention may comprise a CPU (central processing unit), a GPU (graphics processing unit), an internal memory, and an image capture device such as a camera. The image capture device may have a certain storage capacity, i.e., comprise a storage unit. The raw image data captured by the image capture device may be read directly into the content from the storage unit of the data capture device in order to perform masking according to the individual embodiments of the present invention, after which the masked raw image data may be transmitted and / or stored. Of course, it is also conceivable for the image capture device to have no storage capacity, but instead transmit the captured raw image data directly to the internal memory for masking.

[0042] Fig. 5 shows a block diagram of an electronic device 500 that executes a method for processing image data according to an embodiment of the present invention. As in Fig. 5, the electronic device 500 may include at least one processor 510 (in particular, comprising a central processing unit (CPU) and an image processing unit (GPU)), an image capture device 520, an internal memory 540, a communication interface 550, and a memory (e.g., a non-volatile memory) 560, and the above-mentioned components may be interconnected via a bus 530. The at least one processor 510 executes at least one computer-readable instruction (i.e., an element implemented in the form of software) stored or encoded in the memory 560, wherein the computer-readable instruction may be executed by at least one processor, such that masking of the raw image data is performed in the internal memory 540 according to embodiments of the present invention.The masked raw image data can be sent to a remote memory via the communication interface 550 and also stored in the local memory 560.

[0043] Fig. 1 to 5 show the individual embodiments of the methods and devices of the present invention. These embodiments can be combined with one another to achieve different effects without limiting the subject matter. Furthermore, the described units / steps / processes are not to be understood as limiting. The functions of the described units / steps / processes can be combined / modified / altered to achieve the desired effect.

[0044] The functions of the individual units of the method and system of the present invention can be implemented by software or corresponding hardware. The functions of the individual units can be implemented by means of a processor such as a GPU and / or CPU.

[0045] For example, a processor can read computer programs from a memory and execute these computer programs to implement the functions of the described individual units. In one embodiment, the described electronic device of the present invention can also be implemented by a memory and a processor.

[0046] It is understood that the methods of the individual embodiments of the present disclosure may be implemented by computer programs / software. This software may be loaded into the memory of a processor and, when executed, may perform a method according to the embodiments of the present disclosure.

[0047] The embodiments of the present disclosure cover the following two aspects: Immediately at the beginning, computer programs / software of the present disclosure are created / used, and by means of updates, existing programs / software are modified to use computer programs / software of the present disclosure.

[0048] According to further embodiments of the present disclosure, a computer program product is provided, for example, a machine-readable medium (e.g., a computer) such as a CD-ROM containing computer program code. Upon execution of this computer program code, the computer or processor is caused to perform a method according to the embodiments of the present disclosure. This machine-readable medium is, for example, an optical storage medium or a solid-state storage medium that is integrated into other hardware or delivered as part of other hardware.

[0049] Specific embodiments of the present disclosure have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims may be performed in a different order than that in the embodiments and may still achieve the desired result. Moreover, the process depicted in the figures does not necessarily require a particular order shown or a continuous sequence to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0050] Above, the present disclosure has been described with reference to specific embodiments. Those skilled in the art should understand that the technical solutions of the present disclosure can be implemented in various ways without departing from the spirit and basic features of the disclosure. The specific embodiments are merely schematic and not limiting. Furthermore, these embodiments can be arbitrarily combined to achieve the objectives of the present disclosure. The scope of the present disclosure is defined by the appended claims.

[0051] The term "comprising" in the description and claims does not exclude the presence of additional elements or steps. Expressions such as "first," "second," and "steps" in connection with the individual steps, as well as the sequence of the individual steps shown in the figures, do not define their order or number. The functions of the individual elements explained in the description or listed in the claims may also be performed separately or combined, and may be implemented by multiple elements or a single element.

Claims

[1] A method for processing image data, comprising: converting raw image data into visible images; determining specific areas of the visible images that contain sensitive information; determining specific raw image data in the raw image data corresponding to the specific areas; and masking the specific raw image data in the raw image data. [2] The method of claim 1, wherein the raw image data comprises two-dimensional image data or three-dimensional image data. [3] The method of claim 1, wherein the raw image data comprises video image data and the visible images comprise visible multiple images. [4] Method according to claim 3, wherein for each visible image of the visible multiple images, the specific areas with sensitive information in the visible image are determined in parallel and the specific raw image data corresponding to the specific areas in the visible image are determined; the specific raw image data in the raw image data corresponding to the respective visible image among the visible multiple images is masked. [5] The method according to claim 1, wherein by associating the coordinate positions of visible image and raw image data, the specific raw image data corresponding to the specific areas in the raw image data are determined. [6] A method according to any one of claims 1 to 5, further comprising: Retrieving the raw image data from an image capture device; and Send and / or save the masked raw image data. [7] A method according to any one of claims 1 to 5, further comprising: capturing the raw image data by an image capture device; and Send and / or save the masked raw image data. [8] Device for processing image data, comprising: a conversion unit that converts raw image data into visible images; a positioning unit that determines specific areas of the visible images that contain sensitive information; and a processing unit that determines specific raw image data in the raw image data corresponding to the specific areas and masks the specific raw image data in the raw image data. [9] The apparatus of claim 8, wherein the raw image data comprises video image data and the visible images comprise visible multiple images. [10] The apparatus of claim 9, wherein the processing unit comprises a first processing unit and a second processing unit, the positioning unit determines, in parallel, for each visible image of the visible multiple images, the specific areas with sensitive information in the visible image, and the first processing unit determines, in parallel, the specific raw image data corresponding to the specific areas in the visible image; and the second processing unit masks the specific raw image data in the raw image data corresponding to the respective visible image among the visible multiple images. [11] The apparatus according to claim 8, wherein the processing unit determines the specific raw image data corresponding to the specific areas by associating the coordinate positions of visible image and raw image data. [12] Apparatus according to any one of claims 8 to 11, further comprising: a retrieval unit that retrieves raw image data from an image capture device; and a transmitting unit and / or storing unit that transmits and / or stores masked raw image data. [13] Electronic device comprising: a memory in which at least one computer-executable instruction is stored; and one or more processors which, when executing the at least one computer-executable instruction, cause the following: converting raw image data into visible images; determining specific areas of the visible images that contain sensitive information; determining specific raw image data in the raw image data corresponding to the specific areas; and masking the specific raw image data in the raw image data. [14] The electronic device of claim 13, further comprising at least one of the following components: an image capture device that captures raw image data; a communication interface that sends the masked raw image data; and a memory in which the masked raw image data is stored. [15] A computer-readable medium having stored thereon a computer program, wherein, upon execution of the computer program by a processor or a computer, the processor or computer is caused to carry out a method according to any one of claims 1 to 7.

Images