Privacy protection target detection method and device and computer equipment

By executing a secure interaction protocol based on secret sharing in the cloud server, using third-party servers to generate random numbers, the problem of neglecting location privacy protection in the existing technology is solved, efficient privacy protection target detection is achieved, and computing and communication overhead is reduced. .

CN119051904APending Publication Date: 2024-11-29JOINT WARFARE COLLEGE NAT DEFENSE UNIV OF THE CHINESE PEOPLES LIBERATION ARMY
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411052025.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-01
Publication Date
2024-11-29

AI Technical Summary

Technical Problem

The existing privacy protection target detection methods mainly focus on protecting the privacy of target classification, ignore the protection of location privacy, and incur a large amount of overhead during the ROI pooling calculation process and may disclose equal values.

Method used

Using a secure interaction protocol based on secret sharing, by randomly dividing remote sensing images into image shares and executing security protocols in cloud servers, a third-party server generates random numbers to support the execution of security protocols, ensuring the protection of classification and location privacy.

Benefits of technology

It realizes the simultaneous protection of classification and location privacy in object detection, reduces computing and communication overhead, avoids the leakage of location privacy, and provides an efficient privacy protection target detection framework.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119051904A_ABST
    Figure CN119051904A_ABST
Patent Text Reader

Abstract

The invention relates to a privacy protection target detection method and apparatus, and a computer device. The method comprises the steps of allocating remote sensing image shares from a satellite to a corresponding cloud server; wherein the image share is obtained by randomly dividing an original remote sensing image; one cloud server corresponds to one image share; in each cloud server, executing a security protocol on the corresponding image share to obtain a corresponding target detection result share, and transmitting the target detection result share to the user; wherein the third-party server is used for generating a random number and sending the random number to the corresponding cloud server, so as to support the execution of the security protocol. By adopting the method, classification and location privacy can be protected at the same time.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of cloud computing technology, and in particular to a privacy protection target detection method, device and computer equipment. Background Art

[0002] Recently, deep learning techniques have been increasingly used for target detection in remote sensing images. The latest target detection methods mainly use network architectures based on convolutional neural networks, including Single Shot Multi-Box Detector, Region-based Fully Convolutional Network, You Only Look Once (YOLO), RetinaNet, and Faster R-CNN. Among them, Faster R-CNN is widely praised for its excellent accuracy and robustness in detecting and locating targets in images or videos.

[0003] Nowadays, secure neural network inference has received widespread attention. SecureML is one of the earliest schemes to utilize secret sharing. Based on this, some research works have developed hybrid protocol frameworks that combine arithmetic sharing, Boolean sharing, and Yao's garbled circuits. This framework aims to improve the performance of secure computing by leveraging the advantages of different protocols. Delphi takes a different approach by simplifying the neural network model using neural architecture search (NAS) and hyperparameter optimization to achieve efficient secure computing. Falcon designs a scheme that only uses secret sharing to eliminate the overhead of protocol conversion while improving communication efficiency by reducing the integer ring size.

[0004] Unlike previous work that adopted uniform bit width for inference, SIRNN introduced non-uniform (or mixed) bit width to minimize communication overhead. In addition, CryptGPU and Piranha focus on implementing all operations on GPU to accelerate CNN inference. Finally, SecFloat supports 32-bit single-precision floating-point operations, achieving significant improvements in accuracy and efficiency.

[0005] Although there have been a lot of works on secure neural network inference, most of them focus on convolutional neural networks and ignore other components of object detection such as region proposal networks (RPNs) and classification & regression networks.

[0006] To the best of our knowledge, SecRCNN is the earliest privacy-preserving object detection framework. It combines secret sharing with the Faster R-CNN network and improves the security protocol using a coordinate rotation digital computer algorithm to reduce communication overhead. Similarly, Wang et al. also developed an object detection framework that leverages secret sharing. They designed secure division, exponential, and logarithmic protocols that minimize the number of iterations through the idea of ​​fuzzification, thereby reducing the inference cost. These two methods only focus on protecting the privacy of object classification and ignore the protection of location privacy. P2OD is a framework designed to protect both classification and location privacy. However, it incurs a lot of overhead during ROI pooling calculations and inadvertently leaks equal values ​​during secure comparison. Summary of the invention

[0007] Based on this, it is necessary to provide a privacy-preserving target detection method, device and equipment that can simultaneously protect classification and location privacy in order to address the above technical problems.

[0008] A privacy-preserving target detection method, the method comprising:

[0009] Allocating remote sensing image shares from satellites to corresponding cloud servers; wherein the image shares are obtained by randomly dividing the original remote sensing images; and one cloud server corresponds to one image share;

[0010] In each cloud server, a security protocol is executed on the corresponding image share, and a corresponding target detection result share is obtained and transmitted to the user;

[0011] Among them, a third-party server is used to generate random numbers and send them to the corresponding cloud server to support the execution of the security protocol.

[0012] A privacy protection target detection device, the device comprising:

[0013] An image share allocation module is used to allocate remote sensing image shares from satellites to corresponding cloud servers; wherein the image shares are obtained by randomly dividing the original remote sensing images; and one cloud server corresponds to one image share;

[0014] The target detection result calculation module is used to execute the security protocol on the corresponding image share in each cloud server, obtain the corresponding target detection result share and transmit it to the user;

[0015] Among them, a third-party server is used to generate random numbers and send them to the corresponding cloud server to support the execution of the security protocol.

[0016] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0017] Allocating remote sensing image shares from satellites to corresponding cloud servers; wherein the image shares are obtained by randomly dividing the original remote sensing images; and one cloud server corresponds to one image share;

[0018] In each cloud server, a security protocol is executed on the corresponding image share, and the corresponding target detection result share is obtained and transmitted to the user;

[0019] Among them, a third-party server is used to generate random numbers and send them to the corresponding cloud server to support the execution of the security protocol.

[0020] In the above-mentioned privacy-preserving target detection method, device and computer equipment, the cloud server performs a secure interaction protocol based on secret sharing to predict the classification and location of the target. At the same time, a third-party server generates random numbers to support the secure interaction protocol. Since the third-party server itself does not participate in the calculation process, it will not affect the security of the data. In other words, the satellite uploads the secret-shared data to the cloud server, and the user reconstructs the target detection result from the server output. In summary, the present invention proposes a privacy-preserving target detection framework that can protect classification and location privacy at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 1 is a flowchart of a privacy protection target detection method in one embodiment;

[0022] Figure 2 A flowchart of a secure reordering algorithm in one embodiment;

[0023] Figure 3 An example diagram of a secure out-of-order protocol execution in one embodiment;

[0024] Figure 4 is a flowchart of a secure Maxpool algorithm in one embodiment;

[0025] Figure 5 Details of the security index agreement;

[0026] Figure 6 It is the flow chart of the secure NMS algorithm;

[0027] Figure 7 It is the algorithm flow chart of the safe ROI pooling protocol;

[0028] Figure 8 is an example diagram of the secure array access protocol;

[0029] Fig. 9 It is the algorithm flow chart of the secure array access protocol;

[0030] Fig.10 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0032] In one embodiment, Figure 1 As shown, a privacy-preserving target detection method is provided, comprising the following steps:

[0033] Step 102: Allocate the remote sensing image shares from the satellite to the corresponding cloud servers.

[0034] The image share is obtained by randomly dividing the original remote sensing image; one cloud server corresponds to one image share.

[0035] Step 104: In each cloud server, a security protocol is executed on the corresponding image share to obtain the corresponding target detection result share and transmit it to the user.

[0036] Among them, a third-party server is used to generate random numbers and send them to the corresponding cloud server to support the execution of the security protocol.

[0037] Object detection is crucial for remote sensing image analysis and processing. However, due to the limited storage and processing power of users, object detection services must be outsourced. However, this may raise concerns about the privacy of uploaded images. In this solution, four entities are involved: satellites, users, and cloud servers. and And third-party servers

[0038] The satellite has the original image and wants to obtain target detection results without sharing private information with others. To this end, each image is randomly divided into two shares: and Send to and Cloud server response and The user adds up the shares received to obtain the final test result.

[0039] and There are two outsourced cloud servers that share images. and Execute security protocols to obtain target classification and location. Output Share and It is then transmitted to the user.

[0040] A semi-honest server Used to generate random numbers in the offline phase and send them to two cloud servers and To support the execution of security protocols.

[0041] Similar to earlier work, we consider the honest but curious model, i.e., each cloud server faithfully follows the protocol specification while also trying to guess private or sensitive information during the computation. In addition, it is worth noting that two cloud servers will not collude with each other. This ensures that no other information is exchanged between them except the required protocol interactions. These assumptions are not only reasonable but also practical in real-world scenarios. For example, cloud servers may belong to different reputable service providers, whose collusion cost is high and any dishonest behavior will seriously damage their business reputation.

[0042] Semi-honest servers Responsible for generating random numbers, which can be replaced by a lightweight server or user client. Since this server itself does not participate in the computation process, it does not affect the security of the entire model. The satellite uploads the secret shared data to the cloud server and reconstructs the target detection results from the server's output without participating in intermediate computations. We assume the honesty of the satellite and the authenticity of the uploaded data. In addition, we also assume that there is a secure communication channel between all entities.

[0043] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0044] In one embodiment, executing a security protocol on an image share to obtain a target detection result share includes:

[0045] 1) Input the image share into the feature extractor for security feature extraction.

[0046] The feature extractor consists of secure convolutional layers, secure activation layers, and secure pooling layers.

[0047] When performing security feature extraction, VGG-16\ResNet or MobileNet can be used as a feature extractor.

[0048] Secure convolutional layer: For a given input and calculate:

[0049]

[0050]

[0051] Here, W and b are the weight parameters and bias. Note that in this case, we use the publicly available Faster R-CNN model for inference, which means that the values ​​of W and b are available.

[0052] Secure activation layer: The ReLU function is defined as ReLU(x) = max(x, 0). Huang et al. designed a secure ReLU using a secure MSB (Most significant bit) protocol:

[0053]

[0054] The secure pooling layer divides the feature map shares processed by the secure convolution layer and the secure activation layer into fixed-size pooling windows, and outputs the maximum value in each pooling window according to the secure Maxpool protocol to obtain the pooled feature map shares. In this layer, the kernel size and stride can both be set to 2, which allows the input to be divided into smaller arrays for parallel calculation, thereby increasing the calculation speed.

[0055] In each pooling window, the secure Maxpool protocol first uses the secure out-of-order protocol to shuffle the vector shares to obtain out-of-order vector shares, and then compares the out-of-order vector shares to obtain the maximum value.

[0056] 2) Extract the target detection candidate box share according to the pooled feature map share, and map the target detection candidate box shares of different sizes to the feature map of a fixed size, so as to obtain the feature share corresponding to each target detection candidate box, and use it for target classification and bounding box regression, that is, to obtain the target detection result share.

[0057] In one embodiment, the vector shares are shuffled using a secure shuffle protocol to obtain shuffled vector shares, including:

[0058] Offline stage:

[0059] Send the random permutation vector and the first random vector generated by the third-party server, and the second random vector calculated based on the random permutation vector and the first random vector, to the corresponding cloud servers respectively; one cloud server corresponds to one random permutation vector, one first random vector and one second random vector;

[0060] Online stage:

[0061] In each cloud server, a sum of a corresponding original vector share and a first random vector is calculated to obtain a first intermediate vector share;

[0062] In the selected cloud server, the sum of all first intermediate vector shares is calculated to obtain a second intermediate vector share, and the second intermediate vector share is shuffled using a random permutation vector corresponding to the selected cloud server to obtain a third intermediate vector share and send it to other cloud servers; in addition, the corresponding shuffled vector share is obtained according to the second random vector corresponding to the selected cloud server;

[0063] In other cloud servers, the third intermediate vector shares are shuffled according to their corresponding random permutation vectors to obtain initial shuffled vector shares, and corresponding shuffled vector shares are calculated according to the corresponding second random vectors and the initial shuffled vector shares;

[0064] The sum of the cloud server's original vector shares and the sum of the shuffled vector shares contain the same elements, but the order of the elements is different according to the random permutation vector.

[0065] like Figure 2 As shown, a flowchart of a secure reordering algorithm is provided.

[0066] In the Secure Shuffle Protocol (SShu), we want to compute:

[0067] For a vector x and a random permutation vector π, they are related to S i are unknown, so the permuted vector is defined as π(x). Figure 3 The following is an example diagram of the secure out-of-order protocol execution. The secret vector x is [1,2,3,4,5], which is randomly split into two addition shares. The random permutation vector π is [1,4,0,2,3], π0 = [0,2,1,4,3], π1 = [1,0,4,3,2]. Here, First, S i Using a third-party server The vector pair To cover, Send its masked share to Adding the two masked shares yields [4,11,5,10,18]. The result is shuffled by π1 to get [11,4,18,10,5], We further use π0 to shuffle the result and get [11,18,4,5,10]. Finally, Masking this vector gives the final output [4,11,4,4,9], and Output from The vector [-2,-6,-3,-1,-5] of the secret vector x is [-2,-6,-3,-1,-5], and the sum of the two outputs is [2,5,1,3,4], which contains the same elements as the secret vector x, but in a different order according to π.

[0068] In one embodiment, the secure Maxpool protocol first uses the secure out-of-order protocol to shuffle the vector shares to obtain out-of-order vector shares, and then compares the out-of-order vector shares to obtain the maximum value, including:

[0069] When executing the secure Maxpool protocol, first use the secure out-of-order protocol to i The original vector Perform out-of-order processing to obtain out-of-order vector shares

[0070] The first element in the shuffled vector share As the initial maximum value

[0071] In the jth iteration, compare the current maximum value The jth element in the random vector share The size of; if the current maximum Less than the jth element in the shuffled vector share The value of Then the value of the jth element in the random vector share is taken as the new maximum value; where the number of iterations is n-1, n is the total number of elements in the original vector share, Represents a binary bit, that is, 0 or 1, and its subscript indicates which cloud server the share belongs to.

[0072] Given a vector of length n The goal of the Maxpool function is to calculate the maximum value among these n elements. Most existing methods regard the Maxpool function as a combination of safe comparison and safe multiplication protocols. Specifically, this process starts with setting the maximum value: Then apply a security comparison algorithm to determine If this condition is true, is assigned the value 0; otherwise, it is assigned the value 1. Next, safe multiplication is called to compute if If it is equal to 0, the result value Set to 0; otherwise, Assigned to Finally, max is updated to Therefore, if max remains unchanged; otherwise, max is updated to This process is repeated until the vector All elements in are compared. The safe multiplication in this process helps protect the comparison results. But it is worth noting that using secure multiplication will increase communication overhead.

[0073] To solve this problem, we shuffle the elements in the shared vector before comparing them. This makes the comparison result independent of the original values.

[0074] like Figure 4 As shown, a flowchart of the secure Maxpool algorithm is provided.

[0075] In one embodiment, extracting a target detection candidate box share according to a pooled feature map share includes:

[0076] Generate anchor points based on pooled feature map shares, input the anchor points into the Softmax layer, and obtain foreground anchor points. Each foreground anchor point share has a corresponding confidence score.

[0077] Offline stage: the third random number and the fourth random number generated by the third-party server are sent to the corresponding cloud servers respectively; one cloud server corresponds to one third random number and one fourth random number;

[0078] Online stage: Use the secure out-of-order protocol to shuffle the foreground anchor points to obtain out-of-order anchor points; use the secure out-of-order protocol to shuffle the confidence scores to obtain out-of-order confidence scores;

[0079] In each cloud server, a linear transformation is performed on the corresponding random confidence score according to the third random number and the fourth random number to obtain a linear transformation confidence score;

[0080] In the selected cloud server, calculate the sum of all linear transformation confidence scores, obtain the transformation confidence score and input it into the argsort() function, return the index value, sort the disordered anchor point shares in the order of the index value, and obtain the rearranged anchor point shares;

[0081] The rearranged anchor point share with the largest index value is added to the target anchor point share set, and the remaining rearranged anchor point shares are overlapped with it according to the intersection-over-union ratio. The foreground anchor point shares with overlaps higher than the predetermined threshold are deleted to obtain the final target anchor point share set, i.e., the target detection candidate box share.

[0082] RPN plays a key role in generating object detection regions. First, RPN generates anchors based on feature maps as reference points for potential objects. Then, RPN uses a Softmax layer to obtain foreground anchors, which are usually associated with objects of interest. In the Faster R-CNN network, the input of the Softmax layer usually shows a distribution in a relatively narrow range. Therefore, the Softmax function can be calculated as:

[0083]

[0084] To ensure the security of the computation, we can implement the Softmax protocol by using the secure exponential and secure division protocols.

[0085] The current security index calculation process requires polynomial iterations, which leads to more computational and communication overhead. To solve this problem, we design a security index protocol inspired by Beaver et al. that only requires exchanging two elements.

[0086] like Figure 5 As shown in Figure 2, the details of the security exponential protocol are described. Specifically, according to the linearity of the secret sharing scheme, we have:

[0087]

[0088] in, c is a random value. In the online stage, S i Will Masked as and send it to the other party. Finally, S i Compute locally The whole process requires only one round of communication.

[0089] The Softmax layer outputs foreground anchors, but these anchors usually have serious overlap problems. To solve this problem, NMS plays an important role.

[0090] Specifically, and The foreground anchors are sorted according to the confidence scores. The anchor with the highest confidence is selected and added to the final target anchor set. Then, the remaining candidate anchors are overlapped with the selected anchor by the intersection over union (IoU). Candidate anchors with overlaps above a predetermined threshold are removed. This process is repeated until all candidate anchors have been processed. Therefore, and Redundant and highly overlapping candidate anchor points are eliminated to obtain the final set of target anchor points.

[0091] like Figure 6 As shown, a flowchart of the secure NMS algorithm is provided. The implementation details of NMS are summarized as follows:

[0092] In Algorithm 5, and The candidate anchors are first sorted according to their confidence scores. P2OD uses a secure comparison protocol for the sorting operation, which exposes the comparison results and incurs significant computational cost. To address this issue, our method first shuffles the original data so that the comparison results are independent of the original data (line 2). Then, we efficiently perform the sorting operation after linearly transforming the shuffled data (lines 3-5).

[0093] After sorting, select the anchor with the highest score, that is, the highest confidence score (Lines 7-8), and calculate it and the remaining boxes IoU (lines 9-15). Figure 7 As shown in Figure 2, IoU measures the overlap of two anchor bounding boxes by comparing their intersection and union. The coordinates of the top left corner of the intersection are and It is determined by taking the maximum value of the upper left corner coordinates of the two anchor points. Specifically, is set to and The maximum value between The same is true. The coordinates of the lower right corner of the intersection and It is obtained by taking the minimum value of the lower right corner coordinates of the two anchor points. Specifically, is set to and The minimum value between The same is true (line 12). and If , there is an intersection; otherwise, there is no overlap. The width and height of the intersection are and (Line 13). To optimize the calculation, only the width and height of the intersection are disclosed (Line 14). IoU exceeds the threshold N tThe anchors with are removed (line 16). This process (lines 7-16) is repeated until there are no remaining candidate anchors.

[0094] In one embodiment, the target detection candidate box shares of different sizes are mapped to a feature map of a fixed size, thereby obtaining a feature share corresponding to each target detection candidate box and using it for target classification and bounding box regression, including:

[0095] Reshape the target detection candidate box shares of different sizes using the scaling factor to match the size of the feature map to obtain the reshaped target detection candidate box shares;

[0096] Extracting the height and width of the reshaped object detection candidate frame share, and using them to divide the corresponding reshaped object detection candidate frame share into a plurality of sub-regions;

[0097] The secure array access protocol is used to obtain the feature values ​​in each sub-region, and the maximum value is determined for each sub-region. The maximum value is input into the fully connected layer and the softmax layer for target classification and bounding box regression tasks.

[0098] In the original P2OD algorithm, the region of interest pooling (ROI pooling) determines the feature value in the sub-region by comparing the position relationship between the feature map and the sub-region. However, this method will cause a lot of computational and communication overhead. To solve this problem, this solution adopts a secure array access protocol in the secure ROI pooling protocol. Figure 7 As shown, an algorithm flow chart of a safe ROI pooling protocol is provided.

[0099] First of all, Use the scaling factor to transform the detection box Reshape into feature map The same size (line 1). Then extract the height b of the detection box h and width b w , preparing for efficient sub-region division (line 2). Each detection box is divided into 7x7 sub-regions (line 3). Next, the secure array access (SAcc) protocol is used to obtain the feature values ​​in each sub-region (line 4). Finally, PODI uses the efficient method in P2OD to determine the maximum value for each sub-region (line 5).

[0100] In one embodiment, a secure array access protocol is used to obtain the feature value in each sub-region, including:

[0101] Offline stage: using a third-party server to generate row random displacement values ​​and column random displacement values, as well as a first random array and a second random array, and send them to the corresponding cloud server; one cloud server corresponds to one row random displacement value, one column random displacement value, one first random array and one second random array;

[0102] Online stage: using the first random array to mask the original sub-region to obtain a masked sub-region; using the corresponding row random displacement value and column random displacement value to shift the masked sub-region in the row and column directions to obtain a first shifted sub-region;

[0103] After the cloud servers exchange the first shifted sub-regions with each other, the first shifted sub-regions from other cloud servers are shifted in the selected cloud server using the corresponding row random shift values ​​and column random shift values ​​to obtain the second shifted sub-regions;

[0104] The second shifted sub-region is processed according to the second random array to obtain an updated sub-region, and the index value of the updated sub-region is updated using the corresponding row random shift value and column random shift value to obtain the characteristic value in the updated sub-region.

[0105] In this function, we need to use the encrypted index to access the elements in the encrypted array. Specifically, given the encrypted array and secret index We hope to get the corresponding secret results To do this, the secret index can be compared with the indices of all elements in the array. However, the expensive secure comparison protocol leads to a large communication overhead. Blanton et al. proposed a 3-party secret sharing algorithm using blind indexing that requires 4m+4 elements for two rounds of communication, where m represents the array length. Wang et al. improved this protocol by reducing the required elements to 2m+2 in a single round. However, it should be noted that the computational load among the three parties is unbalanced. We design a computationally load-balanced protocol in a 2-party setting, achieving an optimal communication overhead of 2m+2. Figure 8 An example is given to illustrate our solution. Secret Array and secret index Depend on Hold. The rectangle represents the element we want to access. It should be noted that this example only illustrates a single-dimensional shift, and in actual protocol design, multi-dimensional shifts are required to enhance security.

[0106] like Fig. 9 As shown, an algorithm flow chart of a secure array access protocol is provided.

[0107] Offline stage. We use Represents the shift position of the array in the row and column directions, let M i , N i is a random array, where N1+N0←Φ(r x ,Φ(r y , (M0+M1))). Hold them separately.

[0108] Online stage. First, Share the array Masking Then, along the row and column directions, move and positions, get and exchange And continue to Along and Move positions and get at last, calculate use and Update Index to h′, and publish h′ to obtain .

[0109] In summary, in view of the fact that existing research on privacy-preserving object detection networks either lacks location privacy protection or involves excessive computational and communication overhead, this application proposes a privacy-preserving object detection inference framework based on PODI, a Faster R-CNN-based framework that simultaneously protects classification and location privacy. PODI uses an additive secret sharing protocol to support collaborative computing between two cloud servers. By using efficient protocols such as Maxpool, array access, and exponential, PODI significantly reduces computational and communication overhead.

[0110] In one embodiment, a privacy-preserving object detection device is provided, comprising: an image share allocation module and an object detection result calculation module, wherein:

[0111] An image share allocation module is used to allocate image shares from satellites to corresponding cloud servers; wherein the image shares are obtained by randomly dividing the original remote sensing images; and one cloud server corresponds to one image share;

[0112] The target detection result calculation module is used to execute the security protocol on the corresponding image share in each cloud server, obtain the corresponding target detection result share and transmit it to the user;

[0113] Among them, a third-party server is used to generate random numbers and send them to the corresponding cloud server to support the execution of the security protocol.

[0114] For the specific definition of the privacy protection target detection device, please refer to the definition of the privacy protection target detection method above, which will not be repeated here. Each module in the above-mentioned privacy protection target detection device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0115] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig.10 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as image shares. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a privacy protection target detection method is implemented.

[0116] Those skilled in the art will understand that Fig.10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0117] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method in the above embodiment when executing the computer program.

[0118] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method in the above embodiment are implemented.

[0119] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0120] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0121] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A privacy-preserving target detection method, characterized in that: The method comprises: Allocating remote sensing image shares from satellites to corresponding cloud servers; wherein the image shares are obtained by randomly dividing the original remote sensing images; and one cloud server corresponds to one image share; In each of the cloud servers, a security protocol is executed on a corresponding image share to obtain a corresponding target detection result share and transmit it to a user; Among them, a third-party server is used to generate random numbers and send them to the corresponding cloud server to support the execution of the security protocol.

2. The method according to claim 1, characterized in that Execute the security protocol on the image share to obtain the target detection result share, including: The image shares are input into a feature extractor for security feature extraction; the feature extractor includes a security convolution layer, a security activation layer and a security pooling layer; the security pooling layer divides the feature map shares obtained by processing the security convolution layer and the security activation layer into pooling windows of a fixed size, and outputs the maximum value in each pooling window according to the security Maxpool protocol to obtain pooled feature map shares; in each of the pooling windows, the security Maxpool protocol first uses a security shuffle protocol to shuffle the vector shares to obtain shuffled vector shares, and then compares the shuffled vector shares to obtain the maximum value; The target detection candidate box share is extracted according to the pooled feature map share, and the target detection candidate box shares of different sizes are mapped to the feature map of a fixed size, so as to obtain the feature share corresponding to each target detection candidate box, and use it for target classification and bounding box regression, that is, to obtain the target detection result share.

3. The method according to claim 2, characterized in that The vector shares are shuffled using a secure shuffle protocol to obtain shuffled vector shares, including: Offline stage: Send the random permutation vector and the first random vector generated by the third-party server, and the second random vector calculated based on the random permutation vector and the first random vector, to the corresponding cloud servers respectively; one cloud server corresponds to one random permutation vector, one first random vector and one second random vector; Online stage: In each cloud server, a sum of a corresponding original vector share and a first random vector is calculated to obtain a first intermediate vector share; In the selected cloud server, the sum of all first intermediate vector shares is calculated to obtain a second intermediate vector share, and the second intermediate vector share is shuffled using a random permutation vector corresponding to the selected cloud server to obtain a third intermediate vector share and send it to other cloud servers; in addition, the corresponding shuffled vector share is obtained according to the second random vector corresponding to the selected cloud server; In other cloud servers, the third intermediate vector shares are shuffled according to their corresponding random permutation vectors to obtain initial shuffled vector shares, and corresponding shuffled vector shares are calculated according to the corresponding second random vectors and the initial shuffled vector shares; the sum of the original vector shares and the sum of the shuffled vector shares contain the same elements, but the order of the elements is different according to the random permutation vector.

4. The method according to claim 2, characterized in that: The secure Maxpool protocol first uses the secure disorder protocol to disorder the vector shares to obtain disordered vector shares, and then compares the disordered vectors to obtain the maximum value, including: When executing the secure Maxpool protocol, the original vector shares are firstly shuffled using the secure shuffle protocol to obtain shuffled vector shares; Use the first element in the shuffled vector share as the initial maximum value; In the jth iteration, compare the current maximum value with the jth element in the shuffle vector share; if the current maximum value is less than the value of the jth element in the shuffle vector share, and Then the value of the jth element in the random vector share is taken as the new maximum value; wherein the number of iterations is n-1, and n is the total number of elements in the original vector share; Represents a binary bit, that is, 0 or 1, and its subscript indicates which cloud server the share belongs to.

5. The method according to claim 2, characterized in that: Extract the target detection candidate frame share based on the pooled feature map share, including: Generate anchor points based on pooled feature map shares, input the anchor points into the Softmax layer, and obtain foreground anchor points. Each foreground anchor point share has a corresponding confidence score. Offline stage: the third random number and the fourth random number generated by the third-party server are sent to the corresponding cloud servers respectively; one cloud server corresponds to one third random number and one fourth random number; Online stage: Use the secure out-of-order protocol to shuffle the foreground anchor points to obtain out-of-order anchor points; use the secure out-of-order protocol to shuffle the confidence scores to obtain out-of-order confidence scores; In each cloud server, a linear transformation is performed on the corresponding random confidence score according to the third random number and the fourth random number to obtain a linear transformation confidence score; In the selected cloud server, calculate the sum of all linear transformation confidence scores, obtain the transformation confidence score and input it into the argsort() function, return the index value, sort the disordered anchor point shares in the order of the index value, and obtain the rearranged anchor point shares; The rearranged anchor point share with the largest confidence score is added to the target anchor point share set, and the remaining rearranged anchor point shares are overlapped with it according to the intersection-over-union ratio. The foreground anchor point shares with overlap higher than the predetermined threshold are deleted to obtain the final target anchor point share set, i.e., the target detection candidate box share.

6. The method according to claim 2, characterized in that Map the target detection candidate box shares of different sizes to the feature map of fixed size, so as to obtain the feature share corresponding to each target detection candidate box share, and use it for target classification and bounding box regression, including: Reshape the target detection candidate box shares of different sizes using the scaling factor to match the size of the feature map to obtain the reshaped target detection candidate box shares; Extracting the height and width of the reshaped object detection candidate frame share, and using them to divide the corresponding reshaped object detection candidate frame share into a plurality of sub-regions; The secure array access protocol is used to obtain the feature values ​​in each sub-region, and the maximum value is determined for each sub-region. The maximum value is input into the fully connected layer and the softmax layer for target classification and bounding box regression tasks.

7. The method according to claim 6, characterized in that Use the safe array access protocol to obtain the feature values ​​in each sub-region, including: Offline stage: using a third-party server to generate row random displacement values ​​and column random displacement values, as well as a first random array and a second random array, and send them to the corresponding cloud server; one cloud server corresponds to one row random displacement value, one column random displacement value, one first random array and one second random array; Online stage: using the first random array to mask the original sub-region to obtain a masked sub-region; using the corresponding row random displacement value and column random displacement value to shift the masked sub-region in the row and column directions to obtain a first shifted sub-region; After the cloud servers exchange the first shifted sub-regions with each other, the first shifted sub-regions from other cloud servers are shifted in the selected cloud server using the corresponding row random shift values ​​and column random shift values ​​to obtain the second shifted sub-regions; The second shifted sub-region is processed according to the second random array to obtain an updated sub-region, and the index value of the updated sub-region is updated using the corresponding row random shift value and column random shift value to obtain the characteristic value in the updated sub-region.

8. A privacy protection target detection device, characterized in that: The device comprises: An image share allocation module, used to allocate remote sensing image shares from satellites to corresponding cloud servers; wherein the image shares are obtained by randomly dividing the original remote sensing images; one cloud server corresponds to one image share; A target detection result calculation module, used to execute a security protocol on a corresponding image share in each of the cloud servers, obtain a corresponding target detection result share and transmit it to a user; Among them, a third-party server is used to generate random numbers and send them to the corresponding cloud server to support the execution of the security protocol.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.