Image occlusion processing method and device, storage medium and electronic device

Abstract

The present invention discloses a method and device for processing image occlusion, a storage medium and an electronic device. The method comprises: obtaining a first target image sent by a target device, wherein the first target image includes at least one target area, and the target area is an area after occlusion processing; obtaining a second target image according to the network traffic state of the target device, wherein the second target image is used to remove at least one target area that is occluded in the first target image; removing at least one target area that is occluded in the first target image according to the second target image, thereby achieving the purpose of real-time detection of the network traffic state by the target device, determining the current network state (whether it is congested), and adjusting the sent code stream data packets, thereby solving the technical problem of low efficiency in restoring the occluded part of the image in the prior art.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a method and device for processing image occlusion, a storage medium and an electronic device. Background Art

[0002] In recent years, with the development of network and AI technology, personal data protection has received unprecedented attention: the "Personal Information Protection Law of the People's Republic of China", the EU "General Data Protection Regulation" (GDPR), the California Consumer Privacy Act (CCPA) and other laws and regulations have been issued one after another. From this aspect, it can be seen that personal data protection (privacy protection) will become the focus of protection in all walks of life in the future. In the video surveillance industry, due to its special business nature, personal data has become the focus of its protection.

[0003] In the video surveillance system, the face, body, license plate and other information of the data subject will be most intuitively presented in the real-time monitoring screen and video. The current common processing method is irreversible image blur and occlusion. However, due to the particularity of the video surveillance system, in some special cases (such as assisting in case investigations, etc.), the police and other relevant departments need to retrieve the video and assist in the investigation by identifying the personal information in the video. Therefore, how to effectively protect the personal information in the video (usually presented in the form of occlusion, and can be removed when necessary) is the current compliance focus of video surveillance equipment and systems.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present invention provide a method and device for processing image occlusion, a storage medium and an electronic device, so as to at least solve the technical problem in the prior art that the efficiency of restoring the occluded part of an image is low.

[0006] According to one aspect of an embodiment of the present invention, there is provided a method for occlusion processing of an image, comprising: obtaining a first target image sent by a target device, wherein the first target image includes at least one target area, and the target area is an area after occlusion processing; obtaining a second target image according to a network traffic status of the target device, wherein the second target image is used to remove the at least one target area that is occluded in the first target image; and removing the at least one target area that is occluded in the first target image according to the second target image.

[0007] Optionally, acquiring the second target image according to the network traffic status of the target device includes:

[0008] When the network traffic state is in an idle state, obtaining a third target picture sent by the target device, wherein the third target picture is an original picture corresponding to the first target picture, and the second target picture includes the third target picture;

[0009] When the network traffic state is in a non-idle state, a fourth target picture sent by the target device is obtained, wherein the fourth target picture is an original picture of the target area to be occluded in the first target picture, and the second target picture includes the fourth target picture.

[0010] Optionally, removing the at least one target area that has been occluded in the first target picture according to the second target picture includes: obtaining a target algorithm that has been occluded in the first target picture; and re-encoding the first target picture according to the target algorithm and the fourth target picture to remove the at least one target area that has been occluded in the first target picture.

[0011] Optionally, before acquiring the second target image according to the network traffic status of the target device, the method further includes: creating a statistical model for data packet sending of the target device; and detecting the network traffic status according to the statistical model.

[0012] Optionally, detecting the network traffic status according to the statistical model includes: monitoring the data packet sending process; calculating the average time consumed by scheduling the data packets to send data through the statistical model; calculating the amount of data sent each time according to the data packets sent and the time consumed for each scheduling; and determining that the network status is a non-idle state when the amount of data sent is less than or equal to a first predetermined threshold.

[0013] Optionally, the method includes: when the data transmission amount is greater than the first predetermined threshold, determining that the network state is an idle state.

[0014] According to another aspect of an embodiment of the present invention, a device for processing image occlusion is also provided, including: a first acquisition unit, used to acquire a first target image sent by a target device, wherein the first target image includes at least one target area, and the target area is an area after occlusion processing; a second acquisition unit, used to acquire a second target image according to a network traffic status of the target device, wherein the second target image is used to remove the at least one target area that is occluded in the first target image; and a removal unit, used to remove the at least one target area that is occluded in the first target image according to the second target image.

[0015] Optionally, the above-mentioned second acquisition unit includes: a first acquisition module, used to acquire a third target picture sent by the target device when the network traffic state is in an idle state, wherein the third target picture is an original picture corresponding to the first target picture, and the second target picture includes the third target picture; a second acquisition module, used to acquire a fourth target picture sent by the target device when the network traffic state is in a non-idle state, wherein the fourth target picture is an original picture of the target area to be occluded in the first target picture, and the second target picture includes the fourth target picture.

[0016] Optionally, the above-mentioned removal unit includes: a third acquisition module, used to obtain a target algorithm for performing occlusion processing in the first target picture; a removal module, used to re-encode the first target picture according to the target algorithm and the fourth target picture, and remove at least one target area that is subjected to occlusion processing in the first target picture.

[0017] Optionally, the above-mentioned device also includes: a creation unit, used to create a statistical model of data packet sending of the target device before obtaining the second target image according to the network traffic status of the target device; and a detection unit, used to detect the network traffic status according to the statistical model.

[0018] Optionally, the above-mentioned detection unit includes: a monitoring module, used to monitor the data packet sending process; a statistical module, used to statistically schedule the average time taken to send data packets through the statistical model; a calculation module, used to calculate the amount of data sent each time based on the data packets sent and the time taken for each scheduling; and a first determination module, used to determine that the network state is a non-idle state when the data sending amount is less than or equal to a first predetermined threshold.

[0019] Optionally, the above-mentioned device includes: a second determination module, used to determine that the network state is an idle state when the data transmission amount is greater than the first predetermined threshold.

[0020] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the above-mentioned image occlusion processing method when running.

[0021] According to another aspect of an embodiment of the present invention, there is provided an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the above-mentioned method for processing image occlusion through the computer program.

[0022] In an embodiment of the present invention, a first target image sent by a target device is obtained, wherein the first target image includes at least one target area, and the target area is an area after occlusion processing; a second target image is obtained according to a network traffic state of the target device, wherein the second target image is used to remove at least one target area that is occluded in the first target image; and at least one target area that is occluded in the first target image is removed according to the second target image, thereby achieving the purpose of real-time detection of the network traffic state by the target device, determining the current network state (whether it is congested), and adjusting the sent code stream data packets, thereby achieving the technical effect of determining to obtain different target images according to the network traffic state, and the target image is used to restore the occluded part of the image, thereby solving the technical problem of low efficiency in restoring the occluded part of the image in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0024] Figure 1 is a schematic diagram of an application environment of an optional method for processing image occlusion according to an embodiment of the present invention;

[0025] Figure 2 is a flowchart of an optional method for processing image occlusion according to an embodiment of the present invention;

[0026] Figure 3 is a flow chart of an optional real-time optimization method for personal data privacy shielding based on network status detection according to an embodiment of the present invention;

[0027] Figure 4 is a schematic diagram of an optional real-time optimization structure of personal data privacy shielding based on network status detection according to an embodiment of the present invention;

[0028] Figure 5 is a schematic structural diagram of an optional image occlusion processing device according to an embodiment of the present invention;

[0029] Figure 6 is a schematic structural diagram of an optional electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0032] First, some of the nouns in this application are explained as follows:

[0033] Personal data: According to the EU GDPR (General Data Protection Regulation), personal data refers to all information about an identified or identifiable natural person ("data subject"). "Personal data" in this patent proposal can specifically refer to data subject information such as faces, bodies, license plates, etc. in the video;

[0034] Privacy masking: Through image technology, relevant areas in videos and pictures are blurred or covered to effectively process the content, thereby ensuring that the protected data cannot be intuitively identified.

[0035] According to one aspect of an embodiment of the present invention, a method for processing image occlusion is provided. Optionally, as an optional implementation, the method for processing image occlusion can be applied to, but is not limited to, Figure 1 In the hardware environment shown, it may include but is not limited to a terminal device 102, a network 110 and a server 112. The terminal device 102 may be used to monitor a target area and obtain a target video.

[0036] The terminal device 102 may include but is not limited to: a human-computer interaction screen 104, a processor 106 and a memory 108. The human-computer interaction screen 104 is used to obtain a human-computer interaction instruction through a human-computer interaction interface, and is also used to present a first target image with an occluded area; the processor 106 is used to respond to the human-computer interaction instruction and restore the first target image. The memory 108 is used to store the first target image and the second target image. Here, the server 112 may include but is not limited to: a database 114 and a processing engine 116, the processing engine 116 is used to call the first target image and the second target image stored in the database 114; according to the second target image, at least one target area in the first target image that is occluded is removed, so as to achieve the purpose of real-time detection of the network traffic state through the target device, determining the current network state (whether it is congested), and adjusting the sent code stream data packets, thereby achieving the technical effect of determining to obtain different target images according to the network traffic state, and the target image is used to restore the occluded part of the image, thereby solving the technical problem of low efficiency in restoring the occluded part of the image in the prior art.

[0037] The specific process is as follows: the human-computer interaction screen 104 in the terminal device 102 displays a first target image with an occluded area. As in steps S102-S106, the first target image and the second target image are obtained, and the target image and the second target image are sent to the server 112 through the network 110. The server 112 removes at least one target area in the first target image that is occluded according to the second target image. Then the above determined result is returned to the terminal device 102.

[0038] Then, as in steps S102-S106, the terminal device 102 obtains a first target image sent by the target device, wherein the first target image includes at least one target area, and the target area is an area after occlusion processing; a second target image is obtained according to the network traffic status of the target device, wherein the second target image is used to remove at least one target area that is occluded in the first target image; at least one target area that is occluded in the first target image is removed according to the second target image, thereby achieving the purpose of real-time detection of the network traffic status by the target device, determining the current network status (whether it is congested), and adjusting the sent code stream data packets, thereby achieving the technical effect of determining to obtain different target images according to the network traffic status, and the target image is used to restore the occluded part of the image, thereby solving the technical problem of low efficiency in restoring the occluded part of the image in the prior art.

[0039] Optionally, in this embodiment, the above-mentioned method for processing image occlusion may be, but is not limited to, applied to the server 112 to assist the application client in restoring the occluded portion of the first target image. The above-mentioned application client may be, but is not limited to, running in the terminal device 102, and the terminal device 102 may be, but is not limited to, a mobile phone, a tablet computer, a laptop computer, a PC, or other terminal devices that support running the application client. The above-mentioned server 112 and the terminal device 102 may, but is not limited to, realize data interaction through a network, and the above-mentioned network may include, but is not limited to, a wireless network or a wired network. The wireless network includes: Bluetooth, WIFI, and other networks that realize wireless communication. The above-mentioned wired network may include, but is not limited to, a wide area network, a metropolitan area network, and a local area network. The above is only an example, and no limitation is made to this in this embodiment.

[0040] Optionally, as an optional implementation, as Figure 2 As shown, the above-mentioned method for processing image occlusion includes:

[0041] Step S202: Acquire a first target image sent by a target device, wherein the first target image includes at least one target area, and the target area is an area after occlusion processing.

[0042] Step S204: acquiring a second target image according to the network traffic status of the target device, wherein the second target image is used to remove at least one target area that is blocked in the first target image.

[0043] Step S206: removing at least one target area in the first target image that has been occluded according to the second target image.

[0044] Optionally, in this embodiment, the above scheme may include but is not limited to being applied in a video surveillance system to restore the blurred and occluded images in the surveillance screen or video, thereby facilitating users to obtain the unobstructed original images and helping users to obtain data.

[0045] Optionally, in this embodiment, the first target image may include but is not limited to a frame of image in the target video, wherein the first target image is subjected to privacy processing of personal data, that is, the first target image is a picture subjected to occlusion processing. The second target image may include units not limited to the original image of the first target image, that is, the original image before occlusion processing, and also include only images corresponding to the area subjected to occlusion processing. For example, a frame of image in the target video is image A, and after the target area C in image A is blurred and occluded, image B (first target image) is obtained. Then, the second target image may include image A, and may also include image D corresponding to only the target area C.

[0046] Optionally, in the present embodiment, obtaining the second target picture according to the network traffic status of the target device may include: when the network traffic status is in an idle state, obtaining a third target picture sent by the target device, wherein the third target picture is an original picture corresponding to the first target picture, and the second target picture includes the third target picture; when the network traffic status is in a non-idle state, obtaining a fourth target picture sent by the target device, wherein the fourth target picture is an original picture of the target area to be occluded in the first target picture, and the second target picture includes the fourth target picture.

[0047] In this embodiment, the second target image may include but is not limited to being the original image of the first target image when the network traffic is in an idle state, and being the original image of the target area that is occluded in the first target image when the network traffic is in a non-idle state.

[0048] Through the example provided by the present application, a first target image sent by a target device is obtained, wherein the first target image includes at least one target area, and the target area is an area after occlusion processing; a second target image is obtained according to the network traffic status of the target device, wherein the second target image is used to remove at least one target area that is occluded in the first target image; at least one target area that is occluded in the first target image is removed according to the second target image, thereby achieving the purpose of real-time detection of the network traffic status by the target device, determining the current network status (whether it is congested), and adjusting the sent code stream data packets, thereby achieving the technical effect of determining to obtain different target images according to the network traffic status, and the target image is used to restore the occluded part of the image, thereby solving the technical problem of low efficiency in restoring the occluded part of the image in the prior art.

[0049] Optionally, removing at least one target area in the first target image that is subjected to occlusion processing according to the second target image may include:

[0050] Obtain a target algorithm for performing occlusion processing in the first target image;

[0051] The first target picture is re-encoded according to the target algorithm and the fourth target picture, and at least one target area in the first target picture that is subjected to occlusion processing is removed.

[0052] The target algorithm may include but is not limited to an algorithm for performing occlusion processing on the original image, and then the first target image is re-encoded according to the target algorithm in combination with the fourth target image, and the occluded portion in the first target image is restored, that is, the occlusion of the first target image is eliminated. For example, the occluded portion in the first target image is re-encoded by the target algorithm in combination with the image D to obtain the first target image without occlusion, that is, the occluded image is restored.

[0053] Optionally, before obtaining the second target image according to the network traffic status of the target device, the method also includes: creating a statistical model for data packet sending of the target device; and detecting the network traffic status according to the statistical model.

[0054] Among them, the network traffic status is detected according to the statistical model, including: monitoring the data packet sending process; calculating the average time consumed by scheduling data packets to send data through the statistical model; calculating the amount of data sent each time according to the data packets sent and the time consumed for each scheduling; when the data sending amount is less than or equal to a first predetermined threshold, determining that the network status is a non-idle state.

[0055] It should be noted that, when the data transmission amount is greater than the first predetermined threshold, the network state is determined to be an idle state.

[0056] In this embodiment, the network traffic monitor monitors the network traffic status of the network where the device is located in real time through traffic detection and reports it.

[0057] Traffic detection can be implemented in many ways, including but not limited to: 1) collecting statistics through routers and switches and notifying the device; 2) using an external optical splitter to detect the traffic of the device and notify the device; and 3) detecting the network status by creating a statistical model for sending data packets. The statistical model workflow is as follows:

[0058] 1. Monitor the data packet sending process;

[0059] 2. Statistics on the average time it takes to send data;

[0060] 3. Calculate the amount of data sent each time based on the data packets sent and the time consumed for each scheduling;

[0061] 4. Determine the data transmission volume. If the data transmission volume is less than the threshold, it means that the current network is congested (i.e., not idle);

[0062] In this embodiment, a threshold regulator is added, and the Nash equilibrium idea in game theory is adopted to adjust the threshold according to the source data strategy to achieve the best equilibrium state. The specific ideas are as follows:

[0063] Network traffic is an important resource for devices. Since devices need to interact with other devices in the network, we hope to set the threshold as low as possible to spare more network resources for other services. The privacy masking strategy is to ensure the performance of the back-end intelligent analysis server, so we hope to transmit it to the back-end in the form of original images, thereby reducing the performance consumed by the intelligent analysis server to restore data.

[0064] As an optional embodiment, the present application also provides a real-time optimization method for personal data privacy shielding based on network status detection. Figure 3 As shown, a flowchart of a real-time optimization method for personal data privacy shielding based on network status detection.

[0065] Step S31, start;

[0066] Step S32, detecting the network status;

[0067] Step S33, determining whether the network is congested, if yes, executing step S34, if no, executing step S35;

[0068] Step S34, sending the cutout data as source data to a code stream sending module;

[0069] Step S35, sending the original image as source data to the code stream sending module;

[0070] Step S36, sending the source data and the processed occlusion image to the destination address;

[0071] Step S37, end.

[0072] In this embodiment, the network traffic monitor monitors the network status in real time, determines whether the current network is congested through a relevant judgment algorithm, and sends the relevant status to the privacy masking strategy scheduler; the privacy masking strategy scheduler receives the network status, and if the network status is displayed as busy or congested, it controls the AI ​​algorithm to obtain the cutout data of the personal data-related area, and then sends it to the code stream sending module; if the network status is displayed as idle, the original image data is sent to the code stream sending module; after the code stream sending module obtains the data sent by the privacy masking strategy scheduler, it protects the data (such as encryption protection based on cryptographic technology), and then puts the data into the auxiliary frame of the code stream, and sends it to the destination address together with the privacy masked data.

[0073] It should be noted that if Figure 4 The structural diagram of the real-time optimization method for personal data privacy shielding based on network status detection is shown in FIG. The structural diagram of the real-time optimization method for personal data privacy shielding based on network status detection is used to implement the real-time optimization method for personal data privacy shielding based on network status detection.

[0074] The network traffic detector 41 is used to monitor the network traffic status of the network where the device is currently located in real time and notify the privacy shielding strategy scheduler.

[0075] The network traffic monitor 41 monitors the network status of the device in real time through traffic detection and reports it.

[0076] Traffic detection can be implemented in many ways, including but not limited to: 1) collecting statistics through routers and switches and notifying the device; 2) using an external optical splitter to detect the traffic of the device and notify the device. It also includes 3) creating a statistical model for data packet transmission to detect the network status. The general workflow of the statistical model is as follows:

[0077] 1. Monitor the data packet sending process;

[0078] 2. Statistics on the average time it takes to send data;

[0079] 3. Calculate the amount of data sent each time based on the data packets sent and the time consumed for each scheduling;

[0080] 4. Determine the data transmission volume. If the data transmission volume is less than the threshold, it means that the current network is congested (i.e., not idle);

[0081] Due to the different strategies of video bitstream resolution and source data in privacy masking, the threshold used to judge the current network status will change according to the strategy, resulting in the inability to set the threshold in a preset way. In this embodiment, a threshold regulator is added, and the Nash equilibrium idea in game theory is adopted to adjust the threshold according to the source data strategy to achieve the optimal equilibrium state. The specific ideas are as follows:

[0082] Network traffic is an important resource for devices. Since devices need to interact with other devices in the network, we hope to set the threshold as low as possible to spare more network resources for other services. The privacy shielding strategy is to ensure the performance of the backend intelligent analysis server, so we hope to transmit it to the backend in the form of the original image, thereby reducing the performance consumed by the intelligent analysis server to restore the data. The specific process is as follows:

[0083] Step 1: adjust the privacy masking strategy to the original image mode, calculate the amount of data sent, and record it;

[0084] Step 2: adjust the privacy masking strategy to the cutout mode, calculate the data transmission amount, and record it;

[0085] Step 3: Use the sending amount of steps 1 and 2 as the threshold interval, and set the threshold to the sending amount of step 2; adjust the privacy masking strategy to the original image mode;

[0086] Step 4: Increase the threshold value, record the data transmission volume, and compare it with that in step 2. If the change is not obvious, until the threshold is increased, the transmission volume changes significantly, indicating that the equilibrium point has been reached;

[0087] In step 5, the threshold in step 4 is used as the final threshold.

[0088] After acquiring the current network status, the privacy shielding strategy scheduler 43 adjusts the privacy shielding strategy according to the network status and sends the processed data to the code stream sending module.

[0089] Among them, the privacy shielding strategy scheduler 43 mainly adjusts the AI ​​algorithm dynamically according to the scheduling strategy based on the network traffic status, so that the transmitted data is optimized under the current network status.

[0090] The source data strategy includes the following two methods:

[0091] Method 1: Personal data privacy area cutout data (relative to the fourth target image): This data includes the following aspects: the coordinates of the starting point of the area, the length and width of the area, and the specific YUV data in the area. When the user receives this data, combined with the image after privacy masking, it can be restored to the original image containing personal data by re-encoding. Re-encoding requires a certain amount of computing performance. Depending on the amount of personal data in a picture, the computing performance consumed by encoding and restoration is different (the larger the amount of data, the more performance is consumed). The amount of data transmitted over the network in method 1 is small.

[0092] Method 2: Original image data (relative to the third target image): The original image data is the original image containing personal data. The amount of original image data is large (determined by the resolution), but after obtaining the data, the original image containing personal data can be obtained without further processing. This greatly saves the computing performance of the receiving end. If it is necessary to restore the original image, the computing performance of the receiving end does not need to be consumed.

[0093] Among them, the scheduling strategy of the scheduler is as follows:

[0094] Method 1: The original image data transmission mode is adopted by default, which can save the performance of this device and the receiving end.

[0095] Method 2: When receiving a network congestion alarm from a network traffic monitor, adjust the strategy and call the AI ​​algorithm to analyze and obtain relevant data in the personal data privacy area (if the data has been obtained when processing blurred or occluded images in the privacy area, it can be reused directly).

[0096] Method 3: If you receive a message from the network traffic monitor that the network is unobstructed, restore to the original image data transmission mode.

[0097] The code stream sending module 45 sends the video data to the destination address according to the received information.

[0098] After receiving the data sent by the privacy masking strategy scheduler, the code stream sending module 45 performs protection processing on the data (such as encryption processing based on cryptographic technology), places the data in an auxiliary frame of the code stream, and sends it to the destination address in packets.

[0099] The encryption processing based on cryptographic technology can use the channel encryption technology of TLS, or use symmetric encryption (such as AES256) to symmetrically encrypt data.

[0100] Through the embodiments provided by the present application, the device detects the network in real time and determines the current network status (whether it is congested) to adjust the presentation scheme of the source data in the code stream provided to the outside, thereby adjusting the sent code stream data packets to achieve the purpose of optimization. In other words, by detecting the network status, the personal data privacy shielding processing strategy is adjusted and optimized in real time, thereby achieving the purpose of improving the efficiency of network transmission and intelligent identification of the back-end server and ensuring the computing performance of the device.

[0101] Compared with the prior art, the embodiments provided in this application have the following advantages:

[0102] 1) Based on the statistical model of data packet transmission, the network status of the network where the device is located is detected. It can be detected by itself without using third-party equipment, thus ensuring the independence and ease of use of the equipment and functions, and reducing project investment costs.

[0103] 2) Use the privacy masking policy scheduler to perform real-time policy scheduling based on the network status, with the goal of "traffic for performance" to ensure device operation performance.

[0104] 3) Through the device’s own AI analysis capabilities, personal data privacy areas are identified, reducing external dependence, speeding up analysis timeliness, and saving project costs.

[0105] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0106] According to another aspect of an embodiment of the present invention, a device for processing image occlusion for implementing the above-mentioned image occlusion processing method is also provided. Figure 5 As shown, the image occlusion processing device includes: a first acquisition unit 51, a second acquisition unit 53 and a removal unit 55.

[0107] The first acquisition unit 51 is configured to acquire a first target image sent by a target device, wherein the first target image includes at least one target area, and the target area is an area after occlusion processing.

[0108] The second acquisition unit 53 is used to acquire a second target image according to the network traffic state of the target device, wherein the second target image is used to remove at least one target area that is blocked in the first target image.

[0109] The removing unit 55 is configured to remove at least one target area in the first target image that has been occluded according to the second target image.

[0110] Optionally, the second acquisition unit 53 may include: a first acquisition module, used to acquire a third target picture sent by the target device when the network traffic status is in an idle state, wherein the third target picture is an original picture corresponding to the first target picture, and the second target picture includes the third target picture; a second acquisition module, used to acquire a fourth target picture sent by the target device when the network traffic status is in a non-idle state, wherein the fourth target picture is an original picture of the target area to be occluded in the first target picture, and the second target picture includes the fourth target picture.

[0111] Optionally, the above-mentioned removal unit 55 may include: a third acquisition module, used to obtain a target algorithm for performing occlusion processing in the first target image; a removal module, used to re-encode the first target image according to the target algorithm and the fourth target image, and remove at least one target area for performing occlusion processing in the first target image.

[0112] Through the embodiment provided by the present application, the first acquisition unit 51 acquires the first target image sent by the target device, wherein the first target image includes at least one target area, and the target area is the area after occlusion processing; the second acquisition unit 53 acquires the second target image according to the network traffic state of the target device, wherein the second target image is used to remove at least one target area that is occluded in the first target image; the removal unit 55 removes at least one target area that is occluded in the first target image according to the second target image. The purpose of real-time detection of the network traffic state by the target device, determining the current network state (whether it is congested), and adjusting the sent code stream data packets is achieved, thereby achieving the technical effect of determining to obtain different target images according to the network traffic state, and the target image is used to restore the occluded part of the image, thereby solving the technical problem of low efficiency in restoring the occluded part of the image in the prior art.

[0113] Optionally, the above-mentioned device may also include: a creation unit, used to create a statistical model of data packet sending of the target device before obtaining the second target image according to the network traffic status of the target device; and a detection unit, used to detect the network traffic status according to the statistical model.

[0114] Among them, the detection unit may include: a monitoring module, used to monitor the data packet sending process; a statistical module, used to calculate the average time consumed by scheduling data packets to send data through a statistical model; a calculation module, used to calculate the amount of data sent each time based on the data packets sent and the time consumed for each scheduling; a first determination module, used to determine that the network state is a non-idle state when the data transmission amount is less than or equal to a first predetermined threshold.

[0115] Optionally, the above device may further include: a second determination module, configured to determine that the network state is an idle state when the data transmission amount is greater than a first predetermined threshold.

[0116] According to another aspect of the embodiments of the present invention, an electronic device for implementing the above-mentioned image occlusion processing method is also provided. Figure 6 As shown, the electronic device includes a memory 602 and a processor 604. The memory 602 stores a computer program, and the processor 604 is configured to execute the steps in any of the above method embodiments through the computer program.

[0117] Optionally, in this embodiment, the electronic device may be located in at least one network device among a plurality of network devices of a computer network.

[0118] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0119] S1, obtaining a first target image sent by a target device, wherein the first target image includes at least one target area, and the target area is an area after occlusion processing;

[0120] S2, acquiring a second target image according to the network traffic state of the target device, wherein the second target image is used to remove at least one target area that is blocked in the first target image;

[0121] S3, removing at least one target area that has been occluded in the first target image according to the second target image.

[0122] Alternatively, a person skilled in the art may understand that: Figure 6 The structure shown is for illustration only, and the electronic device may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (MID), a PAD, or other terminal devices. Figure 6 The structure of the electronic device is not limited. Figure 6 More or fewer components (such as network interfaces, etc.) as shown in, or with Figure 6 Different configurations are shown.

[0123] Among them, the memory 602 can be used to store software programs and modules, such as the program instructions / modules corresponding to the method and device for processing image occlusion in the embodiment of the present invention. The processor 604 executes various functional applications and data processing by running the software programs and modules stored in the memory 602, that is, realizing the above-mentioned method for processing image occlusion. The memory 602 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 602 may further include a memory remotely arranged relative to the processor 604, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Among them, the memory 602 can be specifically, but not limited to, used for information such as the first target image, the second target image, etc. As an example, such as Figure 6 As shown, the memory 602 may include, but is not limited to, the first acquisition unit 51, the second acquisition unit 53, and the removal unit 55 in the image occlusion processing device. In addition, it may also include, but is not limited to, other module units in the image occlusion processing device, which will not be repeated in this example.

[0124] Optionally, the transmission device 606 is used to receive or send data via a network. Specific examples of the network may include a wired network and a wireless network. In one example, the transmission device 606 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices and routers via a network cable so as to communicate with the Internet or a local area network. In one example, the transmission device 606 is a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0125] In addition, the electronic device further comprises: a display 608 for displaying the first target image to be processed; and a connection bus 610 for connecting various module components in the electronic device.

[0126] According to another aspect of the embodiments of the present invention, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when run.

[0127] Optionally, in this embodiment, the computer-readable storage medium may be configured to store a computer program for performing the following steps:

[0128] S1, obtaining a first target image sent by a target device, wherein the first target image includes at least one target area, and the target area is an area after occlusion processing;

[0129] S2, acquiring a second target image according to the network traffic state of the target device, wherein the second target image is used to remove at least one target area that is blocked in the first target image;

[0130] S3, removing at least one target area that has been occluded in the first target image according to the second target image.

[0131] Optionally, in this embodiment, a person of ordinary skill in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing hardware related to the terminal device through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0132] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0133] If the integrated units in the above embodiments are implemented in the form of software functional units and sold or used as independent products, they can be stored in the above computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling one or more computer devices (which can be personal computers, servers or network devices, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention.

[0134] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0135] In the several embodiments provided in the present application, it should be understood that the disclosed client can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0136] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0137] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0138] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for processing image occlusion, characterized in that: include: Acquire a first target image sent by a target device, wherein the first target image includes at least one target area, and the target area is an area after occlusion processing; Acquire a second target image according to the network traffic state of the target device, wherein the second target image is used to remove the at least one target area that is blocked in the first target image; removing the at least one target area in the first target image that is subjected to occlusion processing according to the second target image; The acquiring of the second target image according to the network traffic state of the target device comprises: when the network traffic state is in an idle state, acquiring a third target image sent by the target device, wherein the third target image is an original image corresponding to the first target image, and the second target image includes the third target image; when the network traffic state is in a non-idle state, acquiring a fourth target image sent by the target device, wherein the fourth target image is an original image corresponding to a target area to be occluded in the first target image, and the second target image includes the fourth target image; The method of removing the at least one target area that has been occluded in the first target image according to the second target image comprises: obtaining a target algorithm that has been occluded in the first target image; and re-encoding the first target image according to the target algorithm and the fourth target image to remove the at least one target area that has been occluded in the first target image.

2. The method according to claim 1, characterized in that Before acquiring the second target image according to the network traffic status of the target device, the method further includes: Creating a statistical model of data packet transmission of the target device; The network traffic status is detected according to the statistical model.

3. The method according to claim 2, characterized in that The detecting the network traffic state according to the statistical model comprises: Monitoring the data packet sending process; The average time taken to send data of the data packet is scheduled by using the statistical model; Calculate the amount of data sent each time according to the data packets sent and the time consumed in each scheduling; When the data transmission amount is less than or equal to a first predetermined threshold, it is determined that the network traffic state is a non-idle state.

4. The method according to claim 3, characterized in that The method comprises: When the data transmission amount is greater than the first predetermined threshold, it is determined that the network traffic state is an idle state.

5. A device for processing image occlusion, characterized in that: include: A first acquisition unit is used to acquire a first target image sent by a target device, wherein the first target image includes at least one target area, and the target area is an area after occlusion processing; A second acquisition unit, configured to acquire a second target image according to a network traffic state of the target device, wherein the second target image is used to remove the at least one target area that is blocked in the first target image; A removing unit, configured to remove the at least one target area subjected to occlusion processing in the first target image according to the second target image; The second acquisition unit includes: a first acquisition module, which is used to acquire, when the network traffic state is in an idle state, a third target picture sent by the target device, wherein the third target picture is an original picture corresponding to the first target picture, and the second target picture includes the third target picture; a second acquisition module, which is used to acquire, when the network traffic state is in a non-idle state, a fourth target picture sent by the target device, wherein the fourth target picture is an original picture corresponding to a target area to be occluded in the first target picture, and the second target picture includes the fourth target picture; The removal unit includes: a third acquisition module, used to obtain a target algorithm for performing occlusion processing in the first target image; and a removal module, used to re-encode the first target image according to the target algorithm and the fourth target image, and remove the at least one target area that is occluded in the first target image.

6. The device according to claim 5, characterized in that The device also includes: A creating unit, configured to create a statistical model of data packet transmission of the target device before acquiring the second target image according to the network traffic state of the target device; A detection unit is used to detect the network traffic status according to the statistical model.

7. The device according to claim 6, characterized in that The detection unit comprises: A monitoring module, used for monitoring the data packet sending process; A statistical module, used for calculating the average time taken to send data of the data packet through the statistical model; A calculation module, used to calculate the amount of data sent each time according to the data packets sent and the time consumed for each scheduling; The first determination module is used to determine that the network traffic state is a non-idle state when the data transmission amount is less than or equal to a first predetermined threshold.

8. The device according to claim 7, characterized in that The device comprises: The second determination module is used to determine that the network traffic state is an idle state when the data transmission amount is greater than the first predetermined threshold.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein the program executes the method described in any one of claims 1 to 4 when executed.

10. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 4 through the computer program.

Images