Mobile terminal manual matting method and system and target searching method and system

Abstract

The embodiment of the invention provides a mobile terminal manual matting method and system and a target searching method and system. The manual matting method for the mobile terminal comprises the following steps: in response to a selection operation on a candidate matting picture, performing adaptive compression processing on the candidate matting picture corresponding to the selection operation to obtain a compressed picture; and sending the compressed picture to a preset server, so that the server performs matting target detection on the compressed picture to obtain coordinates of an initial matting region corresponding to a matting target, sending the coordinates of the initial matting region to a client, initializing the matting region based on the coordinates of the initial matting region by the client, and sending the initial matting region to the client. And then carrying out matting operation on the compressed picture to obtain a matted picture. According to the method, the picture data volume transmitted between the client and the server is reduced, the matting response time is shortened, and the storage resources occupied by the client are reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a mobile terminal manual matting method, a mobile terminal manual matting system, a target search method, a target search system, an electronic device, a storage medium and a computer program product. BACKGROUND

[0002] In the application scenario of mobile terminal manual matting, the existing matting technical solution has many problems, for example, the user cannot smoothly perform manual zooming and moving of the matting area, and the search accuracy is poor. For another example, the picture processing speed is slow, the memory occupation is large, and the response time is long, which leads to low performance of the matting application or the application based on the matting result. The above problems seriously affect the user experience of manual matting on the mobile terminal.

[0003] It can be seen that the mobile terminal manual matting method in the prior art still needs to be improved. SUMMARY

[0004] The embodiments of the present application provide a mobile terminal manual matting method, which can at least effectively shorten the operation response speed of the mobile terminal manual matting, improve the user experience and improve the performance of the matting application.

[0005] Correspondingly, the embodiments of the present application also provide a mobile terminal manual matting system, a target search method, a target search system, an electronic device, a storage medium and a computer program product to ensure the implementation and application of the above-mentioned mobile terminal manual matting method.

[0006] In order to solve the above problems, the embodiments of the present application disclose a mobile terminal manual matting method applied to a client, the method comprising:

[0007] In response to a selection operation on a candidate matting picture, performing adaptive compression processing on the candidate matting picture targeted by the selection operation to obtain a compressed picture;

[0008] Sending the compressed picture to a preset server, so that the server performs matting target detection on the compressed picture to obtain the coordinates of an initial matting area corresponding to the matting target, and sends the coordinates of the initial matting area to the client;

[0009] In response to receiving the coordinates of the initial matting area, initializing a matting area based on the coordinates of the initial matting area, and then performing a matting operation on the compressed picture.

[0010] The embodiments of the present application disclose a mobile terminal manual matting method applied to a server, the method comprising:

[0011] In response to receiving the compressed picture sent by the client, performing cutout target detection on the compressed picture to obtain coordinates of an initial cutout region corresponding to the cutout target, wherein the compressed picture is obtained by the client performing adaptive compression processing on a candidate cutout picture in response to a selection operation on the candidate cutout picture;

[0012] Sending the coordinates of the initial cutout region to the client, so that the client initializes a cutout region based on the coordinates of the initial cutout region after receiving the coordinates of the initial cutout region, and then performs a cutout operation on the compressed picture.

[0013] Embodiments of the present application disclose a target search method applied to a client, the method comprising:

[0014] In response to a selection operation on a candidate cutout picture, performing adaptive compression processing on the candidate cutout picture targeted by the selection operation to obtain a compressed picture;

[0015] Sending the compressed picture to a preset server, so that the server performs cutout target detection on the compressed picture to obtain coordinates of an initial cutout region corresponding to the cutout target, and sends the coordinates of the initial cutout region to the client;

[0016] In response to receiving the coordinates of the initial cutout region, displaying the compressed picture in a cutout preview window, and based on the coordinates of the initial cutout region, highlighting and displaying a cutout region in a cutout interface;

[0017] In response to an adjustment operation on the displayed cutout region, triggering the server to perform cutout and search processing on the compressed picture based on the adjusted cutout region to obtain search results corresponding to the picture content in the adjusted cutout region;

[0018] Refreshing and displaying the search results in a search result list window in real time.

[0019] Embodiments of the present application disclose a target search method applied to a server, the method comprising:

[0020] In response to receiving the compressed picture sent by the client, performing cutout target detection on the compressed picture to obtain coordinates of an initial cutout region corresponding to the cutout target, wherein the compressed picture is obtained by the client performing adaptive compression processing on a candidate cutout picture in response to a selection operation on the candidate cutout picture;

[0021] send the client the coordinates of the initial matting region, so that the client, in response to receiving the coordinates of the initial matting region, displays the compressed picture in a matting preview window and highlights a matting region in a matting interface based on the coordinates of the initial matting region;

[0022] In response to receiving the coordinates of the adjusted matting region for the compressed picture, perform a matting operation on the compressed picture based on the coordinates of the adjusted matting region to obtain a cut-out picture;

[0023] Perform a search operation based on the cut-out picture to obtain a search result;

[0024] Send the client the search result, so that the client, in response to receiving the search result, displays the search result in a search result list window in real time.

[0025] The embodiment of the present application further discloses a mobile terminal manual matting system, comprising a client and a server, the client is used for executing the mobile terminal manual matting method applied to the client as described in the embodiment of the present application, and the server is used for executing the mobile terminal manual matting method applied to the server as described in the embodiment of the present application.

[0026] The embodiment of the present application further discloses a target search system, comprising a client and a server, the client is used for executing the target search method applied to the client as described in the embodiment of the present application, and the server is used for executing the target search method applied to the server as described in the embodiment of the present application.

[0027] The embodiment of the present application further discloses an electronic device, comprising a processor and a memory connected with the processor in communication;

[0028] The memory stores computer execution instructions;

[0029] The processor executes the computer execution instructions stored in the memory to realize the method as described in the embodiment of the present application.

[0030] The embodiment of the present application further discloses a computer readable storage medium, the computer readable storage medium stores computer execution instructions, and the computer execution instructions are executed by a processor to realize the method as described in the embodiment of the present application.

[0031] The embodiment of the present application further discloses a computer program product, comprising computer program / computer executable instructions, and the computer program / computer executable instructions are executed by a processor in an electronic device to realize the method as described in the embodiment of the present application.

[0032] Compared with the prior art, the embodiment of the present application has the following advantages:

[0033] After obtaining the selection operation on the candidate cutout picture, the candidate cutout picture to which the selection operation is performed is adaptively compressed to obtain a compressed picture, and the compressed picture is sent to a preset server, so that the server detects a cutout target of the compressed picture to obtain coordinates of an initial cutout region corresponding to the cutout target, and sends the coordinates of the initial cutout region to the client, so that the client initializes a cutout region based on the coordinates of the initial cutout region, and then performs a cutout operation on the compressed picture. The method effectively reduces the amount of picture data transmitted between the client and the server, shortens the cutout response time, and can reduce the amount of system resources occupied by the client for displaying and storing the cutout picture. On the other hand, the method can reduce the data processing amount of the server for performing cutout target detection, improve the cutout processing speed, and improve the performance of the cutout application. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1 is one of the step flowcharts of the mobile terminal manual cutout method disclosed by the embodiments of the present application;

[0035] Figure 2 is another of the step flowcharts of the mobile terminal manual cutout method disclosed by the embodiments of the present application;

[0036] Figure 3 is a cutout interface display effect schematic diagram when the mobile terminal manual cutout method disclosed by the embodiments of the present application is implemented;

[0037] Figure 4 is a third of the step flowcharts of the mobile terminal manual cutout method disclosed by the embodiments of the present application;

[0038] Figure 5 is an interactive schematic diagram of the mobile terminal manual cutout system disclosed by the embodiments of the present application;

[0039] Figure 6 is one of the step flowcharts of the target search method disclosed by the embodiments of the present application;

[0040] Figure 7 is a search result display effect schematic diagram when the target search method disclosed by the embodiments of the present application is implemented;

[0041] Figure 8 is another of the step flowcharts of the target search method disclosed by the embodiments of the present application;

[0042] Figure 9 is an interactive schematic diagram of the target search system disclosed by the embodiments of the present application;

[0043] Figure 10 is a structural schematic diagram of an exemplary device provided by one of the embodiments of the present application. DETAILED DESCRIPTION

[0044] In order to make the above objectives, characteristics and advantages of the present application more apparent, further specific embodiments of the present application will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0045] The mobile terminal manual matting method disclosed by the embodiments of the present application is applied to the client of a matting system. As shown in the figure, the mobile terminal manual matting method comprises steps 103 to 105. Figure 1

[0046] Step 103: In response to a selection operation on a candidate matting picture, the candidate matting picture targeted by the selection operation is subjected to adaptive compression processing to obtain a compressed picture.

[0047] Taking the application of the mobile terminal manual matting method disclosed by the embodiments of the present application to a matting application as an example, after a user starts the matting application, the matting application first displays a picture list, and the user can select a candidate matting picture in the picture list. The pictures displayed in the picture list are pictures stored locally in the mobile terminal, including but not limited to pictures in the photo album and pictures received through an information transmission application and stored locally.

[0048] Optionally, as shown in the figure, before the step of responding to the selection operation on the candidate matting picture and subjecting the candidate matting picture targeted by the selection operation to adaptive compression processing to obtain a compressed picture, the method further comprises steps 100, 101 and 102. Figure 2

[0049] Step 100: Predict a candidate matting picture based on one or more of the following information: the current user's matting application operation habit, the usage frequency of pictures in the mobile terminal, and the access time of pictures in the mobile terminal.

[0050] For example, by analyzing the picture content type of the matting pictures historically selected by the current user, the type of the candidate matting picture can be predicted. To be specific, if the pictures historically selected by the current user are mostly pictures of people, it can be predicted that a picture of a person is a candidate matting picture.

[0051] For example, the usage frequency of pictures in the mobile terminal can be obtained by calling the interface of the system album in the mobile terminal. Optionally, pictures with high usage frequency can be predicted as candidate matting pictures.

[0052] For example, pictures with the most recent access time can be predicted as candidate matting pictures. ​​

[0053] In the embodiments of the present application, the predicted candidate cutout pictures can include multiple pictures for the user to select.

[0054] In step 101, the candidate cutout pictures are loaded into the local cache of the mobile terminal.

[0055] After the candidate cutout pictures are predicted, the candidate cutout pictures are loaded into the local cache of the mobile terminal as pictures that can be selected by the user. Optionally, a thumbnail or a low-resolution version of the candidate cutout pictures can be loaded into the local cache to prompt the loading speed of the function pictures.

[0056] In step 102, the candidate cutout pictures are preferentially read from the local cache, and the read candidate cutout pictures are displayed.

[0057] Optionally, preferentially reading the candidate cutout pictures from the local cache and displaying the read candidate cutout pictures includes: reading the candidate cutout pictures from the local cache and displaying them at the head position of the picture list; and reading the candidate cutout pictures from a preset picture acquisition channel and displaying them after the head position of the picture list. The preset picture acquisition channel includes one or more of the following: the photo album of the mobile terminal, the downloaded pictures of the communication software, and the cloud.

[0058] Optionally, when the candidate cutout pictures are read from the local cache and displayed at the head position of the picture list, the pictures can be displayed in order of the frequency of use of the pictures from high to low, or in order of the access time of the pictures from recent to old, from the head to the end.

[0059] By predicting the candidate cutout pictures based on the cutout application operation habits of the current user and / or the frequency of use of the pictures in the mobile terminal, and loading the candidate cutout pictures into the local cache of the mobile terminal, when displaying the picture list, the candidate cutout pictures are preferentially read from the local cache, which not only improves the response speed of the cutout application, but also facilitates the user to quickly select the candidate cutout pictures by displaying the candidate cutout pictures read from the local cache at the head position of the picture list, thereby improving the user experience.

[0060] In some optional embodiments, after the candidate cutout pictures are loaded into the local cache of the mobile terminal, the method further includes: monitoring the occupation of the local cache used to load the candidate cutout pictures in real time; and cleaning the candidate cutout pictures loaded into the local cache when the occupation of the local cache meets a preset condition.

[0061] Optionally, the occupation of the local cache for loading the candidate cutout picture is monitored in real time using the prior art. When the occupation size of the local cache is monitored to reach a preset cache cleaning threshold, the candidate cutout pictures stored in the local cache can be automatically cleaned, thereby relieving the cache pressure of the mobile terminal and ensuring the system stability.

[0062] For example, the LRU (Least Recently Used) algorithm can be used to cache the pictures that have been used recently and / or the pictures with the highest frequency in the memory. When the user accesses these pictures again, the pictures can be directly read from the cache, the picture loading time is reduced, and the response speed of the cutout application is improved. On the other hand, for memory usage, the memory occupation of the cutout application can be monitored in real time. When the memory pressure increases, the pictures that have not been used for a long time in the local cache are automatically cleaned, and the system is ensured to operate stably. For example, when the user continuously performs cutout processing on multiple pictures, the cutout application can reasonably allocate memory resources to ensure that the loading and processing of each picture can be carried out smoothly.

[0063] After the candidate cutout picture is selected in response to the selection operation, the client first performs picture compression processing on the selected candidate cutout picture to reduce the size of the picture sent to the server, thereby reducing the amount of picture data sent to the server and improving the response speed of the server.

[0064] Optionally, the candidate cutout picture to which the selection operation is directed is adaptively compressed to obtain a compressed picture, including: obtaining picture information of the candidate cutout picture to which the selection operation is directed, wherein the picture information includes: resolution and / or content complexity index; based on the resolution and / or the content complexity index, the candidate cutout picture is compressed to obtain a compressed picture, so that the picture size is less than or equal to a preset size threshold under the premise of guaranteeing the visual effect of the compressed picture.

[0065] Optionally, after the user selects a candidate cutout picture, the client first obtains the basic information of the selected candidate cutout picture, which includes but is not limited to one or more of the following information: size of the picture file, picture resolution, picture format, and content complexity index of the picture. The content complexity index is used to describe the amount of information and the degree of detail richness in the picture, and the content complexity index includes one or more of the following indexes: color change of picture content, texture complexity, and edge number. The specific method for obtaining the content complexity index is known in the art, and will not be described herein.

[0066] In implementation, a high complexity picture requires higher resolution and more refined compression parameters to maintain quality. After obtaining the resolution and content complexity index of the candidate picture selected by the user, further, if the size of the candidate picture exceeds a preset size threshold (e.g., 1.5 MB), the client compresses the candidate picture according to the resolution and / or the content complexity index of the candidate picture to obtain a compressed picture.

[0067] Optionally, the compression of the candidate picture based on the resolution and / or the content complexity index to obtain a compressed picture comprises: dividing the candidate picture into picture regions corresponding to different content complexity based on the content complexity index; setting a compression quality parameter corresponding to each picture region based on the content complexity corresponding to the picture region, and performing regional compression on the corresponding picture region based on the compression quality parameter to obtain a compressed picture.

[0068] In implementation, the candidate picture can be divided into picture regions with different content complexity based on the content complexity index using existing technology. For example, for a candidate picture showing a hand holding a bag, the hand holding the bag can be divided into a picture region, and the background can be divided into one or more picture regions. For example, for a person outdoor image, the sky can be divided into a picture region, the person can be divided into a picture region, and other backgrounds can be divided into one or more picture regions. Then, for picture regions with lower content complexity, such as the picture regions obtained by dividing the background as described above, the resolution can be compressed to a lower resolution, and for picture regions with higher content complexity, such as the picture regions corresponding to the hand holding the bag and the person as described above, the resolution can be compressed to a higher resolution.

[0069] Specifically, for picture regions with higher current resolution and rich content details, the client will moderately reduce the resolution, while finely adjusting the JPEG (Joint Photographic Experts Group, a picture file format) quantization table, DCT (Discrete Cosine Transform) coefficients and other compression quality parameters. For picture regions with higher current resolution and single content details, the client will significantly reduce the resolution, while finely adjusting the compression quality parameters of the JPEG format, so as to control the file size within a reasonable range while ensuring that the key information of the picture is not lost. For example, the compressed picture is controlled to be 30%-50% of the size of the original candidate picture.

[0070] Optionally, based on the content complexity corresponding to the picture area, a compression quality parameter corresponding to each picture area is set, and a sub-area compression processing is performed on the corresponding picture area based on the compression quality parameter to obtain a compressed picture. In an embodiment of the present application, the sub-area compression processing includes: selecting picture areas in order of content complexity from low to high, and performing compression processing to obtain a current step compressed picture, until the size of the current step compressed picture is less than or equal to a preset size threshold, taking the current step compressed picture as the final compressed picture, and ending the compression. For example, when processing a landscape picture, the algorithm will preferentially compress picture areas with relatively simple picture content such as the sky. If the size of the compressed picture obtained by compressing the picture area with simple picture content meets the requirements, then the picture areas where key elements such as buildings and figures are located with higher content complexity will not be compressed, so that the picture size can be reduced and the details of key elements such as buildings and figures can be preserved.

[0071] Compared with the prior art, the compressed picture obtained by compressing the picture to be cut based on adaptive sub-area compression processing preserves the details of key elements, so that the cutout precision can be improved when performing cutout processing based on the compressed picture, and further, the accuracy of the search result obtained by performing a search operation based on the cutout picture can be improved.

[0072] At this point, the compression processing of the candidate cutout picture selected by the user as the picture to be cut is completed.

[0073] In step 104, the compressed picture is sent to a preset server, so that the server performs cutout target detection on the compressed picture to obtain the coordinates of the initial cutout area corresponding to the cutout target, and sends the coordinates of the initial cutout area to the client.

[0074] Next, the client sends the compressed picture to a preset server. Since the picture has been compressed, the amount of data transmitted between the client and the server can be effectively reduced.

[0075] Optionally, the client can send the compressed picture to the server by sending a cutout request to the server. In an embodiment of the present application, the specific implementation of the client sending the compressed picture to the preset server is not limited.

[0076] After receiving the compressed picture sent by the client, the server first performs matting object detection on the compressed picture to obtain the coordinates of the initial matting region corresponding to the matting object. For example, after receiving the compressed picture obtained after the client performs compression processing on the display picture of the hand-held bag, the server performs target detection on the compressed picture to obtain the coordinates of the picture region where the hand-held bag is located. Then, the server takes the picture region where the hand-held bag is located in the compressed picture as the initial matting region, and sends the coordinates of the initial matting region to the client.

[0077] In specific implementation, the server takes the image subject as the matting object, performs automatic image subject object recognition on the compressed picture according to a preset strategy, and obtains the image region where the image subject object is located. The server performs image subject object recognition on the compressed picture to obtain the image region where the matting object (i.e., the image subject object) is located as the initial matting region, which not only improves the efficiency of the picture search application, but also helps to improve the accuracy of the picture search recall result. The specific implementation of the server performing automatic image subject object recognition on the compressed picture to obtain the image region where the image subject object is located is the prior art, and will not be described herein. The specific implementation of the server performing matting object detection on the compressed picture to obtain the coordinates of the initial matting region corresponding to the matting object is not limited in the embodiments of the present application.

[0078] Optionally, the initial matting region is the picture region where the matting object is located, such as the picture region where the positioning box of the matting object in the compressed picture is located.

[0079] In some optional embodiments, when the compressed picture includes multiple matting objects, the initial matting region can be the picture region where the positioning box with the highest confidence is located. In step 105, in response to receiving the coordinates of the initial matting region, the matting region is initialized based on the coordinates of the initial matting region, and then the matting operation is performed on the compressed picture.

[0080] After receiving the coordinates of the initial matting region sent by the server, the client initializes the matting region based on the coordinates of the initial matting region to obtain the initial matting region. Then, based on the initial matting region, the new matting region is determined according to the user's adjustment of the matting region, and the matting operation is performed on the compressed picture based on the new matting region.

[0081] Optionally, initializing the matting region based on the coordinates of the initial matting region and then performing the matting operation on the compressed picture includes sub-step 1051 and sub-step 1052.

[0082] Sub-step 1051, displaying the compressed picture in the matting preview window, and highlighting the matting region in the matting interface based on the coordinates of the initial matting region.

[0083] After receiving the coordinates of the matting region sent by the server, the client displays a matting preview window in the matting interface, displays the compressed picture in the matting preview window, and highlights the matting region in the matting interface based on the coordinates of the matting region. The display effect is as shown in Figure 3

[0084] Optionally, the real-time highlighting of the matting region includes one or more of the following: flashing the positioning box of the matting region, reducing the brightness of the picture content outside the matting region, and increasing the brightness of the picture content in the matting region.

[0085] Sub-step 1052, in response to the adjustment operation on the displayed matting region, triggering the server to perform a matting operation on the compressed picture based on the adjusted matting region to obtain a cut-out picture.

[0086] Next, the user can manually adjust the matting region displayed by the client, and the matting application will perform a matting operation in real time according to the adjusted matting region.

[0087] Optionally, in response to the adjustment operation on the displayed matting region, triggering the server to perform a matting operation on the compressed picture based on the adjusted matting region to obtain a cut-out picture, includes: in response to the adjustment operation on the matting region, obtaining the coordinates of the adjusted matting region; based on the coordinates of the adjusted matting region, refreshing the display of the adjusted matting region in the matting interface in real time, and sending the coordinates of the adjusted matting region to the server in real time, so that the server performs a matting operation on the compressed picture based on the coordinates of the adjusted matting region to obtain a cut-out picture.

[0088] The cut-out picture is a picture of the matting target in the compressed picture cut out from the compressed picture, for example, the picture of the hand-held bag detected in the foregoing. Optionally, the cut-out picture can include only all picture content (including the matting target and the background) in the adjusted matting region, or can include only the picture content covered by the matting target in the adjusted matting region.

[0089] When the user selects a candidate matting picture and enters the matting interface, the matting application immediately starts the real-time preview function of the matting operation result, and the result of each step of the matting operation performed by the user, such as zooming, panning, and drawing a matting selection area, is immediately presented in the matting preview window.

[0090] ​Optionally, the adjusting operation on the matting region includes, but is not limited to, any one of the following: dragging the positioning frame of the matting region, zooming the matting region, and drawing the matting region. For example, the user can drag the positioning frame of the matting region by clicking the picture area in the matting region and moving the click position, so as to achieve the purpose of dragging the positioning frame of the matting region. For another example, the user can perform a preset action in the matting region of the candidate matting picture displayed on the matting interface by two fingers or three fingers, or perform a preset action on the positioning frame in the matting region of the candidate matting picture displayed on the matting interface by two fingers, so as to achieve the purpose of zooming in or out the matting region. In another embodiment of the present application, the matting region can also be adjusted in other ways, which will not be described in detail in the embodiment of the present application. The specific implementation of adjusting the matting region is not limited in the present application.

[0091] In some optional embodiments, after detecting the adjusting operation on the matting region, the method further includes: performing visual feedback and / or tactile feedback. For example, the positioning frame of the matting region is displayed by flashing. For another example, when zooming the matting region, the zooming effect of the matting region is displayed by using elastic animation. For another example, slight vibration feedback is performed by calling the vibration interface of the mobile terminal. By performing visual feedback and / or tactile feedback after detecting the adjusting operation on the matting region, the intuitive feeling of the user operation can be enhanced, and the user can clearly know that the operation has been received and executed by the system.

[0092] In some optional embodiments, in the matting preview window, the application can use an efficient state machine model to quickly respond to the preview state changes caused by different adjusting operations on the matting region. For example, when the user switches from zooming operation to panning operation, the state machine can quickly adjust the display logic of the matting preview window to ensure the smooth transition of the matting application processing flow.

[0093] In some optional embodiments, in order to further improve the user experience, by analyzing and classifying the errors that may occur during the running of the matting application, a plurality of error categories are defined in advance, and prompt information output for each error is defined. At the same time, an error handling system is set up to monitor and identify errors during the running of the matting application.

[0094] For example, when the network fluctuation causes the compressed picture transmission failure or the matting picture loading failure, the error processing system determines it as a network error; if the algorithm has a calculation error or data anomaly in the matting picture processing process, it is classified as a processing error; when the user input matting operation instruction does not conform to the system specification (such as the matting area exceeding the compressed picture area), it is regarded as a verification error. The error processing system provides customized user prompts for different error types. For example, for network errors, the prompt content includes detailed information of the network connection state (such as "current network signal is weak, please try to switch the network or retry later") and solution suggestions; for processing errors, the prompt content focuses on error cause explanation (such as "an error occurs in the picture processing process, which may be due to incompatible picture format, please replace the picture and retry"); for verification errors, the prompt content will guide the user to operate correctly (such as "your selected area drawing is not standard, please redraw the selected area").

[0095] After the matting application is started, the error processing system is also started to monitor the system running state and be ready to capture possible errors at any time.

[0096] For example, after the coordinates of the adjusted matting area are obtained in response to the adjustment operation on the matting area, the method further includes: determining whether the adjusted matting area exceeds the boundary of the candidate matting picture based on the coordinates of the adjusted matting area; and displaying first error prompt information in the matting preview window in a case where the adjusted matting area exceeds the boundary of the candidate matting picture.

[0097] The first error prompt information is used to indicate that the adjusted matting area exceeds the boundary of the candidate matting picture.

[0098] For example, when the compressed picture displayed in the matting preview window is a horizontal picture, the size of the matting preview window is consistent with the size of the compressed picture, the width of the matting preview window is consistent with the width of the matting interface, and the height of the matting preview window is less than the height of the matting interface. In this case, when the positioning box for selecting the matting area by moving the positioning box may move out of the matting preview window. When the error processing system monitors that this situation occurs, it is considered that the shooting function verification error occurs, and the corresponding error prompt information is input.

[0099] For example, after the compressed picture is sent to the preset server, the method further includes: obtaining a sending result of the compressed picture; and displaying second error prompt information in the matting preview window in a case where the sending result indicates a sending failure, wherein the second error prompt information is used to indicate a network communication error.

[0100] For another example, after sending the adjusted coordinates of the matting region to the server in real time, the method further includes: obtaining a sending result of the coordinates of the matting region; and in a case where the sending result indicates a sending failure, displaying second error prompt information in the matting preview window, where the second error prompt information is used to indicate a network communication error.

[0101] That is, when the error handling system monitors that the data sent by the client to the server fails to be sent, it can be considered that a network error occurs between the client and the server. In this case, the error handling system can output the error prompt information predefined by the network through the matting interface.

[0102] In some optional embodiments, the error handling system can also have an automatic recovery mechanism built in. For some recoverable errors, such as a temporary network interruption, the error handling system can automatically attempt to reconnect the network and load the picture, reducing manual intervention by the user.

[0103] By refining the error types, providing friendly prompts, and supporting automatic recovery, the matting application implementing the matting method can ensure the stability and smoothness of the matting operation.

[0104] To sum up, the mobile terminal manual matting method disclosed in the embodiments of the present application can, after obtaining a selection operation on a candidate matting picture, perform adaptive compression processing on the candidate matting picture targeted by the selection operation to obtain a compressed picture, and send the compressed picture to a preset server, so that the server performs matting target detection on the compressed picture to obtain coordinates of an initial matting region corresponding to the matting target, and sends the coordinates of the initial matting region to the client. This effectively reduces the amount of picture data transmitted between the client and the server, shortens the matting response time, and can reduce the amount of system resources occupied by the client for displaying and storing the matting picture. On the other hand, it can reduce the data processing amount of the server for performing matting target detection, improve the matting processing speed, and improve the performance of the matting application.

[0105] Further, after receiving the coordinates of the matting region sent by the server, the client displays the compressed picture in the matting preview window, highlights the matting region in the matting interface based on the coordinates of the matting region, and in response to an adjustment operation on the displayed matting region, triggers the server to perform a matting operation on the compressed picture based on the adjusted matting region to obtain a cut-out picture. The operation result of the user can be previewed in real time, which helps to improve the user experience.

[0106] Further, by performing region division processing on the candidate matting picture based on the content complexity index, the candidate matting picture is divided into picture regions with different content complexity, and then, the candidate matting picture is processed by region compression based on the content complexity of each picture region to obtain a compressed picture, so that the balance between the size and quality of the compressed picture is effectively realized, the size of the picture is reduced, and the effect of retaining the key target detail information in the picture is achieved.

[0107] On the basis of the above-mentioned embodiments, the present embodiment further provides a mobile terminal manual matting method applied to a server. As shown in Figure 4 The method comprises steps 401 and 402.

[0108] Step 401: In response to receiving a compressed picture sent by a client, performing matting target detection on the compressed picture to obtain the coordinates of an initial matting region corresponding to the matting target, wherein the compressed picture is obtained by performing adaptive compression processing on a candidate matting picture by the client in response to a selection operation on the candidate matting picture.

[0109] The specific implementation of the adaptive compression processing on the candidate matting picture by the client in response to the selection operation is described in the foregoing embodiments, which will not be repeated here.

[0110] The specific implementation of the compressed picture sent by the client is described in the foregoing embodiments, which will not be repeated here.

[0111] The specific implementation of the matting target detection on the compressed picture by the server to obtain the coordinates of the initial matting region corresponding to the matting target is described in the foregoing embodiments, which will not be repeated here.

[0112] Step 402: Sending the coordinates of the initial matting region to the client, so that the client initializes a matting region based on the coordinates of the initial matting region after receiving the coordinates of the initial matting region, and then performs a matting operation on the compressed picture.

[0113] Optionally, the initialization of the matting region based on the coordinates of the initial matting region and the subsequent matting operation on the compressed picture comprise sub-steps 4021 and 4022.

[0114] Sub-step 4021: Displaying the compressed picture in a matting preview window, and highlighting the display of the matting region in a matting interface based on the coordinates of the initial matting region.

[0115] The server sends the coordinates of the initial matting region to the client.

[0116] The specific implementation of the server sending the coordinates of the initial matting region to the client is described in the prior art, and will not be repeated here. The specific implementation of the server sending the coordinates of the initial matting region to the client in the embodiments of the present application is not limited.

[0117] The specific implementation of the client displaying the compressed picture in the matting preview window and highlighting the matting region in the matting interface based on the coordinates of the initial matting region is described in the foregoing embodiments, and will not be repeated here.

[0118] Further, the user can adjust the matting region through the client. The client obtains the coordinates of the adjusted matting region in response to the adjustment operation on the matting region, and refreshes the display of the adjusted matting region in the matting interface in real time based on the coordinates of the adjusted matting region, and sends the coordinates of the adjusted matting region to the server in real time.

[0119] The sub-step 4022, in response to receiving the coordinates of the adjusted matting region of the compressed picture, performing a matting operation on the compressed picture based on the coordinates of the adjusted matting region to obtain a cut-out picture.

[0120] The coordinates of the adjusted matting region are obtained by the client in response to the adjustment operation on the displayed matting region.

[0121] The specific implementation of the server performing a matting operation on the compressed picture based on the coordinates of the adjusted matting region to obtain a cut-out picture is described in the foregoing embodiments, and will not be repeated here.

[0122] In some optional embodiments, the server can send the cut-out picture obtained based on the adjusted matting region to the matting application of the client for display. Alternatively, the server can perform a target search operation based on the cut-out picture obtained based on the adjusted matting region to obtain a picture matching the target in the matting picture.

[0123] To sum up, the mobile terminal manual matting method disclosed in the embodiments of the present application is applied to a server. The method performs matting target detection on a compressed picture received from a client to obtain the coordinates of an initial matting region corresponding to the matting target, wherein the compressed picture is obtained by performing adaptive compression processing on a candidate matting picture corresponding to a selection operation of the client. The method effectively reduces the amount of picture data transmitted between the client and the server, shortens the matting response time, and reduces the amount of system resources occupied by the client for displaying and storing the matting picture.

[0124] Further, the server sends the coordinates of the initial matting region to the client, so that the client displays the compressed picture in a matting preview window and highlights the matting region in a matting interface based on the coordinates of the initial matting region after receiving the coordinates of the initial matting region. Then, the client performs a matting operation on the compressed picture based on the coordinates of the adjusted matting region to obtain a cut-out picture in response to receiving the coordinates of the adjusted matting region. The operation result of the user can be previewed in real time, which helps to improve the user experience.

[0125] Based on the above embodiments, the present embodiment further provides a mobile terminal manual matting system for implementing the mobile terminal manual matting described above. The mobile terminal manual matting system comprises a client and a server. The specific implementation of the mobile terminal manual matting system is described below with reference to the interaction flow shown in Figure 5

[0126] Step 501: The client performs adaptive compression processing on a candidate matting picture corresponding to a selection operation of the client to obtain a compressed picture in response to the selection operation.

[0127] Step 502: The client sends the compressed picture to the server.

[0128] Step 503: The server performs matting target detection on the compressed picture to obtain the coordinates of an initial matting region corresponding to the matting target.

[0129] Step 504: The server sends the coordinates of the initial matting region to the client.

[0130] Step 505: The client displays the compressed picture in a matting preview window and highlights the matting region in a matting interface based on the coordinates of the initial matting region in response to receiving the coordinates of the initial matting region.

[0131] ​Step 506, the client acquires the coordinates of the adjusted cutout region in response to the adjustment operation on the displayed cutout region.

[0132] Step 507, the client refreshes the display of the adjusted cutout region in real time in the cutout interface based on the coordinates of the adjusted cutout region.

[0133] Step 508, the client sends the coordinates of the adjusted cutout region to the server in real time.

[0134] Step 509, the server performs cutout operation on the compressed picture based on the coordinates of the adjusted cutout region to acquire the cutout picture.

[0135] In summary, the embodiment also provides a mobile end manual cutout system. The client performs adaptive compression on the candidate cutout picture in response to the selection operation on the candidate cutout picture to acquire a compressed picture, and sends the compressed picture to the server, which effectively reduces the amount of picture data transmitted between the client and the server, shortens the cutout response time, and reduces the amount of system resources occupied by the client for displaying and storing the cutout picture. Then, the server detects the cutout target of the compressed picture to acquire the coordinates of the initial cutout region corresponding to the cutout target, and sends the coordinates to the client. After that, the client displays the compressed picture in the cutout preview window, highlights the display of the cutout region in the cutout interface based on the coordinates of the initial cutout region, acquires the coordinates of the adjusted cutout region in response to the adjustment operation on the displayed cutout region, refreshes the display of the adjusted cutout region in real time in the cutout interface based on the coordinates of the adjusted cutout region, and sends the coordinates of the adjusted cutout region to the server in real time. Finally, the server performs cutout operation on the compressed picture based on the coordinates of the adjusted cutout region to acquire the cutout picture. The operation result of the user can be previewed in real time, which helps to improve the user experience.

[0136] On the basis of the above-mentioned embodiment, the present embodiment further provides a target searching method applied to a client. As shown in Figure 6 the method comprises steps 601 to 605.

[0137] Step 601, in response to the selection operation on the candidate cutout picture, the candidate cutout picture is adaptively compressed to obtain a compressed picture.

[0138] Step 602, the compressed picture is sent to a preset server, so that the server detects the cutout target of the compressed picture to acquire the coordinates of the initial cutout region corresponding to the cutout target, and sends the coordinates of the initial cutout region to the client.

[0139] In response to receiving the coordinates of the initial matting region, the compressed picture is displayed in the matting preview window, and the matting region is highlighted in the matting interface based on the coordinates of the initial matting region.

[0140] In response to the adjustment operation on the displayed matting region, the server is triggered to perform the matting and search processing on the compressed picture based on the adjusted matting region, and the search result corresponding to the picture content in the adjusted matting region is obtained.

[0141] In the search result list window, the search result is displayed in real time.

[0142] The specific embodiments of steps 601, 602, and 603 are described above, and will not be repeated here.

[0143] Optionally, in response to the adjustment operation on the displayed matting region, the server is triggered to perform the matting and search processing on the compressed picture based on the adjusted matting region, and the search result corresponding to the picture content in the adjusted matting region is obtained, including: in response to the adjustment operation on the matting region, obtaining the coordinates of the adjusted matting region; based on the coordinates of the adjusted matting region, the adjusted matting region is displayed in real time, and the coordinates of the adjusted matting region are sent to the server in real time, so that the server performs the matting operation on the compressed picture based on the coordinates of the adjusted matting region, obtains the cut picture, performs the search operation based on the cut picture, obtains the search result, and sends the search result to the client.

[0144] The cut picture is a picture of the matting target in the compressed picture cut from the compressed picture, for example, the picture of the hand-held bag detected above. Optionally, the cut picture can include all picture content (including the matting target and the background) in the adjusted matting region, or can include only the picture content covered by the matting target in the adjusted matting region.

[0145] In response to the adjustment operation on the matting region, the coordinates of the adjusted matting region are obtained, and the specific embodiments are described above.

[0146] Based on the coordinates of the adjusted matting region, the adjusted matting region is displayed in real time, and the coordinates of the adjusted matting region are sent to the server in real time, and the specific embodiments are described above.

[0147] After receiving the adjusted coordinates of the matting region, the server performs a real-time matting operation on the compressed picture based on the adjusted coordinates of the matting region to obtain a cut-out picture. Then, the server calls a preset search module based on the cut-out picture to perform a search operation to search for a picture similar to the picture content in the cut-out picture. Optionally, the search module can return one or more of the following information: the picture similar to the picture content in the cut-out picture, the picture similar to the picture content in the cut-out picture and target information (such as a product link or product name) associated with the picture, and the like.

[0148] For a specific implementation of calling a preset search module based on the cut-out picture to perform a search operation to search for a picture similar to the picture content in the cut-out picture, refer to the prior art, and no further description is given in the embodiments of the present application. The search results returned by the search module are determined according to the application requirements of the target search method.

[0149] Next, the server sends the search results to the client. After receiving the search results sent by the server, the client displays the search results in a search result list window in real time. In some optional embodiments, the search result display effect is as shown in Figure 7 .

[0150] Optionally, before the candidate cut-out picture is adaptively compressed to obtain a compressed picture in response to the selection operation on the candidate cut-out picture, the method further includes: predicting a candidate cut-out picture based on one or more of the following information: a current user's matting application operation habit, a use frequency of pictures in the mobile terminal, and an access time of pictures in the mobile terminal; loading the candidate cut-out picture to a local cache of the mobile terminal; and preferentially reading the candidate cut-out picture from the local cache and displaying the read candidate cut-out picture.

[0151] After the candidate cut-out picture is predicted, the candidate cut-out picture is loaded to the local cache of the mobile terminal as a picture that the user is likely to select. Optionally, a thumbnail or a low-resolution version of the candidate cut-out picture can be loaded to the local cache to prompt the loading speed of the function picture.

[0152] For a specific implementation of predicting a candidate cut-out picture based on one or more of the following information: a current user's matting application operation habit, a use frequency of pictures in the mobile terminal, and an access time of pictures in the mobile terminal, and loading the candidate cut-out picture to a local cache of the mobile terminal, preferentially reading the candidate cut-out picture from the local cache, and displaying the read candidate cut-out picture, refer to the related description in the foregoing embodiments, and no further description is given herein.

[0153] Optionally, after loading the candidate cutout picture to the local cache of the mobile terminal, the method further comprises: monitoring the occupation condition of the local cache for loading the candidate cutout picture in real time; and cleaning up the candidate cutout picture loaded to the local cache when the occupation condition of the local cache meets a preset condition.

[0154] The specific implementation of monitoring the occupation condition of the local cache for loading the candidate cutout picture in real time and cleaning up the candidate cutout picture loaded to the local cache when the occupation condition of the local cache meets a preset condition is described above in the related description of the foregoing embodiments, which is not repeated here.

[0155] The occupation condition of the local cache for loading the candidate cutout picture is monitored in real time by using the prior art. When the occupation size of the local cache is monitored to reach a preset cache cleaning threshold, the candidate cutout picture stored in the local cache can be automatically cleaned up, thereby relieving the cache pressure of the mobile terminal and ensuring the system running stability.

[0156] In summary, the target search method disclosed in the embodiment, in response to a selection operation on a candidate cutout picture, performs adaptive compression processing on the candidate cutout picture targeted by the selection operation to obtain a compressed picture; sends the compressed picture to a preset server, so that the server performs cutout target detection on the compressed picture to obtain the coordinates of an initial cutout region corresponding to the cutout target, and sends the coordinates of the initial cutout region to the client, effectively reducing the amount of picture data transmitted between the client and the server, and shortening the cutout response time; the client, in response to receiving the coordinates of the initial cutout region, displays the compressed picture in a cutout preview window, and based on the coordinates of the initial cutout region, highlights the display of the cutout region in the cutout interface, and then, in response to an adjustment operation on the displayed cutout region, triggers the server to perform cutout and search processing on the compressed picture based on the adjusted cutout region to obtain search results corresponding to the picture content in the adjusted cutout region, and displays the search results in a search result list window in real time, so that the user's operation results can be previewed in real time, which helps to improve the user experience.

[0157] On the basis of the foregoing embodiments, the embodiment further provides a target search method applied to a server. As shown in Figure 8 The method comprises steps 801 to 805.

[0158] Step 801, in response to receiving the compressed picture sent by the client, performing cutout target detection on the compressed picture to obtain the coordinates of the initial cutout region corresponding to the cutout target.

[0159] Step 802, sending the coordinates of the initial cutout region to the client, so that the client responds to receiving the coordinates of the initial cutout region to display the compressed picture in the cutout preview window, and based on the coordinates of the initial cutout region, the cutout region is highlighted in the cutout interface.

[0160] The specific way in which the server sends the coordinates of the initial cutout region to the client is known in the art and will not be described here.

[0161] The specific way in which the client responds to receiving the coordinates of the initial cutout region to display the compressed picture in the cutout preview window, and based on the coordinates of the initial cutout region, the cutout region is highlighted in the cutout interface is known in the art and will not be described here.

[0162] Step 803, in response to receiving the coordinates of the adjusted cutout region for the compressed picture, performing cutout operation on the compressed picture based on the coordinates of the adjusted cutout region to obtain the cutout picture.

[0163] The specific way in which the server performs cutout operation on the compressed picture based on the coordinates of the adjusted cutout region to obtain the cutout picture is known in the art and will not be described here.

[0164] Step 804, performing search operation based on the cutout picture to obtain search results.

[0165] The specific way in which the search operation is performed based on the cutout picture to obtain search results is known in the art and will not be described here.

[0166] The server performs cutout operation on the compressed picture, which has less data processing than performing cutout operation on the original picture, which can effectively improve the response speed of cutout and search. And because the compression algorithm preserves the key information in the picture, the obtained cutout picture preserves the information of the search target and filters other picture noise interference, which is conducive to improving the accuracy of the search results.

[0167] Step 805, sending the search results to the client, so that the client responds to receiving the search results to display the search results in the search result list window in real time.

[0168] The specific implementation of steps 801 to 805 can refer to the related description in the foregoing embodiments, which will not be repeated here.

[0169] To sum up, the target search method disclosed in this embodiment includes the following steps. The server performs cutout target detection on a compressed picture received from the client to obtain coordinates of an initial cutout region corresponding to the cutout target, wherein the compressed picture is obtained by performing adaptive compression processing on a candidate cutout picture in response to a selection operation of the candidate cutout picture by the client. The client sends the coordinates of the initial cutout region, so that the client displays the compressed picture in a cutout preview window and highlights the cutout region in a cutout interface based on the coordinates of the initial cutout region in response to receiving the coordinates of the initial cutout region. In response to receiving coordinates of an adjusted cutout region of the compressed picture, the server performs a cutout operation on the compressed picture based on the coordinates of the adjusted cutout region to obtain a cutout picture. The server performs a search operation based on the cutout picture to obtain a search result. The server sends the search result to the client, so that the client displays the search result in a search result list window in real time in response to receiving the search result. In this method, the client sends the compressed picture to the server for cutout, which effectively reduces the amount of picture data transmitted between the client and the server, shortens the cutout response time and the search time, and displays the cutout region and the search result in the cutout preview window in real time, so that the user can preview the operation result in real time and improve the user experience.

[0170] On the basis of the foregoing embodiments, this embodiment further provides a target search system for implementing the target search method described above. The target search system includes a client and a server. The specific implementation of the target search system will be described below in conjunction with the interaction flow shown in FIG. 10. Figure 9

[0171] Step 901. The client performs adaptive compression processing on a candidate cutout picture in response to a selection operation of the candidate cutout picture to obtain a compressed picture.

[0172] Step 902. The client sends the compressed picture to a preset server.

[0173] Step 903. The server performs cutout target detection on the compressed picture to obtain coordinates of an initial cutout region corresponding to the cutout target.

[0174] Step 904. The server sends the coordinates of the initial cutout region to the client.

[0175] ​At step 905, the client displays the compressed picture in the matting preview window in response to receiving the coordinates of the initial matting region, and highlights the matting region in the matting interface based on the coordinates of the initial matting region.

[0176] At step 906, the client obtains the coordinates of the adjusted matting region in response to an adjustment operation on the displayed matting region.

[0177] At step 907, the client refreshes the display of the adjusted matting region in real time based on the coordinates of the adjusted matting region.

[0178] At step 908, the client sends the coordinates of the adjusted matting region to the server in real time.

[0179] At step 909, the server performs a matting operation on the compressed picture based on the coordinates of the adjusted matting region to obtain a cut-out picture.

[0180] At step 910, the server performs a search operation based on the cut-out picture to obtain a search result.

[0181] At step 911, the server sends the search result to the client.

[0182] At step 912, the client refreshes the display of the search result in the search result list window in real time.

[0183] The specific implementation of steps 901 to 912 is described above in the previous embodiments, which will not be repeated here.

[0184] In summary, the target search method disclosed in this embodiment effectively reduces the amount of picture data transmitted between the client and the server by sending a compressed picture to the server for matting, shortens the matting response time and search time, and allows the user to preview the operation result in real time, which helps to improve the user experience.

[0185] Based on the above embodiments, this embodiment further provides a mobile terminal manual matting device applied to a client, which comprises:

[0186] A compressed picture obtaining module is configured to perform adaptive compression processing on the candidate matting picture targeted by the selection operation to obtain a compressed picture in response to a selection operation on a candidate matting picture.

[0187] An initial matting region detection module is configured to send the compressed picture to a preset server, so that the server performs matting target detection on the compressed picture to obtain the coordinates of an initial matting region corresponding to the matting target, and sends the coordinates of the initial matting region to the client.

[0188] a matting module, configured to, in response to receiving the coordinates of the initial matting region, initialize a matting region based on the coordinates of the initial matting region, and then perform a matting operation on the compressed picture.

[0189] Optionally, the matting module is further configured to:

[0190] display the compressed picture in a matting preview window, and highlight the matting region in a matting interface based on the coordinates of the initial matting region;

[0191] a matting module, configured to, in response to an adjustment operation on the displayed matting region, trigger the server to perform a matting operation on the compressed picture based on the adjusted matting region, and obtain a cut-out picture.

[0192] Optionally, the matting module is further configured to:

[0193] in response to the adjustment operation on the matting region, obtain coordinates of the adjusted matting region;

[0194] based on the coordinates of the adjusted matting region, refresh the display of the adjusted matting region in the matting interface in real time, and send the coordinates of the adjusted matting region to the server in real time, so that the server performs a matting operation on the compressed picture based on the coordinates of the adjusted matting region in real time, and obtains a cut-out picture.

[0195] Optionally, the compressed picture obtaining module is further configured to:

[0196] obtain picture information of the candidate matting picture targeted by the selection operation, wherein the picture information includes resolution and / or content complexity indicators;

[0197] based on the resolution and / or the content complexity indicators, perform compression processing on the candidate matting picture to obtain a compressed picture, so that the picture size is less than or equal to a preset size threshold while the visual effect of the compressed picture is guaranteed.

[0198] Optionally, the matting module is further configured to:

[0199] based on the content complexity indicators, perform region division processing on the candidate matting picture to determine picture regions corresponding to different content complexities in the candidate matting picture;

[0200] Set a compression quality parameter corresponding to each picture region based on the content complexity corresponding to the picture region, and perform a region-based compression process on the corresponding picture region based on the compression quality parameter to obtain a compressed picture.

[0201] Optionally, before the obtaining the compressed picture by performing the adaptive compression process on the candidate cutout picture corresponding to the selection operation, the apparatus further comprises:

[0202] a preloading module configured to predict a candidate cutout picture based on one or more of a cutout application operation habit of a current user, a usage frequency of pictures in the mobile terminal, and an access time of pictures in the mobile terminal, and load the candidate cutout picture to a local cache of the mobile terminal.

[0203] The preloading module is further configured to preferentially read the candidate cutout picture from the local cache and display the read candidate cutout picture.

[0204] Optionally, after the loading the candidate cutout picture to the local cache of the mobile terminal, the apparatus further comprises:

[0205] a memory management module configured to monitor an occupation condition of the local cache used to load the candidate cutout picture in real time, and

[0206] clean up the candidate cutout picture loaded to the local cache when the occupation condition of the local cache meets a preset condition.

[0207] On the basis of the above-mentioned embodiments, the present embodiment further provides a mobile terminal manual cutout device applied to a server, the device comprising:

[0208] an initial cutout region determination module configured to, in response to receiving a compressed picture sent by a client, perform cutout target detection on the compressed picture to obtain coordinates of an initial cutout region corresponding to the cutout target, wherein the compressed picture is obtained by performing adaptive compression on a candidate cutout picture corresponding to a selection operation of the client.

[0209] a sending module configured to send the coordinates of the initial cutout region to the client, so that the client initializes a cutout region based on the coordinates of the initial cutout region after receiving the coordinates of the initial cutout region, and then performs a cutout operation on the compressed picture.

[0210] Optionally, initializing the cutout region based on the coordinates of the initial cutout region and then performing the cutout operation on the compressed picture comprises:

[0211] display the compressed picture in the preview window of the matting, and highlight the matting region in the matting interface based on the coordinates of the initial matting region;

[0212] In response to receiving the coordinates of the adjusted matting region of the compressed picture, perform a matting operation on the compressed picture based on the coordinates of the adjusted matting region, and obtain a cut-out picture.

[0213] The mobile terminal manual matting device disclosed in the embodiments of the present application is used to implement the mobile terminal manual matting method described above. The specific implementation of each module of the device is described in the specific implementation of the corresponding steps in the foregoing method embodiments, which will not be described here again.

[0214] In summary, the mobile terminal manual matting device disclosed in the embodiments of the present application performs adaptive compression processing on the candidate matting picture targeted by the selection operation after the client obtains the selection operation on the candidate matting picture, obtains a compressed picture, and sends the compressed picture to a preset server, so that the server performs matting target detection on the compressed picture to obtain the coordinates of the initial matting region corresponding to the matting target, and sends the coordinates of the initial matting region to the client. This effectively reduces the amount of picture data transmitted between the client and the server, shortens the matting response time, and can reduce the amount of system resources occupied by the client for displaying and storing matting pictures. On the other hand, it can reduce the data processing amount of the server for performing matting target detection, improve the matting processing speed, and improve the performance of the matting application. Further, after the client receives the coordinates of the matting region sent by the server, the client displays the compressed picture in the preview window of the matting, highlights the matting region in the matting interface based on the coordinates of the matting region, and in response to an adjustment operation on the displayed matting region, triggers the server to perform a matting operation on the compressed picture based on the adjusted matting region, and obtains a cut-out picture. The user's operation result can be previewed in real time, which helps to improve the user experience.

[0215] On the basis of the foregoing embodiments, the present embodiment further provides a target searching device applied to a client, the device comprising:

[0216] The compressed picture obtaining module is configured to perform adaptive compression processing on the candidate matting picture targeted by the selection operation in response to the selection operation on the candidate matting picture, and obtain a compressed picture.

[0217] The initial matting region obtaining module is configured to send the compressed picture to a preset server, so that the server performs matting target detection on the compressed picture to obtain the coordinates of the initial matting region corresponding to the matting target, and sends the coordinates of the initial matting region to the client.

[0218] The first display module is configured to display the compressed picture in a matting preview window in response to receiving the coordinates of the initial matting region, and highlight the matting region in a matting interface based on the coordinates of the initial matting region.

[0219] The search module is configured to trigger the server to perform matting and search processing on the compressed picture based on the adjusted matting region in response to an adjustment operation on the displayed matting region, and obtain search results corresponding to the picture content in the adjusted matting region.

[0220] The second display module is configured to display the search results in a search result list window in real time.

[0221] Optionally, the response to the adjustment operation on the displayed matting region, the triggering of the server to perform matting and search processing on the compressed picture based on the adjusted matting region, and the obtaining of search results corresponding to the picture content in the adjusted matting region include:

[0222] In response to the adjustment operation on the matting region, the coordinates of the adjusted matting region are obtained.

[0223] Based on the coordinates of the adjusted matting region, the adjusted matting region is displayed in real time, and the coordinates of the adjusted matting region are sent to the server in real time, so that the server performs a matting operation on the compressed picture based on the coordinates of the adjusted matting region, obtains a cut-out picture, performs a search operation based on the cut-out picture, obtains search results, and sends the search results to the client.

[0224] On the basis of the above-mentioned embodiments, the present embodiment further provides a target search device applied to a server, the device comprising:

[0225] The initial matting region acquisition module is configured to perform matting target detection on the compressed picture in response to receiving the compressed picture sent by the client, and obtain the coordinates of the initial matting region corresponding to the matting target, wherein the compressed picture is obtained by performing adaptive compression processing on the candidate matting picture targeted by the selection operation in response to the selection operation of the candidate matting picture by the client.

[0226] The first sending module is configured to send the coordinates of the initial matting region to the client, so that the client displays the compressed picture in a matting preview window in response to receiving the coordinates of the initial matting region, and highlights the matting region in a matting interface based on the coordinates of the initial matting region.

[0227] The cutout module is configured to perform a cutout operation on the compressed picture based on the coordinates of the adjusted cutout region in response to receiving the coordinates of the adjusted cutout region of the compressed picture, and obtain a cutout picture.

[0228] The search module is configured to perform a search operation based on the cutout picture and obtain a search result.

[0229] The second sending module is configured to send the search result to the client, so that the client displays the search result in a search result list window in real time in response to receiving the search result.

[0230] The target search device disclosed by the embodiments of the present application is used to implement the target search method described above. The specific implementation of each module of the device is described in the specific implementation of the corresponding steps in the foregoing method embodiments, which will not be described here.

[0231] In summary, the target search device disclosed by the embodiments of the present application performs adaptive compression processing on the candidate cutout picture in response to a selection operation of the candidate cutout picture by the client, to obtain a compressed picture. The compressed picture is sent to a preset server, so that the server performs cutout target detection on the compressed picture to obtain the coordinates of an initial cutout region corresponding to the cutout target, and sends the coordinates of the initial cutout region to the client, effectively reducing the amount of picture data transmitted between the client and the server and shortening the cutout response time. The client displays the compressed picture in a cutout preview window in response to receiving the coordinates of the initial cutout region, and highlights the display of the cutout region in a cutout interface based on the coordinates of the initial cutout region. Then, in response to an adjustment operation on the displayed cutout region, the server is triggered to perform cutout and search processing on the compressed picture based on the adjusted cutout region, to obtain a search result corresponding to the picture content in the adjusted cutout region. The search result is displayed in a search result list window in real time, so that the user can preview the operation result in real time, which helps to improve the user experience.

[0232] The embodiments of the present application also provide a non-volatile readable storage medium, which stores one or more programs. When the one or more programs are applied to a device, the device can execute instructions of the steps of the methods in the embodiments of the present application.

[0233] The embodiments of the present application also provide a computer readable storage medium, which stores computer execution instructions. When the computer execution instructions are executed by a processor, the computer execution instructions are used to implement the method described in the embodiments of the present application.

[0234] The embodiment of the present application further provides an electronic device, comprising a processor and a memory connected with the processor in communication; the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory to implement the method according to the embodiment of the present application. In the embodiment of the present application, the electronic device comprises a server, a terminal device and the like.

[0235] The embodiment of the present application further discloses a computer program product comprising computer program / computer executable instructions, which, when executed by a processor in an electronic device, implement the method according to the embodiment of the present application.

[0236] Embodiments of the present disclosure can be implemented as an apparatus configured to perform desired functions using any suitable hardware, firmware, software, or any combination thereof, which can include a server (cluster), a terminal and the like electronic device. Figure 10 An exemplary apparatus 1000 that can be used to implement various embodiments described herein is shown schematically.

[0237] For one embodiment, Figure 10 An exemplary apparatus 1000 is shown having one or more processors 1002, a control module (chipset) 1004 coupled to at least one of the processor(s) 1002, a memory 1006 coupled to the control module 1004, a non-volatile memory (NVM) / storage device 1008 coupled to the control module 1004, one or more input / output devices 1010 coupled to the control module 1004, and a network interface 1012 coupled to the control module 1004.

[0238] The processor(s) 1002 can include one or more single core or multi core processors, which can include any combination of general-purpose processors or dedicated processors (such as graphics processors, application processors, baseband processors, etc.). In some embodiments, the apparatus 1000 can function as a server, a terminal and the like device described in the embodiments of the present application.

[0239] In some embodiments, the apparatus 1000 can include one or more computer readable media (such as the memory 1006 or the NVM / storage device 1008) having instructions 1014 and one or more processors 1002 incorporated with the one or more computer readable media configured to execute the instructions 1014 to implement modules to perform the actions described in the present disclosure.

[0240] For one embodiment, the control module 1004 can include any suitable interface controllers to provide for any suitable interface to at least one of the processor(s) 1002 and / or any suitable device or component in communication with the control module 1004.

[0241] The control module 1004 can include a memory controller module to provide an interface to the memory 1006. The memory controller module can be a hardware module, a software module, and / or a firmware module.

[0242] The memory 1006 can be used to, for example, load and store data and / or instructions 1014 for the apparatus 1000. For one embodiment, the memory 1006 can include any suitable volatile memory, such as suitable DRAM. In some embodiments, the memory 1006 can include double data rate type four synchronous dynamic random access memory (DDR4 SDRAM).

[0243] For one embodiment, the control module 1004 can include one or more input / output controllers to provide an interface to the NVM / storage device 1008 and the input / output device(s) 1010.

[0244] The NVM / storage device 1008 can be used, for example, to store data and / or instructions 1014. The NVM / storage device 1008 can include any suitable non-volatile memory (e.g., flash memory) and / or can include any suitable non-volatile storage device(s) (e.g., one or more hard disk drives (HDDs), one or more compact disk (CD) drives, and / or one or more digital versatile disk (DVD) drives).

[0245] The NVM / storage device 1008 can include storage resources that are part of the device on which the apparatus 1000 is installed, or that are accessed by the device but not necessarily part of the device. For example, the NVM / storage device 1008 can be accessed over a network via the input / output device(s) 1010.

[0246] The input / output device(s) 1010 can provide an interface for the apparatus 1000 to communicate with any other suitable device, and the input / output device(s) 1010 can include communication components, audio components, sensor components, etc. The network interface 1012 can provide an interface for the apparatus 1000 to communicate over one or more networks, and the apparatus 1000 can wirelessly communicate with one or more components of a wireless network according to any of one or more wireless network standards and / or protocols, such as to access a wireless network based on a communication standard, such as Bluetooth, WiFi, 2G, 3G, 4G, 5G, etc., or a combination thereof.

[0247] For one embodiment, at least one of the processor(s) 1002 can be packaged together with logic for one or more controllers of the control module 1004 (e.g., a memory controller module). For one embodiment, at least one of the processor(s) 1002 can be packaged together with logic for one or more controllers of the control module 1004 to form a system in a package (SiP). For one embodiment, at least one of the processor(s) 1002 can be fabricated together with logic for one or more controllers of the control module 1004 on the same die. For one embodiment, at least one of the processor(s) 1002 can be fabricated together with logic for one or more controllers of the control module 1004 on the same die to form a system on a chip (SoC).

[0248] In various embodiments, the apparatus 1000 can be, but is not limited to, a terminal device such as a server, a desktop computing device, or a mobile computing device (e.g., a laptop computing device, a handheld computing device, a tablet, a netbook, etc.). In various embodiments, the apparatus 1000 can have more or less components, and / or different architectures. For example, in some embodiments, the apparatus 1000 includes one or more cameras, a keyboard, a liquid crystal display (LCD) screen (including touch screen displays), nonvolatile memory port, multiple antennas, a graphics chip, an application-specific integrated circuit (ASIC), and a speaker.

[0249] In various embodiments, the apparatus 1000 can be, but is not limited to, a terminal device such as a server, a desktop computing device, or a mobile computing device (e.g., a laptop computing device, a handheld computing device, a tablet, a netbook, etc.). In various embodiments, the apparatus 1000 can have more or less components, and / or different architectures. For example, in some embodiments, the apparatus 1000 includes one or more cameras, a keyboard, a liquid crystal display (LCD) screen (including touch screen displays), nonvolatile memory port, multiple antennas, a graphics chip, an application-specific integrated circuit (ASIC), and a speaker.

[0250] Embodiments of the present application also provide an electronic device, comprising: a processor; and a memory having stored therein executable code that, when executed by the processor, enables the processor to perform one or more of the methods described in embodiments of the present application. The memory in embodiments of the present application can store various data, such as target files, file and application association data, and the like, and can also include user behavior data, and the like, thereby providing a data basis for various processing.

[0251] Embodiments of the present application also provide one or more machine- readable media having stored thereon executable code that, when executed by a processor, enables the processor to perform one or more of the methods described in embodiments of the present application.

[0252] For the apparatus embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts are referred to the part of the method embodiments.

[0253] The various embodiments in the specification are described in progressive manner, and each embodiment focuses on the difference from other embodiments, and the same or similar parts between the embodiments can be mutually referred to.

[0254] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal devices to produce a machine, so that the instructions executed by the computer or other programmable data processing terminal devices generate a device implemented in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that carries out the function specified in one or more flows and / or blocks.

[0255] These computer program instructions can also be stored in a computer readable memory that can direct the computer or other programmable data processing terminal devices to work in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including instruction apparatus, which implements the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that carries out the function specified in one or more flows and / or blocks.

[0256] These computer program instructions can also be loaded into a computer or other programmable data processing terminal device, so that a series of operation steps are performed on the computer or other programmable terminal device to produce a computer implemented process, so that the instructions executed on the computer or other programmable terminal device provide a process for implementing the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that carries out the function specified in one or more flows and / or blocks.

[0257] Although the preferred embodiments of the present application have been described, those skilled in the art can make additional changes and modifications to the embodiments once they know the basic inventive concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the embodiments of the present application.

[0258] Finally, it is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting; it is not intended to exclude myriad other embodiments of the present application that other out-of-the-ordinary persons skilled in the pertinent art will recognize. In this document, relational terms such as first and second and the like can be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the presence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0259] The mobile terminal manual matting method, the mobile terminal manual matting system, the target searching method, the target searching system, the electronic device, the storage medium and the computer program product provided in the present application are described in detail above, the principles and implementation manners of the present application are described by applying specific examples in the present document, the above description of the embodiments is only used to help understand the method of the present application and its core idea; meanwhile, for the person skilled in the art, according to the idea of the present application, the specific implementation manners and application ranges will be changed, and the above description of the present application should not be understood as a limitation.

Claims

1. A method for manually cutting out images on a mobile device, applied to a client, characterized in that, The method includes: In response to the selection operation of the candidate cutout image, the candidate cutout image targeted by the selection operation is subjected to adaptive compression processing to obtain a compressed image; The compressed image is sent to a preset server, which then performs target detection on the compressed image to obtain the coordinates of the initial cutout region corresponding to the target, and sends the coordinates of the initial cutout region to the client. In response to receiving the coordinates of the initial cutout region, the cutout region is initialized based on the coordinates of the initial cutout region, and then the cutout operation is performed on the compressed image.

2. The method according to claim 1, characterized in that, The process of initializing the cutout region based on the coordinates of the initial cutout region, and then performing a cutout operation on the compressed image, includes: The compressed image is displayed in the cutout preview window, and the cutout area is highlighted in the cutout interface based on the coordinates of the initial cutout area; In response to the adjustment operation of the displayed cutout area, the server is triggered to perform a cutout operation on the compressed image based on the adjusted cutout area to obtain the cutout image.

3. The method according to claim 1, characterized in that, In response to the adjustment operation of the displayed cutout area, the server is triggered to perform a cutout operation on the compressed image based on the adjusted cutout area to obtain the cutout image, including: In response to the adjustment operation on the cutout area, the coordinates of the adjusted cutout area are obtained; Based on the coordinates of the adjusted cutout area, the adjusted cutout area is refreshed and displayed in real time on the cutout interface, and the coordinates of the adjusted cutout area are sent to the server in real time, so that the server can perform cutout operation on the compressed image in real time based on the coordinates of the adjusted cutout area to obtain the cutout image.

4. The method according to claim 1, characterized in that, The adaptive compression processing of the candidate cutout images targeted by the selection operation to obtain compressed images includes: Obtain image information of the candidate cutout image for which the selection operation is applied, wherein the image information includes: resolution and / or content complexity index; Based on the resolution and / or the content complexity index, the candidate cutout images are compressed to obtain compressed images, such that the image size is less than or equal to a preset size threshold while ensuring the visual effect of the compressed images.

5. The method according to claim 4, characterized in that, The step of compressing the candidate cutout image based on the resolution and / or the content complexity index to obtain a compressed image includes: Based on the content complexity index, the candidate cutout images are divided into regions to determine the image regions in the candidate cutout images that correspond to different content complexities. Based on the content complexity corresponding to the image region, a compression quality parameter is set for each image region, and the corresponding image region is compressed in sections based on the compression quality parameter to obtain a compressed image.

6. The method according to claim 1, characterized in that, Before performing adaptive compression processing on the candidate cutout images in response to the selection operation, and obtaining the compressed image, the method further includes: Based on one or more of the following information: the current user's image cutout application operation habits, the frequency of image usage on the mobile device, and the access time of images on the mobile device, candidate cutout images are predicted; The candidate cutout images are loaded into the local cache of the mobile device; The candidate cutout images are read from the local cache first, and the read candidate cutout images are displayed.

7. A method for manual image cutout on a mobile device, applied to a server, characterized in that, The method includes: In response to receiving a compressed image sent by the client, the client performs target detection on the compressed image to obtain the coordinates of the initial cutout region corresponding to the target. The compressed image is obtained by the client performing adaptive compression processing on the candidate cutout image in response to a selection operation on the candidate cutout image. The coordinates of the initial cutout region are sent to the client, so that after receiving the coordinates of the initial cutout region, the client initializes the cutout region based on the coordinates of the initial cutout region, and then performs the cutout operation on the compressed image.

8. A target search method, applied to a client, characterized in that, The method includes: In response to the selection operation of the candidate cutout image, the candidate cutout image targeted by the selection operation is subjected to adaptive compression processing to obtain a compressed image; The compressed image is sent to a preset server, which then performs target detection on the compressed image to obtain the coordinates of the initial cutout region corresponding to the target, and sends the coordinates of the initial cutout region to the client. In response to receiving the coordinates of the initial cutout area, the compressed image is displayed in the cutout preview window, and the cutout area is highlighted in the cutout interface based on the coordinates of the initial cutout area; In response to the adjustment operation of the displayed cutout area, the server is triggered to perform cutout and image search processing on the compressed image based on the adjusted cutout area, and obtain the search results corresponding to the image content in the adjusted cutout area; The search results will be displayed in real time in the image search results list window.

9. The method according to claim 8, characterized in that, In response to the adjustment operation of the displayed cutout area, the server is triggered to perform cutout and image search processing on the compressed image based on the adjusted cutout area, and to obtain the search results corresponding to the image content in the adjusted cutout area, including: In response to the adjustment operation on the cutout area, the coordinates of the adjusted cutout area are obtained; Based on the coordinates of the adjusted cutout area, the adjusted cutout area is refreshed and displayed in real time, and the coordinates of the adjusted cutout area are sent to the server in real time, so that the server can perform cutout operation on the compressed image in real time based on the coordinates of the adjusted cutout area, obtain the cutout image, perform image search operation based on the cutout image, obtain search results, and send the search results to the client.

10. A target search method, applied on a server side, characterized in that, The method includes: In response to receiving a compressed image sent by the client, the client performs target detection on the compressed image to obtain the coordinates of the initial cutout region corresponding to the target. The compressed image is obtained by the client performing adaptive compression processing on the candidate cutout image in response to a selection operation on the candidate cutout image. Send the coordinates of the initial cutout area to the client, so that the client responds to receiving the coordinates of the initial cutout area by displaying the compressed image in the cutout preview window, and highlighting the cutout area in the cutout interface based on the coordinates of the initial cutout area; In response to receiving the adjusted coordinates of the cutout region for the compressed image, a cutout operation is performed on the compressed image based on the adjusted coordinates of the cutout region to obtain the cutout image; Based on the extracted image, perform an image search operation to obtain search results; The search results are sent to the client, causing the client to respond by receiving the search results and refreshing and displaying the search results in the search results list window in real time.

11. A mobile manual background removal system, comprising a client and a server, characterized in that, The client is used to perform the method as described in any one of claims 1-6, and the server is used to perform the method as described in claim 7.

12. A target search system, comprising a client and a server, characterized in that, The client is used to perform the method as described in claim 8 or 9, and the server is used to perform the method as described in claim 10.

13. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1-10.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-10.

15. A computer program product comprising a computer program / computer-executable instructions, characterized in that, When the computer program / computer executable instructions are executed by a processor in an electronic device, the method of any one of claims 1-10 is implemented.

Images