Image processing method and device, and non-volatile storage medium

Abstract

The present invention discloses an image processing method and device, and a non-volatile storage medium. The method comprises: obtaining an input image; performing target detection on the input image to obtain a target area, wherein the target area comprises at least one of the following: a first sub-area and a second sub-area, the first sub-area and the second sub-area respectively corresponding to different parts of the human body; selecting a to-be-focused area from the target area, wherein the to-be-focused area is used to display information to be paid attention to. The present invention solves the technical problem that the image processing scheme in the prior art cannot effectively determine the to-be-focused area of ​​the image, resulting in difficulty in clearly displaying the to-be-focused area.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to an image processing method and device, and a non-volatile storage medium. Background Art

[0002] In current live broadcasting scenarios, anchors usually use general cameras to shoot live video images. Since general cameras do not have the function of automatically identifying the focus area, they can only use the face of the person in the picture as the focus area and make the camera focus clearly on the face of the person. Therefore, when the anchor introduces a product, if he wants the audience to clearly see the product in his hand, he will place the product closer to the camera.

[0003] However, in this case, the product is far away from the person's face and is in an unfocused area. In addition, due to the limited depth of field of general cameras, the product can only be clearly imaged within a certain range from the lens. When the product is outside the clear imaging distance range, the product is in a blurred state and its details cannot be seen clearly. Therefore, the anchor needs to manually cover his or her face so that the camera can refocus the position to improve the display clarity of the product he or she is introducing. However, in many cases, the camera still will not refocus to the appropriate area, causing the product to remain blurred, failing to fully display the product and seriously affecting the live broadcast experience and efficiency.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present application provide an image processing method and device, and a non-volatile storage medium, so as to at least solve the technical problem that the image processing scheme in the prior art cannot effectively determine the area to be focused of the image, resulting in difficulty in clearly displaying the area to be focused.

[0006] According to one aspect of an embodiment of the present application, there is provided an image processing method, comprising: acquiring an input image; performing target detection on the input image to obtain a target area, wherein the target area comprises at least one of the following: a first sub-area and a second sub-area, the first sub-area and the second sub-area respectively corresponding to different parts of the human body; selecting an area to be focused from the target area, wherein the area to be focused is used to display information to be focused.

[0007] According to another aspect of an embodiment of the present application, an image processing method is also provided, including: obtaining a to-be-focused area from an input image, wherein the to-be-focused area is used to display information to be paid attention to; and adjusting a focus state of the to-be-focused area to obtain an image to be output.

[0008] According to another aspect of an embodiment of the present application, an image processing method is also provided, including: acquiring a first image; selecting an area to be focused from the first image, wherein the area to be focused is used to display information to be paid attention to; adjusting the focus state of the area to be focused to obtain a second image; and outputting the second image.

[0009] According to another aspect of an embodiment of the present application, an image processing method is also provided, including: acquiring an input image; performing target detection on the input image to obtain a target area, wherein the target area includes at least one of the following: a first sub-area and a second sub-area, the first sub-area containing a first target object, the first target object having vital signs, and the second sub-area containing a second target object, the second target object having no vital signs and being an operation object of the first target object; selecting an area to be focused from the target area, wherein the area to be focused is used to display information to be focused.

[0010] According to another aspect of an embodiment of the present application, an image processing method is also provided, including: acquiring an input image; performing target detection on the input image to obtain a target area, wherein the target area includes at least one of the following: a first sub-area and a second sub-area, the first sub-area containing a first target object, a local body of the first target object has a lesion, and the second sub-area contains a second target object, and the second target object is an internal organ; selecting a to-be-focused area from the target area, wherein the to-be-focused area is used to display information to be paid attention to.

[0011] According to another aspect of an embodiment of the present application, a non-volatile storage medium is further provided, wherein the non-volatile storage medium includes a stored program, wherein when the program is executed, the device where the non-volatile storage medium is located is controlled to execute any one of the above-mentioned image processing methods.

[0012] According to another aspect of an embodiment of the present application, an image processing device is also provided, including: a processor; and a memory, connected to the above-mentioned processor, for providing the above-mentioned processor with instructions for processing the following processing steps: acquiring an input image; performing target detection on the above-mentioned input image to obtain a target area, wherein the above-mentioned target area includes at least one of the following: a first sub-area and a second sub-area, the first sub-area and the second sub-area respectively corresponding to different parts of the human body; selecting a to-be-focused area from the above-mentioned target area, wherein the above-mentioned to-be-focused area is used to display information to be paid attention to.

[0013] In an embodiment of the present application, a method of locating a target area in an input image and selecting an area to be focused from the target area is adopted, wherein an input image is acquired; target detection is performed on the input image to obtain a target area, wherein the target area includes at least one of the following: a first sub-area and a second sub-area, the first sub-area and the second sub-area respectively corresponding to different parts of the human body; and an area to be focused is selected from the target area, wherein the area to be focused is used to display information to be focused.

[0014] It is easy to notice that the embodiment of the present application can identify the input image, locate the facial area or the hand area in the input image, and select the area to be focused for displaying the information to be paid attention to from the facial area or the hand area, and then can clearly display the area to be focused by focusing, thereby achieving the purpose of improving the efficiency of determining the target area of ​​the image, thereby achieving the technical effect of clearly displaying the area to be focused, and then solving the technical problem that the image processing scheme in the prior art cannot effectively determine the area to be focused of the image, resulting in difficulty in clearly displaying the area to be focused. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0016] Figure 1 is a hardware structure block diagram of a computer terminal (or mobile device) for implementing an image processing method according to an embodiment of the present application;

[0017] Figure 2 is a flow chart of an image processing method according to an embodiment of the present application;

[0018] Figure 3 is a structural schematic diagram of an image processing system according to an embodiment of the present application;

[0019] Figure 4a is a schematic diagram of a hand region in a state of holding an object according to an embodiment of the present application;

[0020] Figure 4b is a schematic diagram of a hand region in another state of holding an object according to an embodiment of the present application;

[0021] Figure 5 is a flowchart of another image processing method according to an embodiment of the present application;

[0022] Figure 6 is a flowchart of another image processing method according to an embodiment of the present application;

[0023] Figure 7 is a flowchart of another image processing method according to an embodiment of the present application;

[0024] Figure 8 is a flowchart of another image processing method according to an embodiment of the present application;

[0025] Fig. 9 is a structural schematic diagram of an image processing device according to an embodiment of the present application;

[0026] Fig.10 is a structural schematic diagram of another image processing device according to an embodiment of the present application;

[0027] Fig.11 is a structural schematic diagram of another image processing device according to an embodiment of the present application;

[0028] Fig.12 is a structural schematic diagram of an image processing device according to an embodiment of the present application;

[0029] Fig.13 It is a structural block diagram of another computer terminal according to an embodiment of the present application. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only embodiments of a part of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0032] First, some nouns or terms that appear in the description of the embodiments of the present application are subject to the following explanations:

[0033] SSD: single shot multibox detection, refers to a single-stage target detection algorithm using candidate boxes, which encapsulates positioning and detection in the forward operation of the network, thereby limiting and improving the operation speed.

[0034] FCOS: Fully Convolutional One-Stage Object Detection refers to a single-stage target detection algorithm that does not use candidate boxes. It solves the target detection problem in a pixel-by-pixel prediction manner, similar to semantic segmentation.

[0035] FasterRCNN: refers to a two-stage object detection algorithm using candidate boxes.

[0036] Example 1

[0037] According to an embodiment of the present application, an embodiment of an image processing method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0038] The method embodiment provided in Embodiment 1 of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 A hardware structure block diagram of a computer terminal (or mobile device) for implementing an image processing method is shown. Figure 1 As shown, the computer terminal 10 (or mobile device 10) may include one or more (102a, 102b, ..., 102n are used to illustrate) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission module 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0039] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10 (or mobile device). As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0040] The memory 104 can be used to store software programs and modules of application software, such as program instructions / data storage devices corresponding to the image processing method in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, realizing the above-mentioned image processing method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0041] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0042] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 (or mobile device).

[0043] Under the above operating environment, this application provides Figure 2 An image processing method is shown. Figure 2 is a flowchart of an image processing method according to an embodiment of the present application, such as Figure 2 As shown, the above image processing method includes:

[0044] Step S202, obtaining an input image;

[0045] Step S204, performing target detection on the input image to obtain a target region, wherein the target region includes at least one of the following: a first sub-region and a second sub-region, the first sub-region and the second sub-region respectively corresponding to different human body parts;

[0046] Step S206, selecting a to-be-focused area from the target area, wherein the to-be-focused area is used to display information to be focused on.

[0047] In an embodiment of the present application, a method of locating a target area in an input image and selecting an area to be focused from the target area is adopted, wherein an input image is acquired; target detection is performed on the input image to obtain a target area, wherein the target area includes at least one of the following: a first sub-area and a second sub-area, the first sub-area and the second sub-area respectively corresponding to different parts of the human body; and an area to be focused is selected from the target area, wherein the area to be focused is used to display information to be focused.

[0048] It is easy to notice that the embodiment of the present application can identify the input image, locate the facial area or the hand area in the input image, and select the area to be focused for displaying the information to be paid attention to from the facial area or the hand area, and then can clearly display the area to be focused by focusing, thereby achieving the purpose of improving the efficiency of determining the target area of ​​the image, thereby achieving the technical effect of clearly displaying the area to be focused, and then solving the technical problem that the image processing scheme in the prior art cannot effectively determine the area to be focused of the image, resulting in difficulty in clearly displaying the area to be focused.

[0049] In an optional embodiment, the above method is applied to a live broadcast scenario, and the above area to be focused is used to display product recommendation information, wherein the above information to be focused on can be the goods / products recommended by the anchor to the audience during the live broadcast.

[0050] Optionally, the input image may be a live video image. By focusing the area to be focused, an image to be output, that is, a live video image to be output and displayed, may be obtained.

[0051] As an optional embodiment, the first sub-region is a face region, and the second sub-region is a hand region.

[0052] It should be noted that the image processing method provided in the embodiment of the present application can be applicable to application scenarios in which the area to be focused in the image is located during live broadcast, and the clarity of the area to be focused is adjusted by focusing after the area to be focused is located. Specifically, the clarity of the area to be focused can be adjusted by, but is not limited to, software algorithms or hardware driver interfaces.

[0053] Taking the application of the above method to a live broadcast scenario as an example, in an embodiment of the present application, the target area in the input image may be located first, for example, the facial area and / or hand area of ​​the host; and then the area to be focused in the above target area may be obtained. Optionally, the position of the area to be focused in the live broadcast video image may be located by detecting the face position, hand position, hand posture, and hand state. For example, when it is detected that the host's hand is located in the center area of ​​the image, and the host's hand posture is in the posture of grasping an object and is maintained for a certain period of time, the object area in the host's hand is considered to be the area to be focused; when the monitored host's hand posture is in a naturally placed posture, the area to be focused is considered to be the host's facial area.

[0054] During the live broadcast process, the live video image screen can be used as the input image. In addition, the input image can also carry other information of the video data, such as timestamp information. Figure 3 As shown, after obtaining the above-mentioned input image, the input image can be sent to the face and hand detection module, and the target detection method is used to perform target detection on the above-mentioned input image to obtain the target area, and the above-mentioned target area is analyzed through the area discrimination module or by using the focus area discrimination model to determine the above-mentioned area to be focused.

[0055] In an optional embodiment, target detection is performed on the input image to obtain the target area, including:

[0056] Step S302: Detect the position information and confidence information of the target area from the input image using a target detection method.

[0057] In an optional embodiment, the above-mentioned target detection method includes at least one of the following: a single-stage target detection method (SSD) using a candidate box, a single-stage target detection method (FCOS) not using a candidate box, and a two-stage target detection method (FasterRCNN) using a candidate box.

[0058] In another optional embodiment, the position information of the target area includes: the first position r of the hand area hand , the second position r of the facial region face The confidence information of the target area includes: the first confidence c of the hand area hand , the second confidence c of the facial region face .

[0059] In an optional embodiment, the position information and confidence information of the target area include: a first position and a first confidence of the second sub-area, and the method further includes:

[0060] Step S402, when the first position is detected and the first confidence level is greater than a preset threshold, acquiring the second sub-region from the target region;

[0061] Step S404, performing hand posture estimation on the second sub-region by using a key point detection method to obtain a first estimation result;

[0062] Step S406: Estimating the hand state of the second sub-region by using a classification and discrimination method to obtain a second estimation result.

[0063] In the above optional embodiment, when the first position r of the hand area is detected hand And the first confidence c hand When it is greater than the preset threshold, the first position r of the hand area is hand The hand area is cropped out from the input image, and subsequent hand posture and hand state estimation calculations are performed. Optionally, the hand posture estimation of the hand area can be performed by key point detection to obtain the position information of each hand joint; the hand state estimation of the above hand area is performed by classification and discrimination, and it can be determined whether the hand area is in at least one of the following states: holding an item state, displaying an item state, blocking the background state, and naturally placed state.

[0064] As an optional embodiment, a key point detection method can be used to extract key points from the hand area, obtain hand posture estimation, and obtain position information of each hand joint; and a classification and discrimination method can be used to determine the current hand state of the hand area, for example, to determine Figure 4a and Figure 4b The hand area shown in two states of holding an item.

[0065] As an optional embodiment, the above-mentioned key point detection method includes but is not limited to posture estimation methods such as the real-time multi-person posture estimation system OpenPose and the precise multi-person posture estimation system alphapose; the above-mentioned classification and discrimination method includes but is not limited to network classification and discrimination methods such as deep residual networks resnet101 and resnet50.

[0066] As an optional embodiment, Figure 3 As shown, after the input image is detected by using the face and hand area detection module, if the face area is detected, the detected face area is directly sent to the area discrimination module to select the area to be focused in the face area; if the hand area is detected, the hand posture and hand state estimation module is used to estimate the hand posture and hand state, and the first estimation result and the second estimation result are sent to the area discrimination module to select the area to be focused in the hand area; if the face area and the hand area are not detected, the entire image screen is used as the area to be focused.

[0067] The region identification module may select the region to be focused in the target region by, but not limited to, the following methods:

[0068] In an optional embodiment, selecting the area to be focused from the target area includes: when a face area is detected from the target area, selecting the face area as the area to be focused.

[0069] In the above optional implementation, if only a facial area, for example, a face area of ​​a host, is detected from the target area, the facial area is selected as the area to be focused.

[0070] In an optional embodiment, selecting the area to be focused from the target area includes:

[0071] Step S502: When a second sub-region is detected from the target region and it is determined based on the first estimation result and the second estimation result that the hand posture and the hand state meet a preset condition, the second sub-region is selected as the region to be focused.

[0072] In the above optional implementation, when only the hand area is detected from the above target area, it is determined whether the hand posture and the hand state meet the preset conditions based on the above first estimation result and the above second estimation result. For example, when it is detected that the host's hand state is holding an object and the hand posture is a posture of displaying the object (without blocking the background), or when it is detected that the host's hand is holding an object and blocking the background, the host's hand is selected as the area to be focused.

[0073] In an optional embodiment, selecting the area to be focused from the target area includes:

[0074] Step S602, when the first sub-region and the second sub-region are detected from the target region, and it is determined based on the first estimation result and the second estimation result that the hand posture and the hand state meet the preset conditions, the second sub-region is selected as the region to be focused;

[0075] Step S604, when the first sub-region and the second sub-region are detected from the target region, and it is determined based on the first estimation result and the second estimation result that the hand posture and the hand state do not satisfy the preset conditions, the first sub-region is selected as the region to be focused.

[0076] In the above optional implementation, when the hand area and the facial area are simultaneously detected from the above target area, if it is detected that the host's hand state is holding an object and the hand posture is a posture of displaying the object, or it is detected that the host's hand is in a state of blocking the background, that is, it is determined that the hand posture and hand state meet the preset conditions, then the host's hand is selected as the area to be focused; if it is detected that the host's hand state is in a natural placement state, it is determined that the hand posture and hand state do not meet the preset conditions, then the host's face is selected as the area to be focused.

[0077] It should be noted that, in addition to the above situation, if the face area and the hand area are not detected, the entire image screen is used as the area to be focused.

[0078] In an optional embodiment, the position information and confidence information of the target area include: a first position and a first confidence of the second sub-area, a second position and a second confidence of the first sub-area, and selecting the area to be focused from the target area includes:

[0079] Step S702: using a focus region discrimination model to analyze the target region to determine the region to be focused.

[0080] Among them, the above-mentioned focus area discrimination model is a model obtained by machine learning training using multiple sets of data, and each set of data in the above-mentioned multiple sets of data includes: a training image, the above-mentioned first position, the above-mentioned first confidence, the above-mentioned second position, the above-mentioned second confidence, hand posture, and hand state.

[0081] Optionally, the training image, the first position, the first confidence, the second position, the second confidence, the hand posture, and the hand state are used as input data of a deep learning network model, and a focus area discrimination model is obtained by network learning through a large amount of labeled data. Then, the target area obtained by performing target detection on the input image is input into the focus area discrimination model, and the focus area discrimination model is used to analyze the target area to determine the area to be focused r attention .

[0082] In an optional embodiment, the above method further includes:

[0083] Step S802, obtaining a first clarity and a second clarity, wherein the first clarity is the clarity of the area to be focused, and the second clarity is the clarity of the remaining areas of the input image except the area to be focused;

[0084] Step S804: when the first definition is lower than the second definition, adjusting the focus state of the area to be focused, and outputting the adjusted image.

[0085] In the above optional embodiment, if Figure 3 As shown, the regional definition adjustment module can be used to adjust the focus area r in the live video image. attention The clarity adjustment is performed so that the area to be focused is clearly presented in the live video image screen. For example, the first clarity p1 of the area to be focused and the second clarity p2 of the remaining areas in the live video image screen except the above-mentioned area to be focused can be calculated first; and when it is detected that the first clarity p1 of the area to be focused is lower than the second clarity p2 of the remaining areas in the input image, the clarity adjustment interface is used to adjust the focus state of the above-mentioned area to be focused so that it is clearly focused, and the adjusted image is output.

[0086] In an optional embodiment, adjusting the focus state of the area to be focused includes at least one of the following: driving a focus adjustment mechanism of the camera to adjust the focus state; performing clarity enhancement calculation on the area to be focused and adjusting the focus state.

[0087] Optionally, in the embodiments of the present application, the method of using the clarity adjustment interface to adjust the focus state of the above-mentioned area to be focused can be one or more: when the camera hardware has a software driver interface, the software driver interface is used to drive the focus mechanism of the camera to focus the above-mentioned area to be focused, so as to improve the clarity of the area to be focused; or a clarity enhancement algorithm is used to perform clarity enhancement calculations on the above-mentioned area to be focused, and the above-mentioned focus state is adjusted to enhance the clarity of the area to be focused. Through the above embodiments of the present application, after locating the area to be focused in the input image, the clarity of the area to be focused can be adjusted so that the area to be focused can be clearly presented in the live video image.

[0088] Optionally, the above-mentioned clarity enhancement algorithm may include but is not limited to: Brenner gradient function calculation method, Laplacian gradient function calculation method, etc.

[0089] Under the above operating environment, this application provides Figure 5 Another image processing method shown. Figure 5 is a flowchart of another image processing method according to an embodiment of the present application. Figure 5 As shown, the above image processing method includes:

[0090] Step S902, obtaining a to-be-focused area from an input image, wherein the to-be-focused area is used to display information to be focused on;

[0091] Step S904: adjusting the focus state of the above-mentioned area to be focused to obtain an image to be output.

[0092] In an embodiment of the present application, a method is adopted in which a region to be focused in an input image is located and a focusing process is performed on the region to be focused. The region to be focused is obtained from the input image, wherein the region to be focused is used to display information to be focused on. The focusing state of the region to be focused is adjusted to obtain an image to be output.

[0093] It is easy to notice that the embodiment of the present application obtains the image to be output by identifying the input image, locating the area to be focused in the input image for displaying the information to be focused, and adjusting the focus state of the area to be focused. Then, the area to be focused can be clearly displayed by focusing, so as to achieve the purpose of improving the efficiency of determining the target area of ​​the image, thereby achieving the technical effect of clearly displaying the area to be focused, and thus solving the technical problem that the image processing scheme in the prior art cannot effectively determine the area to be focused of the image, resulting in difficulty in clearly displaying the area to be focused.

[0094] In an optional embodiment, the above method is applied to a live broadcast scenario, and the above area to be focused is used to display product recommendation information, wherein the above information to be focused on can be the goods / products recommended by the anchor to the audience during the live broadcast.

[0095] Optionally, the input image may be a live video image. By focusing the area to be focused, an image to be output, that is, a live video image to be output and displayed, may be obtained.

[0096] It should be noted that, through the image processing method provided in the embodiment of the present application, the area to be focused in the image screen is located during the live broadcast process, and after the area to be focused is located, the focus state of the above-mentioned area to be focused is adjusted by focusing to obtain the scene of the image to be output. Specifically, the clarity of the area to be focused can be adjusted by but not limited to software algorithms or hardware driver interfaces.

[0097] As an optional embodiment, a target area in an input image can be located, and an area to be focused can be selected from the target area, by acquiring an input image; performing target detection on the input image to obtain a target area, wherein the target area includes at least one of the following: a facial area, a hand area; and selecting an area to be focused from the target area.

[0098] Taking the application of the above method to a live broadcast scenario as an example, in an embodiment of the present application, the target area in the input image may be located first, for example, the facial area and / or hand area of ​​the host; and then the area to be focused in the above target area may be obtained. Optionally, the position of the area to be focused in the live broadcast video image may be located by detecting the face position, hand position, hand posture, and hand state. For example, when it is detected that the host's hand is located in the center area of ​​the image, and the host's hand posture is in the posture of grasping an object and is maintained for a certain period of time, the object area in the host's hand is considered to be the area to be focused; when the monitored host's hand posture is in a naturally placed posture, the area to be focused is considered to be the host's facial area.

[0099] During the live broadcast process, the live video image screen can be used as the input image. In addition, the input image can also carry other information of the video data, such as time stamp information.

[0100] As an optional embodiment, after acquiring the input image, a target detection method is used to perform target detection on the input image to obtain a target area, which can be obtained by Figure 3 The region discrimination module shown analyzes the target region, or uses a focus region discrimination model to analyze the target region to determine the region to be focused.

[0101] In an optional embodiment, the method further includes: obtaining a first clarity and a second clarity, wherein the first clarity is the clarity of the area to be focused, and the second clarity is the clarity of the remaining areas of the input image except the area to be focused; when the first clarity is lower than the second clarity, adjusting the focus state of the area to be focused to obtain the image to be output.

[0102] In the above optional embodiment, the regional definition adjustment module can be used to adjust the focus area r in the live video image screen. attention The clarity is adjusted so that the area to be focused is clearly presented in the live video image. For example, the first clarity p1 of the area to be focused and the second clarity p2 of the remaining areas in the live video image except the area to be focused can be calculated first; and when it is detected that the first clarity p1 of the area to be focused is lower than the second clarity p2 of the remaining areas in the input image, the clarity adjustment interface is used to adjust the focus state of the area to be focused so that it is clearly focused and the image to be output is obtained.

[0103] In an optional embodiment, adjusting the focus state of the area to be focused includes at least one of the following: driving a focus adjustment mechanism of the camera to adjust the focus state; performing clarity enhancement calculation on the area to be focused and adjusting the focus state.

[0104] Optionally, in the embodiment of the present application, the method of using the clarity adjustment interface to adjust the focus state of the above-mentioned area to be focused can be one or more: when the camera hardware has a software driver interface, the focusing mechanism of the camera is driven by the software driver interface to perform focusing processing on the above-mentioned area to be focused, so as to improve the clarity of the area to be focused; or a clarity enhancement algorithm is used to perform clarity enhancement calculation on the above-mentioned area to be focused, and the above-mentioned focus state is adjusted to enhance the clarity of the area to be focused.

[0105] Optionally, the above-mentioned clarity enhancement algorithm may include but is not limited to: Brenner gradient function calculation method, Laplacian gradient function calculation method, etc.

[0106] Under the above operating environment, this application provides Figure 6 Another image processing method shown. Figure 6 is a flowchart of another image processing method according to an embodiment of the present application. Figure 6 As shown, the above image processing method includes:

[0107] Step S1002, acquiring a first image;

[0108] Step S1004, selecting a to-be-focused area from the first image, wherein the to-be-focused area is used to display information to be focused on;

[0109] Step S1006, adjusting the focus state of the above-mentioned area to be focused to obtain a second image;

[0110] Step S1008: output the second image.

[0111] Optionally, the first image may be an input image, such as a live video image, and the second image may be an image to be output after focusing the area to be focused, such as a live video image to be output and displayed after focusing.

[0112] In an embodiment of the present application, a method is adopted in which an area to be focused in an input image is positioned and a focusing process is performed on the area to be focused. The area to be focused is obtained from the input image, wherein the area to be focused is used to display information to be focused on; the focusing state of the area to be focused is adjusted to obtain an image to be output, and the image to be output is output for display.

[0113] It is easy to notice that the embodiment of the present application identifies the input image, locates the area to be focused in the input image for displaying the information to be paid attention to, adjusts the focus state of the above-mentioned area to be focused, obtains the image to be output, and then can clearly display the above-mentioned area to be focused by focusing, thereby achieving the purpose of improving the efficiency of determining the target area of ​​the image, thereby achieving the technical effect of clearly displaying the area to be focused, and further solves the technical problem that the image processing scheme in the prior art cannot effectively determine the area to be focused of the image, resulting in difficulty in clearly displaying the area to be focused.

[0114] In an optional embodiment, the above method is applied to a live broadcast scenario, and the above area to be focused is used to display product recommendation information, wherein the above information to be focused on can be the goods / products recommended by the anchor to the audience during the live broadcast.

[0115] It should be noted that, through the image processing method provided in the embodiment of the present application, the area to be focused in the image screen is located during the live broadcast process, and after the area to be focused is located, the focus state of the above-mentioned area to be focused is adjusted by focusing to obtain the scene of the image to be output. Specifically, the clarity of the area to be focused can be adjusted by but not limited to software algorithms or hardware driver interfaces.

[0116] As an optional embodiment, a target area in an input image can be located, and an area to be focused can be selected from the target area, by acquiring an input image; performing target detection on the input image to obtain a target area, wherein the target area includes at least one of the following: a first sub-area and a second sub-area, the first sub-area and the second sub-area respectively corresponding to different parts of the human body; and selecting an area to be focused from the target area.

[0117] Taking the application of the above method to a live broadcast scenario as an example, in an embodiment of the present application, the target area in the input image may be located first, for example, the facial area and / or hand area of ​​the host; and then the area to be focused in the above target area may be obtained. Optionally, the position of the area to be focused in the live broadcast video image may be located by detecting the face position, hand position, hand posture, and hand state. For example, when it is detected that the host's hand is located in the center area of ​​the image, and the host's hand posture is in the posture of grasping an object and is maintained for a certain period of time, the object area in the host's hand is considered to be the area to be focused; when the monitored host's hand posture is in a naturally placed posture, the area to be focused is considered to be the host's facial area.

[0118] During the live broadcast, the live video image can be used as the input image. In addition, the input image can also carry other information of the video data, such as timestamp information. After obtaining the input image, the input image can be sent to the face and hand detection module, and the target detection method is used to perform target detection on the input image to obtain the target area, and the target area is analyzed by the area discrimination module or the focus area discrimination model to determine the area to be focused.

[0119] In an optional embodiment, the method further includes: obtaining a first clarity and a second clarity, wherein the first clarity is the clarity of the area to be focused, and the second clarity is the clarity of the remaining areas of the input image except the area to be focused; when the first clarity is lower than the second clarity, adjusting the focus state of the area to be focused to obtain the image to be output.

[0120] In the above optional embodiment, the regional definition adjustment module can be used to adjust the focus area r in the live video image screen. attention The clarity is adjusted so that the area to be focused is clearly presented in the live video image. For example, the first clarity p1 of the area to be focused and the second clarity p2 of the remaining areas in the live video image except the area to be focused can be calculated first; and when it is detected that the first clarity p1 of the area to be focused is lower than the second clarity p2 of the remaining areas in the input image, the clarity adjustment interface is used to adjust the focus state of the area to be focused so that it is clearly focused and the image to be output is obtained.

[0121] In an optional embodiment, adjusting the focus state of the area to be focused includes at least one of the following: driving a focus adjustment mechanism of the camera to adjust the focus state; performing clarity enhancement calculation on the area to be focused and adjusting the focus state.

[0122] Optionally, in the embodiment of the present application, the method of using the clarity adjustment interface to adjust the focus state of the above-mentioned area to be focused can be one or more: when the camera hardware has a software driver interface, the focusing mechanism of the camera is driven by the software driver interface to perform focusing processing on the above-mentioned area to be focused, so as to improve the clarity of the area to be focused; or a clarity enhancement algorithm is used to perform clarity enhancement calculation on the above-mentioned area to be focused, and the above-mentioned focus state is adjusted to enhance the clarity of the area to be focused.

[0123] Optionally, the above-mentioned clarity enhancement algorithm may include but is not limited to: Brenner gradient function calculation method, Laplacian gradient function calculation method, etc.

[0124] Under the above operating environment, this application provides Figure 7An image processing method shown in FIG. Figure 7 is a flowchart of another image processing method according to an embodiment of the present application. Figure 7 As shown, the above image processing method includes:

[0125] Step S1102, obtaining an input image;

[0126] Step S1104, performing target detection on the input image to obtain a target area, wherein the target area includes at least one of the following: a first sub-area and a second sub-area, the first sub-area includes a first target object, the first target object has vital signs, and the second sub-area includes a second target object, the second target object does not have vital signs and is an operation object of the first target object;

[0127] Step S1106, selecting a to-be-focused area from the target area, wherein the to-be-focused area is used to display information to be focused on.

[0128] In an optional embodiment, the above method is applied to an industrial scene, and the above area to be focused is used to display equipment information, wherein the above information to be focused on may be equipment manufactured in an industrial process.

[0129] Optionally, the input image is an industrial video image, the first sub-area is a person, the second sub-area is a device, the first target object is a hand, and the second target object is a device held by the hand, such as goods, commodities, etc.

[0130] It should be noted that, through the image processing method provided in the embodiment of the present application, the area to be focused in the image screen is located in the industrial preparation process, and after the area to be focused is located, the focus state of the above-mentioned area to be focused is adjusted by focusing to obtain the scene of the image to be output. Specifically, the clarity of the area to be focused can be adjusted by, but not limited to, software algorithms or hardware driver interfaces.

[0131] As an optional embodiment, a target area in an input image can be located, and an area to be focused can be selected from the target area, by acquiring an input image; performing target detection on the input image to obtain a target area, wherein the target area includes at least one of the following: a first sub-area and a second sub-area, the first sub-area containing a first target object, the first target object having vital signs, the second sub-area containing a second target object, the second target object having no vital signs and being an operation object of the first target object; and selecting an area to be focused from the target area.

[0132] Taking the application of the above method in industrial scenarios as an example, in an embodiment of the present application, the target area in the input image, for example, the facial area and / or hand area of ​​the worker, can be located first; and then the area to be focused in the above target area can be obtained.

[0133] Optionally, the position of the area to be focused in the industrial video image can be located by detecting the hand area, hand posture, and hand state. For example, when the worker's hand area is detected in the central area of ​​the image, and the posture of the worker's hand (first target object) is that of grasping an object (second target object) and is maintained for a certain time, the device in the worker's hand is considered to be the area to be focused.

[0134] In an optional embodiment, an industrial image screen can be used as an input image. In addition, the input image can also carry other information of the video data, such as timestamp information. After obtaining the above input image, the input image can be sent to the face and hand detection module, and the target detection method is used to perform target detection on the above input image to obtain the target area, and the above target area is analyzed by the area discrimination module or the focus area discrimination model to determine the above area to be focused.

[0135] It is easy to notice that the embodiment of the present application performs target detection on the above-mentioned input image to obtain the target area, and selects the area to be focused from the above-mentioned target area, thereby achieving the purpose of improving the efficiency of determining the target area of ​​the image, thereby achieving the technical effect of clearly displaying the area to be focused, and further solving the technical problem that the image processing scheme in the prior art cannot effectively determine the area to be focused of the image, resulting in difficulty in clearly displaying the area to be focused.

[0136] Under the above operating environment, this application provides Figure 8 An image processing method shown in FIG. Figure 8 is a flowchart of another image processing method according to an embodiment of the present application. Figure 8 As shown, the above image processing method includes:

[0137] Step S1202, obtaining an input image;

[0138] Step S1204, performing target detection on the input image to obtain a target region, wherein the target region includes at least one of the following: a first sub-region and a second sub-region, the first sub-region includes a first target object, a local body of the first target object has a lesion, and the second sub-region includes a second target object, and the second target object is an internal organ;

[0139] Step S1206: selecting a to-be-focused area from the target area, wherein the to-be-focused area is used to display information to be focused on.

[0140] Optionally, the input image is a medical video image, the first sub-region is a lesion region, the second sub-region is an organ region, a lesion occurs in a local body of the first target object, and the second target object is an internal organ in the organ region.

[0141] In an optional embodiment, the above method is applied to a medical AI scenario, and the above area to be focused is used to display device information, wherein the above information to be focused on may be an internal organ to be focused on.

[0142] It should be noted that, through the image processing method provided in the embodiment of the present application, the area to be focused in the image screen is located during the process of treating the patient, and after the area to be focused is located, the focus state of the above-mentioned area to be focused is adjusted by focusing to obtain the scene of the image to be output. Specifically, the clarity of the area to be focused can be adjusted by, but not limited to, software algorithms or hardware driver interfaces.

[0143] As an optional embodiment, the target area in the input image can be located, and the area to be focused can be selected from the target area, by acquiring the input image; performing target detection on the input image to obtain the target area, wherein the target area includes at least one of the following: a first sub-area and a second sub-area, the first sub-area contains a first target object, and a local body of the first target object has a lesion, and the second sub-area contains a second target object, and the second target object is an internal organ; and the area to be focused is selected from the target area.

[0144] In an optional embodiment, a medical image screen can be used as an input image. In addition, the input image can also carry other information of the video data, such as timestamp information. After obtaining the above input image, the input image can be sent to the face and hand detection module, and the target detection method is used to perform target detection on the above input image to obtain the target area, and the target area is analyzed by the area discrimination module or the focus area discrimination model to determine the above area to be focused.

[0145] It is easy to notice that the embodiment of the present application performs target detection on the above-mentioned input image to obtain the target area, and selects the area to be focused from the above-mentioned target area, thereby achieving the purpose of improving the efficiency of determining the target area of ​​the image, thereby achieving the technical effect of clearly displaying the area to be focused, and further solving the technical problem that the image processing scheme in the prior art cannot effectively determine the area to be focused of the image, resulting in difficulty in clearly displaying the area to be focused.

[0146] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0147] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a non-volatile storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the above methods of each embodiment of the present invention.

[0148] Example 2

[0149] According to an embodiment of the present application, a device embodiment for implementing the above-mentioned image processing method is also provided. Fig. 9 is a structural schematic diagram of an image processing device according to an embodiment of the present application, such as Fig. 9 As shown, the device includes: a first acquisition module 70, a detection module 72 and a first selection module 74, wherein:

[0150] The first acquisition module 70 is used to acquire an input image; the detection module 72 is used to perform target detection on the above input image to obtain a target area, wherein the above target area includes at least one of the following: a first sub-area and a second sub-area, and the first sub-area and the second sub-area correspond to different parts of the human body respectively; the first selection module 74 is used to select a to-be-focused area from the above target area, wherein the above to-be-focused area is used to display information to be paid attention to.

[0151] It should be noted that the first acquisition module 70, the detection module 72 and the first selection module 74 correspond to steps S202 to S206 in Example 1, and the three modules and the corresponding steps implement the same examples and application scenarios, but are not limited to the contents disclosed in the above-mentioned Example 1. It should be noted that the above-mentioned modules, as part of the device, can be run in the computer terminal 10 provided in Example 1.

[0152] According to an embodiment of the present application, another device embodiment for implementing the above-mentioned image processing method is also provided. Fig.10 is a schematic diagram of the structure of another image processing device according to an embodiment of the present application. Fig.10 As shown, the device includes: a second acquisition module 80 and a first adjustment module 82, wherein:

[0153] The second acquisition module 80 is used to acquire the area to be focused from the input image, wherein the area to be focused is used to display the information to be focused; the first adjustment module 82 is used to adjust the focus state of the area to be focused to obtain the image to be output.

[0154] It should be noted that the second acquisition module 80 and the first adjustment module 82 correspond to steps S902 to S904 in Example 1, and the examples and application scenarios implemented by the two modules and the corresponding steps are the same, but are not limited to the contents disclosed in the above-mentioned Example 1. It should be noted that the above-mentioned modules, as part of the device, can be run in the computer terminal 10 provided in Example 1.

[0155] According to an embodiment of the present application, another device embodiment for implementing the above-mentioned image processing method is also provided. Fig.11 is a schematic diagram of the structure of another image processing device according to an embodiment of the present application. Fig.11 As shown, the device includes: a third acquisition module 90, a second selection module 92, a second adjustment module 94 and an output module 96, wherein:

[0156] The third acquisition module 90 is used to acquire the first image; the second selection module 92 is used to select the area to be focused from the above-mentioned first image, wherein the above-mentioned area to be focused is used to display the information to be paid attention to; the second adjustment module 94 is used for the adjustment module to adjust the focus state of the above-mentioned area to be focused to obtain the second image; the output module 96 is used to output the above-mentioned second image.

[0157] It should be noted that the third acquisition module 90, the second selection module 92, the second adjustment module 94 and the output module 96 correspond to steps S1002 to S1008 in Example 1, and the four modules and the corresponding steps implement the same examples and application scenarios, but are not limited to the contents disclosed in the above-mentioned Example 1. It should be noted that the above-mentioned modules, as part of the device, can be run in the computer terminal 10 provided in Example 1.

[0158] It should also be noted that the preferred implementation of this embodiment can refer to the relevant description in Example 1, which will not be repeated here.

[0159] Example 3

[0160] According to an embodiment of the present application, an embodiment of an image processing device is also provided. The image processing device may be any computing device in a computing device group. Fig.12 is a structural schematic diagram of an image processing device according to an embodiment of the present application, such as Fig.12 As shown, the image processing device includes: a processor 1000 and a memory 1002, wherein:

[0161] Processor 1000; and memory 1002, connected to the processor 1000, for providing instructions for the processor to process the following processing steps: acquiring an input image; performing target detection on the input image to obtain a target area, wherein the target area includes at least one of the following: a first sub-area and a second sub-area, the first sub-area and the second sub-area respectively corresponding to different parts of the human body; selecting a to-be-focused area from the target area, wherein the to-be-focused area is used to display information to be focused on.

[0162] In an embodiment of the present application, a method of locating a target area in an input image and selecting an area to be focused from the target area is adopted, wherein an input image is acquired; target detection is performed on the input image to obtain a target area, wherein the target area includes at least one of the following: a first sub-area and a second sub-area, the first sub-area and the second sub-area respectively corresponding to different parts of the human body; and an area to be focused is selected from the target area, wherein the area to be focused is used to display information to be focused.

[0163] It is easy to notice that the embodiment of the present application identifies the input image and locates the first sub-region or the second sub-region in the input image. The area to be focused for displaying the information to be paid attention to can be selected from the first sub-region or the second sub-region, and then the area to be focused can be clearly displayed by focusing, thereby achieving the purpose of improving the efficiency of determining the target area of ​​the image, thereby achieving the technical effect of clearly displaying the area to be focused, and thus solving the technical problem that the image processing scheme in the prior art cannot effectively determine the area to be focused of the image, resulting in difficulty in clearly displaying the area to be focused.

[0164] It should also be noted that the preferred implementation of this embodiment can refer to the relevant description in Example 1, which will not be repeated here.

[0165] Example 4

[0166] According to an embodiment of the present application, an embodiment of a computer terminal is also provided, and the computer terminal can be any computer terminal device in a computer terminal group. Optionally, in this embodiment, the above-mentioned computer terminal can also be replaced by a terminal device such as a mobile terminal.

[0167] Optionally, in this embodiment, the computer terminal may be located in at least one network device among a plurality of network devices of the computer network.

[0168] In this embodiment, the above-mentioned computer terminal can execute the program code of the following steps in the image processing method: obtaining an input image; performing target detection on the above-mentioned input image to obtain a target area, wherein the above-mentioned target area includes at least one of the following: a first sub-area and a second sub-area, and the first sub-area and the second sub-area correspond to different parts of the human body respectively; selecting a to-be-focused area from the above-mentioned target area, wherein the above-mentioned to-be-focused area is used to display information to be paid attention to.

[0169] Optionally, Fig.13 is a structural block diagram of another computer terminal according to an embodiment of the present application, such as Fig.13 As shown, the computer terminal may include: one or more (only one is shown in the figure) processors 1102 , a memory 1104 , and a peripheral interface 1106 .

[0170] Among them, the memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the image processing method and device in the embodiment of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, realizing the above-mentioned image processing method. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the computer terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0171] The processor can call the information and application stored in the memory through the transmission device to execute the following steps: obtain an input image; perform target detection on the above input image to obtain a target area, wherein the above target area includes at least one of the following: a first sub-area and a second sub-area, and the first sub-area and the second sub-area correspond to different parts of the human body respectively; select a to-be-focused area from the above target area, wherein the above to-be-focused area is used to display information to be paid attention to.

[0172] Optionally, the processor may also execute program code of the following steps: detecting the position information and confidence information of the target area from the input image using a target detection method.

[0173] Optionally, the processor may also execute program codes of the following steps: a single-stage target detection method using a candidate box, a single-stage target detection method without using a candidate box, and a two-stage target detection method using a candidate box.

[0174] Optionally, the processor may also execute the program code of the following steps: when the first position is detected and the first confidence level is greater than a preset threshold, obtaining the second sub-region from the target region; performing hand posture estimation on the second sub-region by using a key point detection method to obtain a first estimation result; performing hand state estimation on the second sub-region by using a classification and discrimination method to obtain a second estimation result.

[0175] Optionally, the processor may further execute program code of the following steps: when a first sub-region is detected from the target region, selecting the first sub-region as the region to be focused.

[0176] Optionally, the processor may also execute the program code of the following steps: when a second sub-area is detected from the target area, and when it is determined based on the first estimation result and the second estimation result that the hand posture and the hand state meet preset conditions, the second sub-area is selected as the area to be focused.

[0177] Optionally, the processor may also execute program code for the following steps: when the first sub-region and the second sub-region are detected from the target area, and it is determined based on the first estimation result and the second estimation result that the hand posture and the hand state meet the preset conditions, the second sub-region is selected as the area to be focused; when the first sub-region and the second sub-region are detected from the target area, and it is determined based on the first estimation result and the second estimation result that the hand posture and the hand state do not meet the preset conditions, the first sub-region is selected as the area to be focused.

[0178] Optionally, the processor may also execute the program code of the following steps: analyzing the target area by using a focus area discrimination model to determine the area to be focused, wherein the focus area discrimination model is a model obtained by machine learning training using multiple sets of data, and each set of data in the multiple sets of data includes: a training image, the first position, the first confidence, the second position, the second confidence, a hand posture, and a hand state.

[0179] Optionally, the processor may also execute program codes of the following steps: obtaining a first clarity and a second clarity, wherein the first clarity is the clarity of the area to be focused, and the second clarity is the clarity of the remaining areas in the input image except the area to be focused; when the first clarity is lower than the second clarity, adjusting the focus state of the area to be focused, and outputting the adjusted image.

[0180] Optionally, the processor may also execute program codes of the following steps: driving a focusing mechanism of the camera to adjust the focusing state; performing clarity enhancement calculation on the area to be focused, and adjusting the focusing state.

[0181] Optionally, the processor may also execute program codes of the following steps: obtaining a region to be focused from an input image, wherein the region to be focused is used to display information to be focused on; and adjusting a focus state of the region to be focused to obtain an image to be output.

[0182] Optionally, the processor may also execute program codes of the following steps: acquiring a first image; selecting an area to be focused from the first image, wherein the area to be focused is used to display information to be focused; adjusting a focus state of the area to be focused to obtain a second image; and outputting the second image.

[0183] According to an embodiment of the present application, an input image is acquired by locating a target area in an input image and selecting an area to be focused from the target area; target detection is performed on the input image to obtain a target area, wherein the target area includes at least one of the following: a first sub-area and a second sub-area, the first sub-area and the second sub-area respectively corresponding to different parts of the human body; and an area to be focused is selected from the target area, wherein the area to be focused is used to display information to be focused.

[0184] It is easy to notice that the embodiment of the present application identifies the input image and locates the first sub-region or the second sub-region in the input image. The area to be focused for displaying the information to be paid attention to can be selected from the first sub-region or the second sub-region, and then the area to be focused can be clearly displayed by focusing, thereby achieving the purpose of improving the efficiency of determining the target area of ​​the image, thereby achieving the technical effect of clearly displaying the area to be focused, and thus solving the technical problem that the image processing scheme in the prior art cannot effectively determine the area to be focused of the image, resulting in difficulty in clearly displaying the area to be focused.

[0185] It can be understood by those skilled in the art that Fig.13 The structure shown is for illustration only, and the computer terminal may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a handheld computer, a mobile Internet device (Mobile Internet Devices, MID), a PAD, or other terminal devices. Fig.13 It does not limit the structure of the above electronic device. For example, the computer terminal may also include Fig.13 More or fewer components (such as network interfaces, display devices, etc.) shown in, or having Fig.13 Different configurations are shown.

[0186] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable non-volatile storage medium. The non-volatile storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0187] Example 5

[0188] According to an embodiment of the present application, an embodiment of a non-volatile storage medium is also provided. Optionally, in the embodiment of the present application, the non-volatile storage medium can be used to store the program code executed by the image processing method provided in the above embodiment 1.

[0189] Optionally, in this embodiment, the non-volatile storage medium may be located in any computer terminal in a computer terminal group in a computer network, or in any mobile terminal in a mobile terminal group.

[0190] Optionally, in this embodiment, the non-volatile storage medium is configured to store program codes for executing the following steps: acquiring an input image; performing target detection on the input image to obtain a target area, wherein the target area includes at least one of the following: a first sub-area and a second sub-area, the first sub-area and the second sub-area respectively corresponding to different parts of the human body; selecting a to-be-focused area from the target area, wherein the to-be-focused area is used to display information to be paid attention to.

[0191] Optionally, in this embodiment, the non-volatile storage medium is configured to store program codes for executing the following steps: obtaining a to-be-focused area from an input image, wherein the to-be-focused area is used to display information to be paid attention to; and adjusting a focus state of the to-be-focused area to obtain an image to be output.

[0192] Optionally, in this embodiment, the non-volatile storage medium is configured to store program codes for executing the following steps: acquiring a first image; selecting an area to be focused from the first image, wherein the area to be focused is used to display information to be paid attention to; adjusting a focus state of the area to be focused to obtain a second image; and outputting the second image.

[0193] Optionally, in this embodiment, the non-volatile storage medium is configured to store program codes for executing the following steps:

[0194] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0195] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0196] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0197] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0198] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0199] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable non-volatile storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a non-volatile storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned non-volatile storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0200] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. An image processing method, characterized in that: include: Get the input image; Performing target detection on the input image to obtain a target area, wherein the target area includes the following: a first sub-area and a second sub-area, the first sub-area and the second sub-area respectively correspond to different human body parts, the first sub-area is a face area, and the second sub-area is a hand area; Performing hand posture estimation on the second sub-region to obtain a first estimation result, and performing hand state estimation on the second sub-region to obtain a second estimation result; Selecting a to-be-focused area from the target area, wherein the to-be-focused area is used to display information to be focused on; The selecting of the area to be focused from the target area includes: when the first sub-area and the second sub-area are detected from the target area, and it is determined based on the first estimation result and the second estimation result that the hand posture and the hand state meet a preset condition, the second sub-area is selected as the area to be focused, wherein the preset condition is used to indicate that the hand posture is a posture of displaying an object and the hand state is a state of holding an object, or the hand state is a state of holding an object and blocking the background.

2. The method according to claim 1, characterized in that Performing target detection on the input image, obtaining the target area includes: The position information and confidence information of the target area are detected from the input image using a target detection method.

3. The method according to claim 2, characterized in that The target detection method includes at least one of the following: A single-stage target detection method using a candidate box, a single-stage target detection method without a candidate box, and a two-stage target detection method using a candidate box.

4. The method according to claim 2, characterized in that: The position information and confidence information of the target area include: a first position and a first confidence of the second sub-area, and the performing hand posture estimation on the second sub-area to obtain a first estimation result, and the performing hand state estimation on the second sub-area to obtain a second estimation result include: When the first position is detected and the first confidence level is greater than a preset threshold, acquiring the second sub-region from the target region; Using a key point detection method to estimate the hand posture of the second sub-region to obtain a first estimation result; The hand state of the second sub-region is estimated by using a classification and discrimination method to obtain a second estimation result.

5. The method according to claim 1, characterized in that Selecting the area to be focused from the target area includes: When only the first sub-region is detected from the target region, the first sub-region is selected as the region to be focused.

6. The method according to claim 4, characterized in that Selecting the area to be focused from the target area includes: When only the second sub-region is detected from the target region, and it is determined based on the first estimation result and the second estimation result that the hand posture and the hand state meet a preset condition, the second sub-region is selected as the region to be focused.

7. The method according to claim 4, characterized in that Selecting the area to be focused from the target area includes: When the first sub-region and the second sub-region are detected from the target region, and it is determined based on the first estimation result and the second estimation result that the hand posture and the hand state do not meet the preset condition, the first sub-region is selected as the area to be focused.

8. The method according to claim 1, characterized in that The position information and confidence information of the target area include: a first position and a first confidence of the second sub-area, a second position and a second confidence of the first sub-area, and selecting the area to be focused from the target area includes: The target area is analyzed by using a focus area discrimination model to determine the area to be focused, wherein the focus area discrimination model is a model obtained by machine learning training using multiple sets of data, and each set of data in the multiple sets of data includes: a training image, the first position, the first confidence, the second position, the second confidence, a hand posture, and a hand state.

9. The method according to claim 1, characterized in that: The method further comprises: Acquire a first clarity and a second clarity, wherein the first clarity is the clarity of the area to be focused, and the second clarity is the clarity of the remaining areas of the input image except the area to be focused; When the first definition is lower than the second definition, the focus state of the area to be focused is adjusted, and the adjusted image is output.

10. The method according to claim 9, characterized in that Adjusting the focus state of the area to be focused includes at least one of the following: Driving a focusing mechanism of the camera to adjust the focusing state; Perform clarity enhancement calculation on the area to be focused and adjust the focus state.

11. The method according to claim 1, characterized in that: The method is applied to a live webcasting scenario, and the area to be focused is used to display product recommendation information.

12. An image processing method, characterized in that: include: Acquire a region to be focused from an input image, wherein the region to be focused is used to display information to be focused, and the region to be focused is selected from a target region, and the target region includes the following: a first sub-region and a second sub-region, the first sub-region and the second sub-region respectively correspond to different parts of the human body, the first sub-region is a face region, the second sub-region is a hand region, and the target region is obtained by performing target detection on the input image; Performing hand posture estimation on the second sub-region to obtain a first estimation result, and performing hand state estimation on the second sub-region to obtain a second estimation result; The selecting the area to be focused from the target area comprises: when the first sub-area and the second sub-area are detected from the target area, and it is determined based on the first estimation result and the second estimation result that the hand posture and the hand state meet a preset condition, selecting the second sub-area as the area to be focused, wherein the preset condition is used to indicate that the hand posture is a posture of displaying an object and the hand state is a state of holding an object, or the hand state is a state of holding an object and blocking a background; The focus state of the area to be focused is adjusted to obtain an image to be output.

13. An image processing method, characterized in that: include: acquiring a first image; Selecting a region to be focused from the first image, wherein the region to be focused is used to display information to be focused, and the region to be focused is selected from a target region, and the target region includes the following: a first sub-region and a second sub-region, the first sub-region and the second sub-region respectively correspond to different parts of the human body, the first sub-region is a face region, the second sub-region is a hand region, and the target region is obtained by performing target detection on the first image; Performing hand posture estimation on the second sub-region to obtain a first estimation result, and performing hand state estimation on the second sub-region to obtain a second estimation result; The selecting the area to be focused from the target area comprises: when the first sub-area and the second sub-area are detected from the target area, and it is determined based on the first estimation result and the second estimation result that the hand posture and the hand state meet a preset condition, selecting the second sub-area as the area to be focused, wherein the preset condition is used to indicate that the hand posture is a posture of displaying an object and the hand state is a state of holding an object, or the hand state is a state of holding an object and blocking a background; Adjusting the focus state of the area to be focused to obtain a second image; The second image is output.

14. A non-volatile storage medium, characterized in that: The non-volatile storage medium includes a stored program, wherein when the program is executed, the device where the non-volatile storage medium is located is controlled to execute the image processing method according to any one of claims 1 to 13.

15. An image processing device, characterized in that: include: processor; as well as A memory, connected to the processor, configured to provide the processor with instructions for processing the following processing steps: Get the input image; Performing target detection on the input image to obtain a target area, wherein the target area includes the following: a first sub-area and a second sub-area, the first sub-area and the second sub-area respectively correspond to different human body parts, the first sub-area is a face area, and the second sub-area is a hand area; Performing hand posture estimation on the second sub-region to obtain a first estimation result, and performing hand state estimation on the second sub-region to obtain a second estimation result; A to-be-focused area is selected from the target area, wherein the to-be-focused area is used to display information to be paid attention to; the selecting of the to-be-focused area from the target area comprises: when the first sub-area and the second sub-area are detected from the target area, and it is determined based on the first estimation result and the second estimation result that the hand posture and the hand state satisfy a preset condition, the second sub-area is selected as the to-be-focused area, wherein the preset condition is used to indicate that the hand posture is a posture of displaying an object and the hand state is a state of holding an object, or the hand state is a state of holding an object and blocking the background.

Images