Image processing method and related device

Abstract

The present disclosure provides an image processing method and related equipment. The method comprises: acquiring a to-be-processed image; determining a key point, a pose angle and a target region contour point of a target object in the to-be-processed image; determining a center reference point of the target object based on the key point and the pose angle; and adjusting the position of the target region contour point based on the center reference point to obtain a target image.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to an image processing method and related equipment. Background Technology

[0002] In existing image processing methods, a fixed set of points within an object is typically used as reference points to process the object's contour, making it more consistent with user expectations. For example, the center of the head is manually determined, and the top contour is deformed accordingly to better match the user's desired shape. However, this method is inconsistent in its effectiveness. On one hand, manually determined center points are often inaccurate, with significant deviations. On the other hand, manual deformation cannot adapt to different head angles, resulting in poor contour processing when the head is at certain angles. Summary of the Invention

[0003] This disclosure proposes an image processing method, apparatus, device, storage medium, and program product to address, to some extent, the technical problem of poor contour processing results in image processing.

[0004] In a first aspect, this disclosure provides an image processing method, comprising:

[0005] Obtain the image to be processed;

[0006] Determine the key points, pose angles, and contour points of the target region of the target object in the image to be processed;

[0007] The center reference point of the target object is determined based on the key points and attitude angles;

[0008] The target image is obtained by adjusting the position of the contour points of the target region based on the central reference point.

[0009] A second aspect of this disclosure provides an image processing apparatus, comprising:

[0010] The acquisition module is used to acquire the image to be processed;

[0011] The detection module is used to determine the key points, pose angles, and contour points of the target area of ​​the target object in the image to be processed.

[0012] The reference point module is used to determine the center reference point of the target object based on the key points and attitude angles;

[0013] The contour adjustment module is used to adjust the position of the contour points of the target region based on the central reference point to obtain the target image.

[0014] A third aspect of this disclosure provides an electronic device, characterized in that it includes one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and executed by the one or more processors, the programs including instructions for performing the method according to the first or second aspect.

[0015] A fourth aspect of this disclosure provides a non-volatile computer-readable storage medium comprising a computer program that, when executed by one or more processors, causes the processors to perform the method described in the first or second aspect.

[0016] A fifth aspect of this disclosure provides a computer program product including computer program instructions that, when executed on a computer, cause the computer to perform the method described in the first aspect.

[0017] As can be seen from the above, the image processing method and related apparatus provided in this disclosure determine a central reference point by using the pose angle and key points of the target object, and then adjust the contour based on this central reference point to obtain the target image. This approach of adjusting the contour of the target object by combining its pose angle improves the reliability of the reference point at different angles, thereby enhancing the effectiveness and reliability of contour processing. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this disclosure or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of the image processing architecture according to an embodiment of the present disclosure.

[0020] Figure 2 This is a schematic diagram of the hardware structure of an exemplary electronic device according to an embodiment of the present disclosure.

[0021] Figure 3 This is a schematic flowchart illustrating an image processing method according to an embodiment of the present disclosure.

[0022] Figures 4-5 This is a schematic diagram of the attitude angles according to an embodiment of the present disclosure.

[0023] Figure 6 This is a schematic diagram of cranial contour point detection according to an embodiment of the present disclosure.

[0024] Figure 7 This is a schematic diagram of image processing according to an embodiment of the present disclosure.

[0025] Figure 8 This is a schematic diagram of an image processing apparatus according to an embodiment of the present disclosure. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description is provided in conjunction with specific embodiments and the accompanying drawings.

[0027] It should be noted that, unless otherwise defined, the technical or scientific terms used in the embodiments of this disclosure should have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms "first," "second," and similar terms used in the embodiments of this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0028] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0029] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0030] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0031] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0032] Figure 1 A schematic diagram of an image processing architecture according to an embodiment of the present disclosure is shown. (Reference) Figure 1 The image processing architecture 100 may include a server 110, a terminal 120, and a network 130 providing a communication link. The server 110 and the terminal 120 can be connected via a wired or wireless network 130. The server 110 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, security services, and CDN.

[0033] Terminal 120 can be implemented in hardware or software. For example, when terminal 120 is implemented in hardware, it can be various electronic devices with a display screen and support page display, including but not limited to smartphones, tablets, e-book readers, laptops, and desktop computers. When terminal 120 is implemented in software, it can be installed in the electronic devices listed above; it can be implemented as multiple software programs or software modules (e.g., software programs or software modules used to provide distributed services) or as a single software program or software module, without specific limitations.

[0034] It should be noted that the image processing method provided in this application embodiment can be executed by the terminal 120 or by the server 110. It should be understood that... Figure 1 The number of terminals, networks, and servers shown is for illustrative purposes only and is not intended to be a limitation. Any number of terminals, networks, and servers can be used depending on implementation needs.

[0035] Figure 2 A schematic diagram of the hardware structure of an exemplary electronic device 200 provided in an embodiment of this disclosure is shown. For example... Figure 2 As shown, the electronic device 200 may include: a processor 202, a memory 204, a network module 206, a peripheral interface 208, and a bus 210. The processor 202, memory 204, network module 206, and peripheral interface 208 are interconnected within the electronic device 200 via the bus 210.

[0036] Processor 202 may be a central processing unit (CPU), image processor, neural network processor (NPU), microcontroller (MCU), programmable logic device, digital signal processor (DSP), application-specific integrated circuit (ASIC), or one or more integrated circuits. Processor 202 can be used to perform functions related to the techniques described in this disclosure. In some embodiments, processor 202 may also include multiple processors integrated as a single logic component. For example, such as... Figure 2 As shown, processor 202 may include multiple processors 202a, 202b and 202c.

[0037] Memory 204 can be configured to store data (e.g., instructions, computer code, etc.). Figure 2 As shown, the data stored in memory 204 may include program instructions (e.g., program instructions for implementing the image processing method of embodiments of this disclosure) and data to be processed (e.g., the memory may store configuration files of other modules, etc.). Processor 202 may also access the program instructions and data stored in memory 204 and execute the program instructions to operate on the data to be processed. Memory 204 may include volatile or non-volatile storage devices. In some embodiments, memory 204 may include random access memory (RAM), read-only memory (ROM), optical disk, magnetic disk, hard disk, solid-state drive (SSD), flash memory, memory stick, etc.

[0038] Network module 206 can be configured to provide communication with other external devices to electronic device 200 via a network. This network can be any wired or wireless network capable of transmitting and receiving data. For example, the network can be a wired network, a local wireless network (e.g., Bluetooth, WiFi, Near Field Communication (NFC), etc.), a cellular network, the Internet, or a combination thereof. It is understood that the type of network is not limited to the specific examples described above. In some embodiments, network module 306 may include any combination of any number of network interface controllers (NICs), radio frequency modules, transceivers, modems, routers, gateways, adapters, cellular network chips, etc.

[0039] The peripheral interface 208 can be configured to connect the electronic device 200 to one or more peripheral devices to enable information input and output. For example, peripheral devices may include input devices such as keyboards, mice, touchpads, touch screens, microphones, and various sensors, as well as output devices such as displays, speakers, vibrators, and indicator lights.

[0040] Bus 210 can be configured to transfer information between various components of electronic device 200 (e.g., processor 202, memory 204, network module 206, and peripheral interface 208), such as internal buses (e.g., processor-memory bus), external buses (USB port, PCI-E bus), etc.

[0041] It should be noted that although the architecture of the above-described electronic device 200 only shows the processor 202, memory 204, network module 206, peripheral interface 208, and bus 210, in specific implementations, the architecture of the electronic device 200 may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the architecture of the above-described electronic device 200 may only include the components necessary for implementing the embodiments of this disclosure, and does not necessarily include all the components shown in the figures.

[0042] Image processing often requires contour adjustments, and existing contour adjustment processes typically use the center point of the object being adjusted as a reference point. For example, when optimizing the contour of the skull in an image, a center point is often manually selected, and the contour of the skull is manually adjusted based on that center point. However, due to the instability of manual adjustments, the contour processing results are often poor, the processing efficiency is low, and the workload for users is increased. Therefore, improving the contour processing effect, reliability, and efficiency of images has become an urgent technical problem to be solved.

[0043] Therefore, this disclosure provides an image processing method and related apparatus. A central reference point is determined by the pose angle and key points of the target object, and the contour is adjusted based on this central reference point to obtain the target image. This contour adjustment of the target object in conjunction with its pose angle improves the reliability of the reference point at different angles, thereby enhancing the effectiveness and reliability of contour processing.

[0044] See Figure 3 , Figure 3 A schematic flowchart of an image processing method according to an embodiment of the present disclosure is shown. The image processing method according to an embodiment of the present disclosure can be deployed on a terminal, such as a mobile phone, tablet computer, etc. Figure 3 In the image processing method 300, the following steps may be further included.

[0045] In step S310, the image to be processed is acquired.

[0046] The image to be processed can be uploaded locally or obtained via the network, and can be obtained in real time or not. The image to be processed can be image data or video frames from video data.

[0047] In step S320, the key points, pose angles, and contour points of the target area of ​​the target object in the image to be processed are determined.

[0048] The target object can refer to an object in the image, such as a person. Keypoints refer to points in the image that represent the features of the target object. Attitude angles (or Euler angles) refer to the angles of the target object in the body coordinate system; attitude angles can include yaw, pitch, and roll. See also Figure 4-5 , Figure 4-5 A schematic diagram of attitude angles according to an embodiment of the present disclosure is shown. For example... Figure 4 As shown, the aircraft coordinate system can be a spatial coordinate system XYZ, the pitch angle can be a rotation around the X-axis, the yaw angle can be a rotation around the Y-axis, and the roll angle can be a rotation around the Z-axis. Corresponding to the target object, for example, when the target object is a face, the pitch angle can correspond to tilting the head up or down, the yaw angle can correspond to tilting the head left or right, and the roll angle can correspond to turning the head to the left or right shoulder. Figure 5 As shown. The target region can refer to at least a portion of the target object, and the contour points of the target region can be partial contour points of the target object. In some embodiments, the contour points of the target region can be determined based on the key points of the target object. For example, when the target object is a human portrait, facial key points can be detected first, and the tip of the nose among the facial key points can be used as the center point of the face; then, portrait segmentation is performed to obtain a face mask, and in this face mask, the contour points of the human portrait are searched from the inside out using a gradient search method within a certain radiation range of the center point of the face to obtain the contour points of the top of the head and their coordinates, i.e., the contour points of the target region, such as... Figure 6 As shown, Figure 6 A schematic diagram of cranial contour point detection according to an embodiment of the present disclosure is shown.

[0049] In some embodiments, determining the key points and pose angles of the target object in the image to be processed may include:

[0050] The key points and pose angles of the target object are determined based on the trained target detection model.

[0051] Specifically, the trained object detection model can be used to detect whether a target object exists in the image to be processed, and when the target object is detected, to determine its key points and pose angles. For example, the object detection model can be a face detection model. The image to be processed can be input into the trained face detection model, which can determine whether a face exists in the image. If a face exists, it outputs the key points of the face and their coordinates, as well as the pose angles of the face. It should be understood that the object detection model can be trained on an initial neural network using a first training image labeled with the key points of the target object and the corresponding pose angles as training data to obtain the trained object detection model.

[0052] In some embodiments, determining the contour points of the target region of the target object in the image to be processed may include:

[0053] The target region contour points of the target object are determined based on the trained contour point model.

[0054] Specifically, a well-trained target region contour point model can be used to detect contour points and their location information within a target object. For example, the target region contour point model can be used to detect the cranial contour points of a human figure. The image to be processed can be input into the target region contour point model, which will then output the cranial contour points and their coordinates in the image.

[0055] In some embodiments, method 300 further includes:

[0056] The initial model is trained based on the target region contour point training data to obtain the trained target region contour point model; wherein, the target region contour point training data includes semantic annotation information used to indicate the location.

[0057] The target region contour point training data can include at least one training sample, each labeled with a target region contour point and corresponding annotation information indicating the location of that contour point. Specifically, the training sample can be a training image including a human face, where key points and corresponding annotation information are marked on the face; for example, key point A and the annotation "temple" are used to mark the location of the temple. In this way, during model training, the semantics of the annotation information can be combined to further improve the accuracy of the target region contour point detection model. It is evident that compared to... Figure 6The target region contour point detection described in this disclosure, using a target region contour point model according to the embodiments of this disclosure, not only shortens the processing steps but also improves performance in various scenarios such as side profiles by incorporating semantic information. It also reduces computational resource overhead, significantly minimizing performance loss, enabling real-time processing on mobile devices and improving image processing efficiency. This addresses the problems in existing technologies, such as the poor stability of cranial contour points, which leads to unstable final deformation effects and an inability to adapt well to the head's appearance at different angles, especially in large-angle side profiles where image contour processing is poor and significant deviations easily occur in the range and direction of deformation.

[0058] It should be understood that a trained detection model and a contour point model can be two parts of one model or two separate models; there is no restriction on this.

[0059] In step S330, the center reference point of the target object is determined based on the key points and attitude angles.

[0060] The center reference point indicates the center position of the target object in its current pose and is associated with the pose of the target object. Compared with traditional contour processing methods that use a fixed type of center point, such as using the tip of the nose as the center point for all face images, the image processing method of this disclosure takes into account the influence of the target object's pose on the center point, thus improving the accuracy of the center point and improving the effect and reliability of contour processing.

[0061] In some embodiments, determining the center reference point of the target object based on the key points and attitude angles includes:

[0062] An initial point is obtained by interpolation based on the first preset key point among the key points;

[0063] Candidate points are determined from the initial point based on the yaw angle in the attitude angles;

[0064] The center reference point is obtained by adjusting the position of the candidate point based on the pitch angle in the attitude angle.

[0065] The first preset keypoints can include points whose positions are less affected by other factors such as attitude angles, such as the left temple keypoint, right temple keypoint, and center keypoint between the eyes in facial keypoints. The first preset keypoints can also include the left eye vertex keypoint and the eye vertex keypoint. The first preset keypoints can be located or approximately located on a straight line, so interpolation based on the first preset keypoints yields an initial point located on a straight line. Then, combining the yaw angle from the attitude angles, a candidate point centered on the initial point is determined, and the position of this candidate point in the vertical direction is adjusted based on the pitch angle to obtain the center reference point used for contour processing. It is evident that this center reference point is related to the attitude angles of the target object; even for the same target object in different attitudes, the center reference point is different. Therefore, the center reference point determined based on the attitude angles of the target object can improve the effect and reliability of contour processing.

[0066] In some embodiments, interpolation is performed based on the position of a first preset key point among the key points to obtain an initial point, including:

[0067] Linear interpolation is performed between the first preset key points to obtain the interpolation point until the adjacent spacing between the initial points reaches the interpolation spacing; wherein, the initial point includes the interpolation point and the first preset key point, and the interpolation spacing includes the ratio of the sum of the adjacent spacing of the initial point to the first number of contour points of the target area.

[0068] Specifically, see Figure 7 , Figure 7A schematic diagram of image processing according to an embodiment of the present disclosure is shown. For processing the cranial contour in a face image, linear interpolation can be performed based on the keypoints P1 (left temple), P2 (left eye vertex), P3 (center between the two eyes), P4 (right eye vertex), and P5 (right temple) to obtain m initial points 'center' along the direction from P1 to P5. The linear interpolation can be based on the average of the coordinates of two points to obtain the coordinates of the interpolation point between them. For example, interpolation is performed between P1 (left temple) and P2 (left eye vertex) to obtain interpolation point c1. The coordinates (x_c1, y_c1) of interpolation point c1 can include the average of the coordinates (x_p1, y_p1) of P1 (left temple) and (x_p2, y_p2) of P2 (left eye vertex). Then, interpolation is performed between the keypoint P1 on the left temple and the interpolation point c1 to obtain interpolation point c2; interpolation is performed between interpolation point c1 and the keypoint P2 at the apex of the left eye to obtain interpolation point c3. Similarly, interpolation can be performed between two points: the keypoint P1 on the left temple, the keypoint P2 at the apex of the left eye, the central keypoint P3 between the two eyes, the keypoint P4 at the apex of the right eye, and the keypoint P5 on the right temple, to obtain interpolation points c4-c12, until the adjacent distance between the initial points reaches the interpolation interval. The initial points may include the first preset keypoints P1-P5 and the interpolation points c1-c12, and the interpolation interval may include the ratio B1 = ∑d / n1 of the sum of the distances ∑d of all adjacent initial points to the first number n1 of the target region contour points. It should be understood that the above examples are for illustrative purposes only and are not intended to limit the number and location of the first preset key points. The first preset key points may include more or fewer. Nor are they intended to limit the number of initial points. The initial points may include more or fewer, and no limitation is made here.

[0069] In some embodiments, determining candidate points from the initial point based on the yaw angle in the attitude angles includes:

[0070] The yaw coefficient is determined based on the yaw angle;

[0071] The candidate points are determined from the initial points based on the yaw coefficient.

[0072] The yaw angle can range from -90° to 90°. Specifically, the ratio B2 = (yaw - (-90°)) / 180, which is the difference between the yaw angle and the first preset value (e.g., -90°), is the yaw coefficient. B2*m (where m is the number of initial points) is rounded down (i.e., the decimal part is discarded) to obtain floor(B2*m) (floor is the rounding function), which represents the candidate point position located at the center among the initial points. Based on this candidate point position, the coordinates of the candidate point can be obtained. The initial point positions can be recorded as the 1st, 2nd, ..., mth points along the direction from the left temple keypoint P1 to the right temple keypoint P5. The position of the candidate point is the floor(B2*m)th initial point. For example, if m is 20 and yaw is 80, then the yaw coefficient B2 = 0.944, B2*m = 18.889. Rounding down B2*m yields the 18th initial candidate point. Similarly, if m is 20 and yaw is -50, then the yaw coefficient B2 = 0.222, B2*m = 4.444. Rounding down B2*m yields the 4th initial candidate point. It is evident that the position of the center reference point differs depending on the attitude angle. Compared to traditional methods that do not consider the influence of the target object's attitude and use key points of the same type as the center reference point, the method in this embodiment combines attitude angles to determine the center reference point, thereby improving the accuracy and reliability of subsequent contour processing.

[0073] In some embodiments, adjusting the position of the candidate point based on the pitch angle in the attitude angle to obtain the center reference point includes:

[0074] The pitch coefficient is determined based on the pitch angle;

[0075] The displacement distance and displacement reference direction are determined based on the second preset key point among the key points;

[0076] The center reference point is determined based on the candidate point, the pitch coefficient, the displacement distance, and the displacement reference direction.

[0077] Specifically, the position of the candidate point in the vertical direction can be adjusted based on the pitch angle. The displacement distance and displacement reference direction can be determined based on a second preset key point. This second preset key point may include a key point located on the centerline of the target object, for example... Figure 7As shown, the facial key points include the central key point P3 between the eyes (e.g., the central key point P3 between the eyes can be the root of the nose key point), the first nasal bridge key point P6, and the second nasal bridge key point P7 located on the bridge of the nose (the second nasal bridge key point P7 can be located below the first nasal bridge key point P6; for example, the first nasal bridge key point P6 can be the center point between the root of the nose key point and the tip of the nose key point, and the second nasal bridge key point P7 can be the tip of the nose key point). Based on this, the displacement reference direction can be determined as P7-P6, that is, from the first nasal bridge key point P6 to the second nasal bridge key point P7; the displacement distance h can be the average of (d1+d2) / 2 of the first distance d1 from the central key point P3 between the eyes to the first nasal bridge key point P6 and the second distance d2 from the central key point P3 between the eyes to the second nasal bridge key point P7. The pitch angle can be within the range of [-45°, 45°]. The pitch coefficient can be obtained by calculating the difference between the pitch angle and a third preset value (e.g., -45°) and a fourth preset value (e.g., 90°), using the ratio B3 = (pitch - (-45°)) / 90. Then, the displacement of the candidate point in the vertical direction can be obtained by multiplying the pitch coefficient, the displacement distance h, and the displacement reference direction. The sum of this displacement and the coordinates of the candidate point yields the coordinates of the center reference point C. When the value of B3 is greater than 0, the candidate point moves a displacement distance h along the first nose bridge key point P6 towards the second nose bridge key point P7 to obtain the center reference C; when the value of B3 is less than 0, the candidate point moves a displacement distance h along the second nose bridge key point P7 towards the first nose bridge key point P6 to obtain the center reference C.

[0078] In step S340, the position of the contour points of the target region is adjusted based on the central reference point to obtain the target image.

[0079] Among them, see Figure 7 When the target object is a face, the target region contour point 710 can be the cranial vault contour point. The target region contour point 710 can be adjusted based on the central reference point C to obtain the user's desired target image. Specifically, the adjustment range of the target region contour point 710 can be determined first based on the central reference point C. Then, based on the deformation intensity, liquefaction deformation is applied to the target region contour point 710 within the adjustment range to adjust its position and obtain the target image.

[0080] In some embodiments, adjusting the position of the contour points of the target region based on the central reference point to obtain a target image includes:

[0081] Determine the deformation intensity of the contour points of the target region;

[0082] The adjustment range of the target region contour points is determined based on the central reference point and the target region contour points.

[0083] The target region contour points are subjected to liquefaction deformation processing based on the deformation intensity within the adjustment range to obtain the target image.

[0084] Deformation intensity refers to the degree of change in the contour points of the target area. Higher deformation intensity indicates greater deformation of the contour points, while lower deformation intensity indicates less deformation. Specifically, the deformation intensity (intensity) can be set by the user or use a preset value (or default value), where intensity ∈ [0.0, 1.0]. An adjustment range can be determined on both sides of the contour points of the target area based on a central reference point, and then the contour points of the target area can be adjusted within this range based on the deformation intensity.

[0085] In some embodiments, determining the adjustment range of the target region contour points based on the center reference point and the target region contour points includes:

[0086] Using the central reference point as the center of the sector, along the straight line where the central reference point and the target region contour point are located, inner expansion points and outer expansion points are obtained on both sides of the target region contour point.

[0087] A mesh is constructed based on the target region contour points, the inner extension points, and the outer extension points, with the mesh located on both sides of the target region contour points;

[0088] The area of ​​the grid is defined as the adjustment range.

[0089] Specifically, after determining the center reference point of the cranial contour, the mesh required for deformation can be constructed, such as... Figure 7 As shown, for the target object in the image to be processed, the inner extension point 720 can be determined at a distance h1 from the target region contour point 710, based on the central reference point C as the center of the sector, and the outer extension point 730 can be determined at a distance h2 from the target region contour point 710, along the line connecting the central reference point C and the target region contour point 710. Then, connecting the target region contour point 710, the extended inner extension point 720, and the outer extension point 730 yields the grid located at the target region contour point 710, forming the adjustment range. It should be understood that the distances h1 and h2 can be determined by the user or are preset values, and they can be the same or different; no restrictions are placed here.

[0090] After determining the adjustment range, liquefaction deformation processing can be used to adjust the contour points of the target area. Liquefaction deformation includes local translation, local magnification, and local reduction. Specifically, the position of the contour points of the target area can be adjusted based on local translation. For the contour point X of the target area, its Euclidean distance ||XC|| to the central reference point C can be calculated, and based on a preset value r... maxThe translation coefficient *infect* = 1 - (||XC|| / r) max ) 2 The translation reference direction can be defined as the direction from the central reference point C to the outward expansion point M. Then, the displacement can be obtained as the product S of the Euclidean distance ||XC|| between the target region contour point X and the central reference point C, the translation coefficient *infect*, and the translation reference direction. The coordinates of the adjusted target region contour point U can be obtained based on the difference between the coordinates of the target region contour point X and this product S.

[0091] As can be seen, the image processing method according to the embodiments of this disclosure can not only adjust the contour of the target object by combining the pose angle of the target object, but also improve the reliability of the reference points at different angles, thereby improving the effect and reliability of contour processing. Furthermore, it directly outputs contour key points based on a trained contour key point model, eliminating the need to determine contour key points based on a combination of target key point detection, target segmentation, and target edge detection, thus improving the efficiency of contour processing and reducing processing costs.

[0092] It should be noted that the method of this disclosure embodiment can be executed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario, where multiple devices cooperate to complete the task. In such a distributed scenario, one of these devices may execute only one or more steps of the method of this disclosure embodiment, and the multiple devices will interact with each other to complete the method described.

[0093] It should be noted that the above description describes some embodiments of this disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the above embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0094] Based on the same technical concept, corresponding to any of the above embodiments, this disclosure also provides an image processing apparatus, see [link to relevant documentation]. Figure 8 The image processing apparatus includes:

[0095] The acquisition module is used to acquire the image to be processed;

[0096] The detection module is used to determine the key points, pose angles, and contour points of the target area of ​​the target object in the image to be processed.

[0097] The reference point module is used to determine the center reference point of the target object based on the key points and attitude angles;

[0098] The contour adjustment module is used to adjust the position of the contour points of the target region based on the central reference point to obtain the target image.

[0099] For ease of description, the above apparatus is described in terms of its functions, divided into various modules. Of course, in implementing this disclosure, the functions of each module can be implemented in one or more software and / or hardware.

[0100] The apparatus of the above embodiments is used to implement the corresponding image processing method in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0101] Based on the same technical concept, corresponding to the methods of any of the above embodiments, this disclosure also provides a non-transitory computer-readable storage medium that stores computer instructions for causing the computer to perform the image processing method as described in any of the above embodiments.

[0102] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0103] The computer instructions stored in the storage medium of the above embodiments are used to cause the computer to execute the image processing method as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0104] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of this disclosure (including the claims) is limited to these examples; within the framework of this disclosure, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the embodiments of this disclosure as described above, which are not provided in detail for the sake of brevity.

[0105] Additionally, to simplify the description and discussion, and to avoid obscuring the embodiments of this disclosure, the provided drawings may or may not show well-known power / ground connections to integrated circuit (IC) chips and other components. Furthermore, the apparatus may be shown in block diagram form to avoid obscuring the embodiments of this disclosure, and this also takes into account the fact that the details of implementation of these block diagram apparatuses are highly dependent on the platform on which the embodiments of this disclosure will be implemented (i.e., these details should be fully understood by those skilled in the art). While specific details (e.g., circuitry) have been set forth to describe exemplary embodiments of this disclosure, it will be apparent to those skilled in the art that the embodiments of this disclosure may be implemented without these specific details or with variations thereof. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0106] Although this disclosure has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed.

[0107] This disclosure is intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. An image processing method, the method comprising: Obtain the image to be processed; Determine the key points, pose angles, and contour points of the target region of the target object in the image to be processed; Determining the center reference point of the target object based on the key points and attitude angles includes: performing interpolation processing based on a first preset key point among the key points to obtain an initial point; determining a candidate point from the initial point based on the yaw angle among the attitude angles; adjusting the position of the candidate point based on the pitch angle among the attitude angles to obtain the center reference point; the first preset key point is located or approximately located on a straight line; The target image is obtained by adjusting the position of the contour points of the target region based on the central reference point, including: Determine the deformation intensity of the contour points of the target region; Determining the adjustment range of the target region contour point based on the central reference point and the target region contour point includes: taking the central reference point as the center of a sector, extending along the straight line where the central reference point and the target region contour point are located, and obtaining inner extension points and outer extension points on both sides of the target region contour point; constructing a grid based on the target region contour point, the inner extension points, and the outer extension points, wherein the grid is located on both sides of the target region contour point; and determining the area of ​​the grid as the adjustment range. The target region contour points are subjected to liquefaction deformation processing based on the deformation intensity within the adjustment range to obtain the target image, including: calculating the Euclidean distance between the target region contour points and the central reference point; The translation coefficient is obtained based on the Euclidean distance and the preset value; The coordinate displacement is obtained by multiplying the Euclidean distance, the translation coefficient, and the translation reference direction; wherein, the translation reference direction is the direction from the central reference point to the outer expansion point; Based on the difference between the coordinates of the contour points of the target region and the coordinate displacement, the position of the contour points of the target region is adjusted to obtain the target image.

2. The method according to claim 1, wherein, An initial point is obtained by interpolation based on the position of the first preset key point among the key points, including: Linear interpolation is performed between the first preset key points to obtain interpolation points until the adjacent spacing between the initial points reaches the interpolation spacing; wherein, the initial points include the interpolation points and the first preset key points, and the interpolation spacing is the ratio of the sum of the adjacent spacings of the initial points to the first number of contour points of the target area.

3. The method according to claim 2, determining candidate points from the initial points based on the yaw angle in the attitude angles, includes: The yaw coefficient is determined based on the yaw angle; The candidate points are determined from the initial points based on the yaw coefficient.

4. The method according to claim 2, wherein, The center reference point is obtained by adjusting the position of the candidate point based on the pitch angle in the attitude angle, including: The pitch coefficient is determined based on the pitch angle; The displacement distance and displacement reference direction are determined based on the second preset key point among the key points; The center reference point is determined based on the candidate point, the pitch coefficient, the displacement distance, and the displacement reference direction.

5. An image processing apparatus, comprising: The acquisition module is used to acquire the image to be processed; The detection module is used to determine the key points, pose angles, and contour points of the target area of ​​the target object in the image to be processed. The reference point module is used to determine the center reference point of the target object based on the key points and attitude angles, including: performing interpolation processing based on a first preset key point among the key points to obtain an initial point; determining a candidate point from the initial point based on the yaw angle among the attitude angles; adjusting the position of the candidate point based on the pitch angle among the attitude angles to obtain the center reference point; the first preset key point is located or approximately located on a straight line; The contour adjustment module is used to adjust the position of the contour points of the target region based on the central reference point to obtain the target image, including: Determine the deformation intensity of the contour points of the target region; Determining the adjustment range of the target region contour point based on the central reference point and the target region contour point includes: taking the central reference point as the center of a sector, extending along the straight line where the central reference point and the target region contour point are located, and obtaining inner extension points and outer extension points on both sides of the target region contour point; constructing a grid based on the target region contour point, the inner extension points, and the outer extension points, wherein the grid is located on both sides of the target region contour point; and determining the area of ​​the grid as the adjustment range. The target region contour points are subjected to liquefaction deformation processing based on the deformation intensity within the adjustment range to obtain the target image, including: calculating the Euclidean distance between the target region contour points and the central reference point; obtaining a translation coefficient based on the Euclidean distance and a preset value; obtaining a coordinate displacement based on the product of the Euclidean distance, the translation coefficient, and the translation reference direction; wherein the translation reference direction is the direction from the central reference point to the outward expansion point; and adjusting the position of the target region contour points based on the difference between the coordinates of the target region contour points and the coordinate displacement to obtain the target image.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the method as claimed in any one of claims 1 to 4.

7. A non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the method of any one of claims 1 to 4.

Images