Image local processing method, device, electronic device and storage medium

Abstract

The present application provides a method, device, electronic device and storage medium for local image processing, including: determining the target modification part and the occluded part of the target model in the image, and determining the depth value of each pixel point of the target modification part; determining the inclusion model of the occluded part, and determining that the depth value of each pixel point of the target modification part is within the depth value interval inside the inclusion model; determining that the area composed of pixel points whose depth values are outside the depth value interval is the area to be modified, and performing a modification operation on the area to be modified. The present application determines the inclusion model of the occluded part, determines the depth value interval of the inclusion model, and then determines whether the depth value of the target modification part is within the depth value interval, accurately distinguishing the area to be modified, thereby improving the accuracy and precision of the target modification part confirmation, improving the overall cropping freedom, and thereby quickly solving the problem of model penetration and improving the expressiveness of the picture.

Description

Technical Field

[0001] The present application relates to the field of model making technology, and in particular to a method, device, electronic device and storage medium for local image processing. Background Art

[0002] In the current field of animation and game production, in order to make objects such as people and objects more realistic, the use of three-dimensional (3D) models is increasing in the field, so as to create 3D animations or 3D games. When creating these objects, they are generally equipped with a variety of accessory models, such as different clothing, jewelry, and decorations.

[0003] Since the 3D model and these component models are usually established separately, the two models are prone to problems with interplay when assembled. For example, when a character model is wearing an accessory such as a hat, in order to make the position of the hat more harmonious, in most cases it will intersect with the character model and cause interplay. Existing solutions generally use plane cutting to modify the interplay parts. However, this modification method can only perform simple overall leveling modifications based on a plane, which limits the style of the accessory and reduces the freedom of cutting. As the styles of accessories become more diverse and complex, this method of cutting with low freedom of cutting has made it difficult to accurately determine the target modification parts, resulting in more manual modification. This increases the workload of engineers, reduces the efficiency of model making, and reduces the accuracy of modifications. Summary of the Invention

[0004] In view of this, the present application proposes a local image processing method, device, electronic device and storage medium to quickly and accurately determine the position of the model to be modified, improve the cropping freedom, and thus quickly solve the problem of model penetration in the image.

[0005] Based on the above objectives, this application provides a method for local image processing, including:

[0006] Determine the target modification part and the blocked part of the target model in the image, and determine the depth value of each pixel point of the target modification part;

[0007] Determine an inclusion model of the occluded portion, and determine a depth value interval within which a depth value of each pixel point of the target modification portion is located within the inclusion model;

[0008] Determine an area composed of pixels whose depth values are outside the depth value interval as an area to be modified, and perform a modification operation on the area to be modified.

[0009] Based on the same concept, the present application also provides an image local processing device, including:

[0010] A determination module, configured to determine a target modification portion and an occluded portion of a target model in an image, and determine a depth value of each pixel point of the target modification portion;

[0011] a calculation module, configured to determine an inclusion model of the occluded portion, and determine a depth value interval within which the depth value of each pixel point of the target modification portion is located within the inclusion model;

[0012] The modification module is configured to determine an area composed of pixels whose depth values are outside the depth value interval as an area to be modified, and perform a modification operation on the area to be modified.

[0013] Based on the same concept, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the above methods when executing the program.

[0014] Based on the same concept, the present application also provides a computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable the computer to implement any of the methods described above.

[0015] From the above, it can be seen that the present application provides a method, device, electronic device and storage medium for local image processing, including: determining the target modification part and the occluded part of the target model in the image, and determining the depth value of each pixel point of the target modification part; determining the inclusion model of the occluded part, and determining that the depth value of each pixel point of the target modification part is within the depth value interval inside the inclusion model; determining that the area composed of pixel points whose depth values are outside the depth value interval is the area to be modified, and performing a modification operation on the area to be modified. The present application determines the inclusion model of the occluded part, determines the depth value interval of the inclusion model, and then determines whether the depth value of the target modification part is within the depth value interval, accurately distinguishes the area to be modified, improves the accuracy and precision of the target modification part confirmation, improves the overall cropping freedom, and thereby quickly solves the problem of model penetration and improves the expressiveness of the picture. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0017] Figure 1 A flowchart of a local image processing method proposed in an embodiment of the present application;

[0018] Figure 2 A partial schematic diagram of a target model proposed in an embodiment of the present application;

[0019] Figure 3 A schematic diagram of a model of an occluded area and a corresponding inclusion model proposed in an embodiment of the present application;

[0020] Figure 4a This is a schematic diagram of the initial state of the linked list creation process using the linked list method on a pixel-by-pixel basis proposed in an embodiment of the present application;

[0021] Figure 4b This is a schematic diagram of the first state of the linked list creation process using the linked list method on a pixel-by-pixel basis according to an embodiment of the present application;

[0022] Figure 4c This is a schematic diagram of the second state of the linked list creation process using the linked list method on a pixel-by-pixel basis according to an embodiment of the present application;

[0023] Figure 4d This is a schematic diagram of the third state of the linked list creation process using the linked list method on a pixel-by-pixel basis proposed in an embodiment of the present application;

[0024] Figure 5 A schematic diagram illustrating the principle of a method for determining a depth value interval proposed in an embodiment of the present application;

[0025] Figure 6 A schematic structural diagram of a local image processing device proposed in an embodiment of the present application;

[0026] Figure 7 This is a schematic diagram of the electronic device structure proposed in an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solutions and advantages of this specification more clear, this specification is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0028] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should have the usual meanings understood by people with ordinary skills in the field to which this application belongs. The "first", "second" and similar words used in the embodiments of the present application do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements, objects or method steps that appear before the word cover the elements, objects or method steps listed after the word and their equivalents, without excluding other elements, objects or method steps. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0029] As mentioned in the background section, 3D models are increasingly being used in animation and game production, and the images of these models are becoming increasingly diverse. For example, in the current gaming market, there are a growing number of options for players to customize their characters, from hairstyles and fashion to accessories. With the introduction of these diverse models, interplay between models inevitably arises. For example, when a character wears a hat, the interplay between the character's hairstyle and the hat becomes a pressing issue. Current games generally avoid matching hats or only support a very limited selection of hat styles, limiting the flexibility and effectiveness of culling. Some currently available games have implemented simple clipping solutions to address the issue of clipping between accessories such as hats. This existing approach involves defining a clipping plane for the hat and clipping hair and accessories located on one side of this plane. This specific implementation involves passing the plane definition into the vertex shader. During the vertex shading phase, the positional relationship between pixels and the plane is determined, and the alpha value of the vertex color located above the plane is set to 0.0. When drawing hair using this method, since the hair on the upper side of the plane is completely transparent, it will not appear on the screen after color blending, achieving the goal of removing hair outside the hat. However, this method of through-mold clipping can only be performed based on the plane and can only handle the situation where the hat brim is flush, resulting in a relatively simple form for accessories such as hats. At the same time, this "one-size-fits-all" approach cannot guarantee the accuracy and precision of clipping or modification.

[0030] In view of the above actual situation, the embodiment of the present application proposes a local image processing solution, which determines the inclusion model of the occluded part, uses a linked list method to determine the depth value range of the inclusion model pixel by pixel, and then judges whether the depth value of the target modification part is within the depth value range, accurately distinguishes the area to be modified, improves the accuracy and precision of the confirmation of the target modification part, improves the overall cropping freedom, and quickly solves the problem of model penetration, thereby improving the expressiveness of the picture.

[0031] like Figure 1 FIG. 1 is a flow chart of a local image processing method proposed in this application, which specifically includes:

[0032] Step 101: Determine a target modification portion and an occluded portion of a target model in an image, and determine a depth value of each pixel point of the target modification portion.

[0033] In this step, the image is a real-time or fixed picture scene that the user sees when using applications such as games and animation playback. For example, when the user is playing a game, the user changes the clothes of a virtual object in a specific scene or an unspecified scene, etc. The target model is a character or object model used for animation production or game production. These models are usually provided with some accessories or pendants, so that there may be a problem of penetration between the model and these accessories. Therefore, the occluded parts of the target model are these accessory models or parts that will cause the problem of penetration, and the target modification part is the part of the target model where these accessory models or parts are worn or added. For example: for a 3D character model wearing a hat, the hat is the occluded part, and the hair or head of the corresponding 3D character model is the target modification part; or for a 3D tree model with decorative lights (such as a Christmas tree with a five-pointed star light, etc.), the decorative lights are the occluded parts, and the part of the corresponding 3D tree model where the decorative lights are installed is the target modification part (such as the pointed top of the Christmas tree model, etc.), etc. In the target model, the relative position relationship between the occluded part and the target modification part is generally set in advance, such as Figure 2 As shown, taking the character model wearing a hat as an example, after the character is created, the position where the hat is worn and the height of the hat have been set. Figure 2 The lotus leaf hat shown clearly shows parts such as hair protruding from the model. The lotus leaf hat is the occluded part, and the hair or head is the target part to be modified. For example, a character model wearing a hat is the target model, and the target part to be modified is the head part or hair part of the character model that is designed to wear the hat. The occluded part is the hat model worn on the head part or hair part. The hat model has a fixed shape and is attached to the head part or hair part of the character model.

[0034] After that, the depth value is the distance from each pixel in the depth image to the camera or pixel position. A depth image, also known as a range image, refers to an image that uses the distance (depth) from the image collector (camera or viewpoint position, etc.) to each pixel in the scene as a pixel value, which directly reflects the geometric shape of the visible surface of the scene. In the image frame provided by the depth data stream, each pixel represents the distance (usually in millimeters) from the object at each specific point (x, y) coordinate in the field of view of the depth sensor set at the camera position or viewpoint position to the object closest to the camera plane. In 3D computer graphics and computer vision, a depth map is an image or image channel that contains information about the distance from the surface of the scene object to the viewpoint, which is used to simulate 3D shapes or reconstruct them. To illustrate, the depth of an image typically ranges from 0 to 1. Depth represents the distance from each point on the image to the camera, lens, or viewpoint. The viewpoint or camera position is the pre-set image channel or shooting position. The depth at the viewpoint is 0, while the depth at the viewpoint's maximum viewing distance is 1. This allows the depth of each point on the image to be determined. This allows the depth of each pixel to be modified in the target area to be determined.

[0035] Step 102: Determine an inclusion model of the occluded portion, and determine a depth value interval within which the depth value of each pixel point of the target modification portion is located within the inclusion model.

[0036] In this step, the inclusion model of the occluded part is the coverage area model or shadow area model of the occluded part, etc. It is a closed geometric model, and the area included in it is the area that cannot be modified. Figure 3 The figure shows a schematic diagram of a shielding part (hat) and its corresponding inclusion in a specific application scenario, where Figure 3 a is the shielding part, i.e., the schematic diagram of the lotus leaf hat model; Figure 3 b is a schematic diagram of the inclusion model of the occluded part; Figure 3Figure c is a schematic diagram of an occlusion area superimposed with an inclusion. The inclusion is typically designed and created by the designer when creating the occlusion area. It is then integrated with the target model along with the occlusion area. The inclusion model is created based on the shape of the occlusion area, reflecting the internal space of the occlusion area and preventing modification. Using the inclusion model, the occlusion range of the occlusion area can be quickly determined, and thus the modification area of the target modification area can be quickly determined. For example, using a hat as an occlusion area, the target modification area is the hair. In a specific embodiment, a layer of the hat model can be copied first. The outer edge of the hat model is then found and extruded along it to create a model piece. The extrusion function is then repeated multiple times, combined with the engineer's input, to adjust the model's points, lines, and surfaces. This allows the inclusion model to have a larger range of physical movement than the character's hair when wearing the hat. Finally, after the inclusion model completely encompasses the hat's range of motion, the inclusion model is closed, making it a completely closed body. Additionally, when duplicating the hat model, you can slightly resize the duplicate by adjusting the PushValue (lowering it by approximately 0.05) to improve the fit of the inclusion with the hat model. In specific application scenarios, you can use the Extrude tool in 3D Max. The Extrude tool is a DCC modeling tool that allows you to extrude a face through an edge or face. This tool is simply for faster creation of the desired model. For example, using the hat model as an example, the extrusion process involves extrude a line from one edge of the hat, creating another line, adjusting the size of this line, and then extruding again, repeating this process until the desired inclusion is achieved. The new extruded surface is typically further from the center of the hat and extends diagonally downward along the hat (i.e., along the hair's range of motion). Finally, after completely enclosing the hair's range of motion, the model is closed. The extruded lines are merged into a single point. In specific application scenarios, approximately 15 to 20 extrusions are performed before the model is closed. Finally, as an actual model, the inclusion model can facilitate users or designers to create or modify the model during the game or animation production process. However, the inclusion model will not be rendered or drawn in the final rendering or drawing step. Therefore, although the inclusion model will be loaded into the game or animation together, it will not be displayed and observed by the user during the final presentation.

[0037] Afterwards, the depth value of each pixel point of the target modification part is determined to be within the depth value interval inside the inclusion model. In some embodiments, the determination can be made using a pixel-by-pixel linked list method. The pixel-by-pixel linked list method is the Per-Pixel Linked Lists method. Through the idea of Per-Pixel Linked Lists, two buffers can be used to record the spatial information of all pixel points of the inclusion model. In a specific embodiment, if Figures 4a to 4d As shown in the figure, since the pixel size of the inclusion model may not be equal to that of the image, that is, the contrast may not be equal, and the coverage of the pixels of the two may not be the same, the triangle element is used as an easily identifiable symbol to record the pixel by pixel using the linked list method. Figure 4a As shown in the figure, it shows the image in the initial state and the fragment buffer and the first node buffer for recording information. Each grid in the image represents a pixel. The first node buffer is a two-dimensional buffer whose size is consistent with the contrast of the image. Each grid in the buffer corresponds to each grid in the image. The fragment buffer is a one-dimensional buffer whose size is an integer multiple of the contrast of the image. Figure 4b As shown, when the first element is rendered, the pixels it mainly occupies are determined (they can be determined by the occupied ratio, etc.), and the corresponding number or index is recorded in the corresponding position of the first node buffer, and the corresponding attributes of the pixel are recorded in the first column of the fragment buffer. For example, the first element corresponds to a pixel, and the index of the pixel is 0. The first column of the fragment buffer records the color, texture, and the number or index of the pixel previously recorded. Figure 4c As shown in , when the second element is rendered, similar steps are performed as for the first element, and the pixels mainly occupied by the second element are arranged and numbered or indexed in sequence. Figure 4d As shown, when the third element is rendered, similar steps are performed as before. When the occupied pixel overlaps with the previous one, since the previous index value has been recorded in the first node buffer, the new index value directly replaces the previous index value, and the fragment buffer can record the index value recorded before the pixel, as shown in the following example. Figure 4d The index value 3 in the fragment buffer and its corresponding column in the fragment buffer. And so on. Finally, the first node buffer records the number or index of the last element at each pixel in the entire image. Then, by backtracking through the fragment buffer, the spatial position relationship of all pixels can be recorded.

[0038] Afterwards, the depth value of each pixel point on the inclusion model can also be obtained using a depth value determination method similar to step 101, and then, based on the spatial position of each point and the corresponding depth value, it can be determined which depth value intervals are located inside the inclusion.

[0039] Step 103 : Determine that the region composed of the pixel points whose depth values are outside the depth value interval is the region to be modified, and perform a modification operation on the region to be modified.

[0040] In this step, after determining the depth value of each pixel to be modified and the depth value range within the inclusion model, the two are compared to determine which pixels have depth values that are not within the depth value range. The area formed or composed of these pixels with depth values that are not within the depth value range is the area to be modified, or the area that causes the penetration of the model. The area to be modified can then be modified by deleting, hiding, setting transparency, not drawing or rendering, etc., so that the modified area is invisible to the user (visually) on the graphical user interface.

[0041] Afterwards, the modified image can be output for processing, which can include displaying the image for users to view or allowing engineers to make further adjustments to the image, add attachments, or perform other tasks. In other words, the modified image can be output for storage, display, use, or further processing. The specific output method for the modified image can be flexibly selected based on different application scenarios and implementation needs.

[0042] For example, for an application scenario where the method of this embodiment is executed on a single device, the modified image can be directly output in a displayed manner on the display component (display, projector, etc.) of the current device, so that the operator of the current device can directly see the content of the modified image from the display component.

[0043] For another example, in an application scenario where the method of this embodiment is executed on a system composed of multiple devices, the modified image can be sent to other preset devices serving as receivers in the system, i.e., synchronization terminals, through any data communication method (wired connection, NFC, Bluetooth, wifi, cellular mobile network, etc.), so that the synchronization terminals can perform subsequent processing on them. Optionally, the synchronization terminal can be a preset server, which is generally set up in the cloud and serves as a data processing and storage center, capable of storing and distributing the modified image; wherein the recipient of the distribution is a terminal device, and the holders or operators of these terminal devices can be the current user, bound engineers, downstream animation, game production engineers, animation, and public databases of game production companies to record the work results of each engineer, etc.

[0044] For another example, in an application scenario where the method of this embodiment is executed on a system composed of multiple devices, the modified image can be sent directly to a preset terminal device through any data communication method, and the terminal device can be one or more of the ones listed in the preceding paragraphs.

[0045] From the above, it can be seen that an image local processing method of an embodiment of the present application includes: determining the target modification part and the occluded part of the target model in the image, and determining the depth value of each pixel point of the target modification part; determining the inclusion model of the occluded part, and determining that the depth value of each pixel point of the target modification part is within the depth value interval inside the inclusion model; determining that the area composed of pixel points whose depth values are outside the depth value interval is the area to be modified, and performing a modification operation on the area to be modified. The present application determines the inclusion model of the occluded part, determines the depth value interval of the inclusion model, and then determines whether the depth value of the target modification part is within the depth value interval, accurately distinguishes the area to be modified, improves the accuracy and precision of the target modification part confirmation, improves the overall cropping freedom, and thereby quickly solves the problem of model penetration and improves the expressiveness of the picture.

[0046] It should be noted that the method of the embodiment of the present application can be performed by a single device, such as a computer or server. The method of the embodiment of the present application can also be applied in a distributed scenario and completed by multiple devices working together. In the case of such a distributed scenario, one of the multiple devices may only perform one or more steps of the method of the embodiment of the present application, and the multiple devices will interact with each other to complete the method described.

[0047] It should be noted that the above description is of specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0048] In an optional exemplary embodiment, determining that the depth values of each pixel point of the target modification part are located in a depth value interval inside the inclusion model includes: using a linked list method pixel by pixel to determine the spatial range of the inclusion model in the image and the depth values of each surface pixel point of the inclusion model; determining a first correspondence between each of the surface pixel points based on the spatial range, and generating the depth value interval based on the depth values of the corresponding surface pixel points.

[0049] In this embodiment, based on the pixel-by-pixel linked list method described in the previous embodiment, the depth value of each surface pixel can be recorded using its corresponding buffer. For example, each column in the fragment buffer stores the depth value of its corresponding surface pixel. Thus, two buffers using the pixel-by-pixel linked list method can be used to store the spatial extent of the inclusion model in the image and the depth value of each surface pixel. The spatial extent of the inclusion model can be determined by the combination of pixels located on its surface. Subsequently, based on the concept of depth value and the spatial extent of the inclusion model in the image, the surface pixels along each line of sight can be determined, i.e., a first correspondence. For example, within the concept of an image, there is a viewpoint, which simulates the position from which the image is observed, such as the position of a human eye or a camera lens. Light received at this position generates the image, while light emitted from this point corresponds to the entire image. Each line of sight emanating from this viewpoint can correspond to a pixel in the image. Thus, a line of sight can pass through multiple pixels on the inclusion model, thereby forming a first correspondence between the pixels along these lines of sight. The depth values of these pixel points are recorded, and the depth value ranges of each depth within the inclusion can be finally determined based on the depth values of these points.

[0050] In an optional exemplary embodiment, the method of using a linked list method pixel by pixel to determine the spatial range of the inclusion model in the image and the depth value of each surface pixel of the inclusion model includes: establishing a fragment buffer and a first node buffer, and rendering the inclusion model; during the rendering process, determining a second correspondence between each image pixel of the image and the surface pixel, using the fragment buffer to record the depth value of each surface pixel and the previous surface pixel of the surface pixel determined according to the second correspondence, and using the first node buffer to record the position of each image pixel and the last surface pixel corresponding to each image pixel determined according to the second correspondence; determining the spatial range of the inclusion model in the image and the depth value of each surface pixel of the inclusion model through the fragment buffer and the first node buffer.

[0051] In this embodiment, a method is used to determine the spatial range and the depth value of each surface pixel point by using a linked list method pixel by pixel. Figure 4a As shown in the figure, it shows the image in the initial state and the fragment buffer and the first node buffer for recording information. Each grid in the image represents a pixel. The first node buffer is a two-dimensional buffer whose size is consistent with the contrast of the image. Each grid in the buffer corresponds to each grid in the image. The fragment buffer is a one-dimensional buffer whose size is an integer multiple of the contrast of the image. Figure 4b As shown, when the first element is rendered, the pixels it mainly occupies are determined (this can be determined by using the occupation ratio, etc.), and the corresponding numbers or indexes are recorded in the corresponding positions of the first node buffer, and the corresponding attributes of the pixel are recorded in the first column of the fragment buffer. For example, the first element corresponds to a pixel, and the index of the pixel is 0. The first column of the fragment buffer records the color, texture, and the number or index previously recorded for the pixel corresponding to pixel 0. Since the fragment buffer is a one-dimensional buffer, it generally stores or records a series of arrays, such as array A[0,1,2,3…N], which is an array of N elements, where the sequence numbers 0, 1, 2, 3…are the indices of the pixels corresponding to each number. As Figure 4c As shown in , when the second element is rendered, similar steps are performed as for the first element, and the pixels mainly occupied by the second element are arranged and numbered or indexed in sequence. Figure 4d As shown, when the third element is rendered, similar steps are performed as before. When the occupied pixel overlaps with the previous one, since the previous index value has been recorded in the first node buffer, the new index value directly replaces the previous index value, and the fragment buffer can record the index value recorded before the pixel, as shown in the following example. Figure 4d The index value 3 in the fragment buffer corresponds to the corresponding column in the fragment buffer. And so on. Ultimately, the first node buffer records the number or index of the last element at each pixel in the entire image. By backtracking through the fragment buffer, the spatial position relationship of all pixels can be recorded. Each column in the fragment buffer also records the depth value of that point, thereby determining the depth value of each surface pixel of the inclusion model.

[0052] In a specific embodiment, an unordered access view (UAV) can be created for the fragment buffer and the head node buffer, and write operations can be performed on the two UAVs in the pixel shader. In order to record all pixels in an orderly manner, it is necessary to use the counter feature of the UAV and define the fragment buffer as a counter buffer. When the UAV of the fragment buffer is executed once, the built-in counter is increased by 1, and the return value of the counter is used as the index value of the pixel point in the fragment buffer, such as Figures 4a to 4d 0 to 4 in . Then use the atomic swap function to write the pixel index value to the first node buffer, write the new index value, and replace the previous pixel index value. The old pixel index value is written to the fragment buffer, and then all pixels at this screen coordinate can be traced back through the index value of the last pixel recorded in the node buffer.

[0053] In an optional exemplary embodiment, determining the second correspondence between each image pixel point of the image and the surface pixel point includes: when rendering, when the range occupied by a surface pixel point on an image pixel point exceeds a set threshold, determining that the surface pixel point has the second correspondence with the image pixel point.

[0054] In this embodiment, since the image pixels and the surface pixels may not be consistent, the image pixels occupied by each surface pixel can be determined by taking a proportion exceeding a certain threshold, for example Figures 4b to 4d For example, a triangle occupies an image pixel after a certain threshold (e.g., 50%), and is determined to occupy the image pixel. This ultimately determines a second correspondence between each image pixel and each surface pixel.

[0055] In an optional exemplary embodiment, the establishment of the fragment buffer and the first node buffer also includes: determining the number of target models in the image; in response to the number of target models exceeding 1, generating the first node buffer by performing video memory sharing processing on the image, so as to make the first node buffer.

[0056] In this embodiment, since the target model may be a human, animal, or plant model, its image occupancy is relatively small. Furthermore, in specific scenarios such as games, users may move the target model, a virtual object, to a public location with a large number of virtual objects. This may result in multiple target objects in a single image, each of which may have modified and occluded parts. Furthermore, in specific application scenarios, if each inclusion model corresponding to each target model requires a fragment buffer and a head-node buffer to record spatial information, the video memory overhead will be very high when there are many inclusion models in the scene. To address this issue, each target model can be numbered and the pixels belonging to different inclusion models can be identified by the number. This allows a single head-node buffer to record all inclusion information for the entire screen. This utilizes a shared video memory processing approach, establishing a single buffer in the video memory to handle multiple target models. Multiple target models within this buffer are then distinguished by numbering or indexing, significantly reducing video memory usage.

[0057] In a specific embodiment, the length and width of the screen resolution are w and h respectively, and the memory size of the fragment buffer is M n , the memory size of the first node buffer is M h , then M n ≥(2×w×h)×8,M h=(w×h)×4. If the number of visible inclusions on the screen is n, the total memory overhead of the non-shared fragment buffer and the first node buffer is n×(M n +M h ), and the total overhead of shared video memory is n×M n +M h . It can be seen that it has significantly reduced some video memory.

[0058] In an optional exemplary embodiment, the response to the number of target models exceeding one further includes: generating the fragment buffer by numbering the inclusion model corresponding to each target model, so that the fragment buffer corresponds to the inclusion models of all the target models and records the number of each inclusion model; projecting all the inclusion models into the image, recording the number of projected pixels, and adjusting the video memory size used by the fragment buffer according to the number of projected pixels.

[0059] In this embodiment, according to the previous embodiment, the number of each inclusion model can be matched with the surface pixel of the inclusion model, and the value of the number can be assigned to each surface pixel and recorded in the fragment buffer. Then, the inclusion model corresponding to each surface pixel can be determined through the fragment buffer. Then, according to the previous embodiment, record T n is the total memory overhead of the fragment buffer, and it can be found that T n It has not decreased, T n =n×M n . Considering that the inclusion may only take up a small space on the screen, if we directly use n×M n is the video memory size of the fragment buffer. Although it is safe, it will cause a lot of waste. Therefore, we can estimate the total actual screen size of the inclusion model by projecting all the inclusion models once, which is recorded as W (number of pixels 2), and then we have: T n =2×ceil(W÷(w×h))×(w×h)×8, where ceil is the rounding function. It can be seen that when W is an integer multiple of w×h, T n = 2 × W × 8. This method can be used to significantly reduce the use of video memory again.

[0060] Then, in some embodiments, according to the requirements of the specific application scenario for image accuracy, w and h are taken as half of the length and width of the screen, which can further save half of the video memory overhead.

[0061] In an optional exemplary embodiment, generating the depth value interval based on the depth value of the corresponding surface pixel point includes: determining the viewpoint position of the image, and the line of sight emitted from the viewpoint position corresponds to the image pixel point of the image; in response to the viewpoint position being located inside the inclusion model, using the depth value of the viewpoint position as the first interval node of the depth value interval on each line of sight to establish the depth value interval; in response to the viewpoint position being located outside the inclusion model, using the depth value of the first surface pixel point on each line of sight as the first interval node of the depth value interval to establish the depth value interval.

[0062] In this embodiment, if Figure 5 As shown, the irregular curve area corresponding to A is a schematic diagram of the cross section of an inclusion model, O1 and O2 are schematic diagrams of the viewpoint positions outside and inside the inclusion model, and p1, p2, p3, and p4 are the four intersection points of the line of sight emitted from O1 and the cross section A of the inclusion model. It can be seen that the odd-numbered intersection points of the line of sight emitted from the outside and the inclusion model must be the points that enter the interior of the inclusion model, and the even-numbered intersection points must be the points that exit the interior of the inclusion model. Based on the depth values corresponding to these points, the corresponding depth value interval can be determined to be {[p1, p2], [p3, p4]}, and so on, all depth value intervals can be determined. The corresponding p1 ′ It is an intersection point between the line of sight emitted from O2 and the section A of the inclusion model. It can be seen that the odd-numbered intersection points of the line of sight emitted from the inside and the inclusion model must be points emitted from the inside of the inclusion model, and the even-numbered intersection points must be points injected into the inside of the inclusion model. Then, according to the depth values corresponding to these points and the depth value of the viewpoint position, the entire depth value range can be determined.

[0063] In an optional exemplary embodiment, determining the target modification part and the occluded part of the target model in the image includes: determining the attached attachment on the target model, and judging whether the attached attachment and the attached part of the target model are interspersed; if so, using the attached attachment as the occluded part, and using the attached part of the target model to which the occluded part is attached as the target modification part.

[0064] In this embodiment, the component that penetrates the model is generally an attached accessory on the target model. The attached accessory is an additional component of the target model, which can be various components, such as a hat, brooch, tie, etc. of a character model. When these components are set on the target model, they usually have an attachment site, such as a hat attached to the head and a tie attached to the chest. Then, the occlusion site and the target modification site can be determined by directly checking whether the attached component and the attachment site intersect on the model. Taking a character model wearing a hat as an example, the character model itself is the target model, and the target modification site is the head site or hair site of the character model used to wear the hat, and the occlusion site is the hat model worn on the head site or hair site. The hat model has a fixed shape and is attached to the head site or hair site of the character model.

[0065] In an optional exemplary embodiment, determining the inclusion model of the occluded part includes: copying the model of the occluded part to generate an initial model; determining the maximum movement range of the target modification part, determining the edge vertices of the initial model, and performing multiple extrusion operations on the edge vertices according to the maximum movement range, so that the distance between the extruded vertices and the center of the initial model is greater than the distance between the vertices before extrusion and the center of the initial model; in response to the intermediate model composed of vertices generated by the extrusion operation enclosing the maximum movement range of the target modification part, closing the intermediate model to generate the inclusion model. In this way, the corresponding inclusion model is quickly generated.

[0066] In this embodiment, taking a hat model as an example, a layer of the hat model can be copied to form an initial model. The outer edge of the hat model is then found and extruded along the outer edge to extrude the model piece. The extrusion operation involves extruding a circle of lines along the hat's edge to create another circle of lines. This circle is then resized and extruded again, repeating this process to achieve the desired inclusion shape. The newly extruded surface is generally farther from the center of the hat and extends diagonally downward along the hat (i.e., extending along the range and motion of the hair). Finally, after completely enclosing the range of motion of the hair, the model is closed. The extrusion function is then repeated multiple times, and the points, lines, and surfaces of the model are adjusted based on the engineer's input, so that the inclusion model has a larger range of motion than the character's hair when wearing the hat. Finally, after the inclusion model completely encloses the range of motion of the hat, it is closed, making it a completely closed body. In addition, when copying the hat model, you can make the copied model slightly smaller. You can adjust the PushValue of the model (down by about 0.05) to improve the matching degree between the inclusion and the hat model.

[0067] In an optional exemplary embodiment, determining the depth value of each pixel point of the target modification part includes: determining the viewpoint position of the image, determining the corresponding farthest line of sight position based on the viewpoint position, and determining the depth value of each pixel point based on the positional relationship between each pixel point and the viewpoint position and / or the farthest line of sight position.

[0068] In this embodiment, at a pre-set viewpoint position, the viewpoint position is the nearest camera position, and the farthest sight line position is the farthest camera position. The depth value at the viewpoint position on a sight line ray is 0, and the depth value at the farthest sight line position is 1. Therefore, each pixel point located on the target modification portion corresponds to a sight line ray, so that the position of the pixel point on the sight line ray relative to the viewpoint position and / or the farthest sight line position is fixed, and finally its depth value is determined in the range of 0 to 1. Finally, the depth values of all pixels are obtained. It can be seen that the depth values of pixels located on the same section may be the same, but due to the relative relationship between each pixel point and the viewpoint, the corresponding sight lines are different, so that the image formed according to the depth value can also determine the actual position of each pixel point.

[0069] In an optional exemplary embodiment, performing the modification operation on the area to be modified includes: performing a non-drawing operation on the area to be modified.

[0070] In this embodiment, since the production processes for colored objects and transparent objects are two completely different sets of processes when making models, when addressing problems such as mold penetration, if a transparent drawing method is used, for a hair, part may need to be drawn in color and part may need to be drawn transparently. The two drawing methods are completely different, which will increase the workload of engineers, and the splicing effect after drawing may not be very accurate. Therefore, by not drawing, the corresponding pixel points are not drawn directly, thereby fundamentally reducing the workload of engineers, being simple and accurate, and improving overall efficiency.

[0071] Based on the same concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides an image local processing device.

[0072] refer to Figure 6 , the local image processing device comprises:

[0073] A determination module 610 is configured to determine a target modification portion and an occluded portion of a target model in an image, and determine a depth value of each pixel point of the target modification portion;

[0074] A calculation module 620 is configured to determine an inclusion model of the occluded portion and determine a depth value interval within which the depth value of each pixel point of the target modification portion is located within the inclusion model;

[0075] The modification module 630 is configured to determine an area consisting of pixels whose depth values are outside the depth value range as an area to be modified, and perform a modification operation on the area to be modified.

[0076] For the convenience of description, the above devices are described as being divided into various modules according to their functions. Of course, when implementing the embodiments of the present application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0077] The device of the above embodiment is used to implement the corresponding local image processing method in the above embodiment, and has the beneficial effects of the corresponding local image processing method embodiment, which will not be described in detail here.

[0078] In an optional exemplary embodiment, the calculation module 620 is further configured to:

[0079] Determine the spatial extent of the inclusion model in the image and the depth value of each surface pixel of the inclusion model by using a linked list method pixel by pixel;

[0080] A first correspondence between each of the surface pixels is determined according to the spatial range, and the depth value interval is generated according to the depth values of the corresponding surface pixels.

[0081] In an optional exemplary embodiment, the calculation module 620 is further configured to:

[0082] Establishing a fragment buffer and a first node buffer, and rendering the inclusion model;

[0083] During the rendering process, determining a second correspondence between each image pixel of the image and the surface pixel, recording, using the fragment buffer, a depth value of each surface pixel and a previous surface pixel of the surface pixel determined according to the second correspondence, and recording, using the head node buffer, a position of each image pixel and a last surface pixel corresponding to each image pixel determined according to the second correspondence;

[0084] The spatial range of the inclusion model in the image and the depth value of each surface pixel point of the inclusion model are determined through the fragment buffer and the head node buffer.

[0085] In an optional exemplary embodiment, the calculation module 620 is further configured to:

[0086] During rendering, when the range occupied by a surface pixel on an image pixel exceeds a set threshold, it is determined that the second corresponding relationship exists between the surface pixel and the image pixel.

[0087] In an optional exemplary embodiment, the calculation module 620 is further configured to:

[0088] Determining the number of target models in the image;

[0089] In response to the number of the target models being greater than one, the first node buffer is generated by performing video memory sharing processing on the image, so as to make the first node buffer.

[0090] In an optional exemplary embodiment, the calculation module 620 is further configured to:

[0091] The fragment buffer is generated by numbering the inclusion model corresponding to each target model, so that the fragment buffer corresponds to the inclusion models of all the target models and records the number of each inclusion model; all the inclusion models are projected in the image, and the number of projected pixels is recorded to adjust the video memory size used by the fragment buffer according to the number of projected pixels.

[0092] In an optional exemplary embodiment, the calculation module 620 is further configured to:

[0093] Determining a viewpoint position of the image, wherein a line of sight emitted from the viewpoint position corresponds to an image pixel point of the image;

[0094] In response to the viewpoint being located inside the inclusion model, taking the depth value of the viewpoint as the first interval node of the depth value interval on each line of sight, and establishing the depth value interval;

[0095] In response to the viewpoint being located outside the inclusion model, the depth value of the first surface pixel point on each line of sight is used as the first interval node of the depth value interval to establish the depth value interval.

[0096] In an optional exemplary embodiment, the determining module 610 is further configured to:

[0097] Determining an attached attachment on the target model, and judging whether the attached attachment intersects with an attached portion of the target model;

[0098] If so, the attached attachment is used as the occluding part, and the attached part on the target model to which the occluding part is attached is used as the target modified part.

[0099] In an optional exemplary embodiment, the calculation module 620 is further configured to:

[0100] Copying the model of the blocked part to generate an initial model;

[0101] Determining a maximum movement range of the target modification portion, determining edge vertices of the initial model, and performing multiple extrusion operations on the edge vertices according to the maximum movement range, so that the distance between the extruded vertices and the center of the initial model is greater than the distance between the vertices before extrusion and the center of the initial model;

[0102] In response to the intermediate model composed of vertices generated by the extrusion operation enclosing the maximum movement range of the target modification part, the intermediate model is closed to generate the inclusion model.

[0103] In an optional exemplary embodiment, the determining module 610 is further configured to:

[0104] Determine the viewpoint position of the image, determine the corresponding farthest sight line position according to the viewpoint position, and determine the depth value of each pixel point according to the positional relationship between each pixel point and the viewpoint position and / or the farthest sight line position.

[0105] In an optional exemplary embodiment, the modification module 630 is further configured to:

[0106] A non-drawing operation is performed on the area to be modified.

[0107] Based on the same concept, corresponding to any of the above-mentioned embodiments and methods, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and runnable on the processor, wherein when the processor executes the program, the image local processing method described in any of the above embodiments is implemented.

[0108] Figure 7 10 is a schematic diagram showing a more specific hardware structure of an electronic device provided in this embodiment. The device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are communicatively connected to each other within the device via the bus 1050.

[0109] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0110] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage devices, dynamic storage devices, etc. The memory 1020 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0111] The input / output interface 1030 is used to connect input / output modules to implement information input and output. The input / output modules can be configured as components within the device (not shown in the figure) or can be externally connected to the device to provide corresponding functions. Input devices may include a keyboard, mouse, touch screen, microphone, various sensors, etc., and output devices may include a display, speaker, vibrator, indicator light, etc.

[0112] The communication interface 1040 is used to connect to a communication module (not shown) to enable communication between the device and other devices. The communication module can communicate via a wired method (such as USB, network cable, etc.) or a wireless method (such as mobile network, WiFi, Bluetooth, etc.).

[0113] The bus 1050 comprises a path for transmitting information between the various components of the device (eg, the processor 1010 , the memory 1020 , the input / output interface 1030 , and the communication interface 1040 ).

[0114] It should be noted that although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040, and the bus 1050, in a specific implementation, the device may also include other components necessary for normal operation. In addition, it will be understood by those skilled in the art that the above device may only include the components necessary to implement the embodiments of this specification, and does not necessarily include all the components shown in the figure.

[0115] The electronic device of the above embodiment is used to implement the corresponding local image processing method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0116] Based on the same concept, corresponding to any of the above-mentioned embodiments and methods, the present application also provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the image local processing method described in any of the above embodiments.

[0117] The computer-readable media of this embodiment include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0118] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the local image processing method described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0119] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application (including the claims) is limited to these examples. Within the scope of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0120] In addition, for simplicity of description and discussion, and in order not to make the embodiment of the application difficult to understand, the known power supply / ground connection with integrated circuit (IC) chip and other components may or may not be shown in the accompanying drawings provided. In addition, the device can be shown in the form of a block diagram to avoid making the embodiment of the application difficult to understand, and this also takes into account the following fact, that is, the details of the embodiment of these block diagram devices are highly dependent on the platform to be implemented in the embodiment of the application (that is, these details should be fully within the scope of understanding of those skilled in the art). When specific details (for example, circuit) are set forth to describe exemplary embodiments of the application, it will be apparent to those skilled in the art that the embodiment of the application can be implemented without these specific details or when these specific details are changed. Therefore, these descriptions should be considered to be illustrative rather than restrictive.

[0121] Although the present invention has been described in conjunction with specific embodiments thereof, many alternatives, modifications, and variations of these embodiments will be apparent to those skilled in the art based on the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may utilize the embodiments discussed.

[0122] The embodiments of the present application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of this application.

Claims

1. A local image processing method, characterized in that: include: Determine the target modification part and the blocked part of the target model in the image, and determine the depth value of each pixel point of the target modification part; Determining an inclusion model of the occluded portion, and determining a depth value interval within the inclusion model among the depth values of each pixel of the target modification portion; wherein the inclusion model is a closed model obtained by performing an extrusion operation along the outer edge of the target model to obtain a larger range than the movable range of the target modification portion; Determine an area composed of pixels whose depth values are outside the depth value interval as an area to be modified, and perform a modification operation on the area to be modified.

2. The method according to claim 1, characterized in that The depth value interval within the inclusion model among the depth values of the pixels of the target modification portion is determined, including: Determine the spatial extent of the inclusion model in the image and the depth value of each surface pixel of the inclusion model by using a linked list method pixel by pixel; A first correspondence between each of the surface pixels is determined according to the spatial range, and the depth value interval is generated according to the depth values of the corresponding surface pixels.

3. The method according to claim 2, characterized in that The method of using a linked list method pixel by pixel to determine the spatial range of the inclusion model in the image and the depth value of each surface pixel point of the inclusion model includes: Establishing a fragment buffer and a first node buffer, and rendering the inclusion model; During the rendering process, determining a second correspondence between each image pixel of the image and the surface pixel, recording, using the fragment buffer, a depth value of each surface pixel and a previous surface pixel of the surface pixel determined according to the second correspondence, and recording, using the head node buffer, a position of each image pixel and a last surface pixel corresponding to each image pixel determined according to the second correspondence; The spatial range of the inclusion model in the image and the depth value of each surface pixel point of the inclusion model are determined through the fragment buffer and the head node buffer.

4. The method according to claim 3, characterized in that The determining of a second correspondence between each image pixel point of the image and the surface pixel point includes: During rendering, when the range occupied by a surface pixel on an image pixel exceeds a set threshold, it is determined that the second corresponding relationship exists between the surface pixel and the image pixel.

5. The method according to claim 3, characterized in that The step of establishing the fragment buffer and the head node buffer further includes: Determining the number of target models in the image; In response to the number of the target models exceeding one, the first node buffer is generated by performing video memory sharing processing on the image, so that the first node buffer corresponds to the inclusion models of all the target models.

6. The method according to claim 5, characterized in that In response to the number of the target models exceeding one, the method further includes: The fragment buffer is generated by numbering the inclusion model corresponding to each target model, so that the fragment buffer corresponds to the inclusion models of all the target models and records the number of each inclusion model; all the inclusion models are projected in the image, and the number of projected pixels is recorded to adjust the video memory size used by the fragment buffer according to the number of projected pixels.

7. The method according to claim 2, characterized in that Generating the depth value interval according to the depth value of the corresponding surface pixel point includes: Determining a viewpoint position of the image, wherein a line of sight emitted from the viewpoint position corresponds to an image pixel point of the image; In response to the viewpoint being located inside the inclusion model, taking the depth value of the viewpoint as the first interval node of the depth value interval on each line of sight, and establishing the depth value interval; In response to the viewpoint being located outside the inclusion model, the depth value of the first surface pixel point on each line of sight is used as the first interval node of the depth value interval to establish the depth value interval.

8. The method according to claim 1, characterized in that Determining the target modification part and the blocked part of the target model in the image includes: Determining an attached attachment on the target model, and judging whether the attached attachment intersects with an attached portion of the target model; If so, the attached attachment is used as the occluding part, and the attached part on the target model to which the occluding part is attached is used as the target modified part.

9. The method according to claim 1, characterized in that Determining the inclusion model of the occluded part includes: Copying the model of the blocked part to generate an initial model; Determining a maximum movement range of the target modification portion, determining edge vertices of the initial model, and performing multiple extrusion operations on the edge vertices according to the maximum movement range, so that the distance between the extruded vertices and the center of the initial model is greater than the distance between the vertices before extrusion and the center of the initial model; In response to the intermediate model composed of vertices generated by the extrusion operation enclosing the maximum movement range of the target modification part, the intermediate model is closed to generate the inclusion model.

10. The method according to claim 1, characterized in that Determining the depth value of each pixel point of the target modification part includes: Determine the viewpoint position of the image, determine the corresponding farthest sight line position according to the viewpoint position, and determine the depth value of each pixel point according to the positional relationship between each pixel point and the viewpoint position and / or the farthest sight line position.

11. The method according to claim 1, wherein The performing the modification operation on the to-be-modified area includes: A non-drawing operation is performed on the area to be modified.

12. A local image processing device, characterized in that: include: A determination module, configured to determine a target modification portion and an occluded portion of a target model in an image, and determine a depth value of each pixel point of the target modification portion; a calculation module, configured to determine an inclusion model of the occluded portion, and determine a depth value interval within the inclusion model among the depth values of each pixel of the target modification portion; wherein the inclusion model is a closed model obtained by performing an extrusion operation along the outer edge of the target model to obtain a larger range than the movable range of the target modification portion; The modification module is configured to determine an area composed of pixels whose depth values are outside the depth value interval as an area to be modified, and perform a modification operation on the area to be modified.

13. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method according to any one of claims 1 to 11 is implemented.

14. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to implement the method according to any one of claims 1 to 11.

Images