Picture processing method and device, storage medium and electronic device

Abstract

The application discloses a picture processing method and device, a storage medium and an electronic device, relates to the technical field of smart homes, and comprises the following steps: performing semantic segmentation and line segment detection on a reference picture, determining a target line segment set, wherein the target line segment set comprises a plurality of line segments in the reference picture located in a semantic region; determining an initial placement position of a target picture in the reference picture, and adjusting the initial placement position according to the target line segment set to obtain a target placement position, wherein the target picture is located above the reference picture; and processing the target picture according to the target placement position to display the target picture at the target placement position of the reference picture. The above scheme solves the problem that a target picture cannot be embedded into a reference picture.

Description

Technical Field

[0001] This application relates to the field of smart home technology, and more specifically, to an image processing method and apparatus, a storage medium, and an electronic device. Background Technology

[0002] In home renovation, especially kitchen remodeling, image processing technology is increasingly becoming an important tool for users to preview the effects of renovations. With the continuous upgrading of home appliances, users' demands for upgraded kitchen functions, optimized layouts, and personalized spaces are growing. Currently, several image processing methods exist on the market to assist in the design and visualization of kitchen remodeling:

[0003] 3D modeling technology: By inputting images or videos from multiple angles, a 3D model of the kitchen is constructed. The appliance layout is then automatically adjusted based on user needs and habits to optimize space. However, 3D modeling technology often requires a large amount of image data input and has high computational complexity, making it relatively complex for ordinary users and lacking real-time performance.

[0004] Text description replacement technology: This technique uses deep learning models to replace items in an image based on user-provided text descriptions, such as replacing "dishwasher" with a specific location in an image. While this method achieves item replacement to some extent, it is limited by the model's ability to understand text and the variety of items in the database. It cannot accept user-specified images and suffers from shortcomings in replacement accuracy and object matching.

[0005] Simple image overlay techniques: directly overlaying product images onto a designated position in a kitchen background image without considering the shape and position of the cabinets or the rationality of product placement. This results in generated renderings that often do not match the actual scene and lack realism and reference value.

[0006] While the aforementioned technologies satisfy users' preview needs for partial kitchen renovations to some extent, they still have considerable room for improvement in terms of user experience and practicality because they do not fully consider the diversity of cabinets, the complexity of product placement, and noise interference in reference images.

[0007] There is currently no effective solution to the problem that the target image cannot be well embedded into the reference image in related technologies.

[0008] Therefore, it is necessary to improve the relevant technology to overcome the aforementioned defects. Summary of the Invention

[0009] This invention provides an image processing method, apparatus, storage medium, and electronic device to at least solve the problem of not being able to effectively embed a target image into a reference image.

[0010] According to one aspect of the present invention, an image processing method is provided, comprising: performing semantic segmentation and line segment detection on a reference image to determine a target line segment set, wherein the target line segment set includes multiple line segments located in a semantic region of the reference image; determining an initial placement position of a target image in the reference image, and adjusting the initial placement position according to the target line segment set to obtain a target placement position, wherein the target image is located above the reference image; and processing the target image according to the target placement position to display the target image at the target placement position in the reference image.

[0011] In an exemplary embodiment, adjusting the initial placement position according to the target line segment set to obtain the target placement position includes: when the initial placement position is in the form of a matrix and the four vertices of the rectangle are a first point, a second point, a third point, and a fourth point, respectively, determining a first reference point on the line segment between the first point and the second point, a second reference point on the line segment between the third point and the fourth point, a third reference point on the line segment between the first point and the third point, and a fourth reference point on the line segment between the second point and the fourth point, wherein the first point is located to the left of the second point, the first point is located above the third point, and the second point is located above the fourth point; determining a first horizontal line segment and a second horizontal line segment in the target line segment set based on the first reference point and the second reference point, and determining a first vertical line segment and a second vertical line segment in the target line segment set based on the third reference point and the fourth reference point; adjusting the initial placement position according to the four vertices of the rectangle, the first horizontal line segment, the second horizontal line segment, the first vertical line segment, and the second vertical line segment to obtain the target placement position.

[0012] In an exemplary embodiment, determining a first reference point on the line segment between the first point and the second point, and determining a second reference point on the line segment between the third point and the fourth point, includes: determining the product type corresponding to the target image, and determining a corresponding reference point selection rule based on the product type; determining the first reference point on the line segment between the first point and the second point, and determining the second reference point on the line segment between the third point and the fourth point, according to the reference point selection rule; determining a third reference point on the line segment between the first point and the third point, and determining a fourth reference point on the line segment between the second point and the fourth point, includes: determining the third reference point on the line segment between the first point and the third point, and determining the fourth reference point on the line segment between the second point and the fourth point, according to the reference point selection rule.

[0013] In an exemplary embodiment, determining a first horizontal line segment and a second horizontal line segment in the target line segment set based on the first reference point and the second reference point includes: determining the first horizontal line segment closest to the first reference point and the second horizontal line segment closest to the second reference point in the target line segment set; determining a first vertical line segment and a second vertical line segment in the target line segment set based on the third reference point and the fourth reference point includes: determining the first vertical line segment closest to the third reference point and the second vertical line segment closest to the fourth reference point in the target line segment set.

[0014] In an exemplary embodiment, before adjusting the initial placement position based on the four vertices of the rectangle, the first horizontal line segment, the second horizontal line segment, the first vertical line segment, and the second vertical line segment to obtain the target placement position, the method further includes: extending the first horizontal line segment and the second horizontal line segment when they are not parallel, and determining the intersection point as a fifth reference point; updating the first horizontal line segment to the line connecting the first point and the fifth reference point, and updating the second horizontal line segment to the line connecting the third point and the fifth reference point.

[0015] In an exemplary embodiment, adjusting the initial placement position based on the four vertices of the rectangle, the first horizontal line segment, the second horizontal line segment, the first vertical line segment, and the second vertical line segment to obtain a target placement position includes: determining a second target point on the first horizontal line segment based on the distance between the first point and the second point and the slope of the first horizontal line segment, wherein the second target point is located to the right of the first point; determining a third vertical line segment based on the first vertical line segment and the first point, and determining a fourth vertical line segment based on the second vertical line segment and the second target point, wherein the slope of the third vertical line segment is equal to the slope of the first vertical line segment and the third vertical line segment passes through the first point; the slope of the fourth vertical line segment is equal to the slope of the second vertical line segment and the fourth vertical line segment passes through the second target point; determining the intersection of the third vertical line segment and the second horizontal line segment as the third target point, and determining the intersection of the fourth vertical line segment and the second horizontal line segment as the fourth target point; and determining the target placement position based on the first point, the second target point, the third target point, and the fourth target point.

[0016] In an exemplary embodiment, semantic segmentation and line segment detection are performed on a reference image to determine a target set of line segments, including: performing semantic segmentation on the reference image using a semantic segmentation model to determine multiple semantic regions of the reference image; and performing line segment detection on the reference image using a line segment detection model, and determining the target set of line segments from the detected line segments, wherein the line segments in the target set of line segments include horizontal line segments and vertical line segments, and the slope of the horizontal line segments is less than 45 degrees or greater than 135 degrees.

[0017] According to another aspect of the present invention, an image processing apparatus is also provided, comprising: a determining module, configured to perform semantic segmentation and line segment detection on a reference image to determine a target line segment set, wherein the target line segment set includes multiple line segments located in a semantic region of the reference image; an adjusting module, configured to determine an initial placement position of a target image in the reference image, and adjust the initial placement position according to the target line segment set to obtain a target placement position, wherein the target image is located above the reference image; and a processing module, configured to process the target image according to the target placement position to display the target image at the target placement position in the reference image.

[0018] According to another aspect of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer program, and the computer program is configured to execute the above-described image processing method at runtime.

[0019] According to another aspect of the present invention, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the image processing method described above through the computer program.

[0020] This application utilizes semantic segmentation technology to accurately identify semantic regions such as objects (e.g., cabinets) in a reference image. Combined with line segment detection, it identifies key line segments of the cabinet structure. This process ensures the accuracy and completeness of the understanding of the reference image, providing a solid foundation for subsequent localization. Secondly, based on the initial placement of the target image, its specific position in the reference image is intelligently adjusted. This adjustment is based on the target line segment set, taking into account the shape, angle, and occlusion of objects, thus improving the accuracy and rationality of localization. Finally, the target image is processed according to its placement position. The entire solution, through three key steps—semantic understanding, intelligent adjustment, and precise fusion—solves the problems of inaccurate positioning and unnatural fusion effects in the image embedding process of existing technologies, achieving highly realistic embedding of the target image into the reference image. Attached Figure Description

[0021] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0022] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a schematic diagram of the hardware environment for an image processing method according to an embodiment of this application;

[0024] Figure 2 This is a flowchart (a) of an image processing method according to an embodiment of the present invention;

[0025] Figure 3 This is a schematic diagram illustrating the effect of semantic segmentation and line segment detection according to an embodiment of the present invention;

[0026] Figure 4 This is a schematic diagram of an initial placement position according to an embodiment of the present invention;

[0027] Figure 5 This is a schematic diagram of a processing result according to an embodiment of the present invention;

[0028] Figure 6 This is a flowchart (II) of an image processing method according to an embodiment of the present invention;

[0029] Figure 7 This is a structural block diagram of an optional image processing apparatus according to an embodiment of the present invention. Detailed Implementation

[0030] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0031] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0032] According to one aspect of the embodiments of this application, an image processing method is provided. This image processing method is widely used in whole-house intelligent digital control application scenarios such as smart homes, smart home ecosystems, and intelligencehouse ecosystems. Optionally, in this embodiment, the above-mentioned image processing method can be applied to, for example... Figure 1 The hardware environment shown consists of terminal device 102 and server 104. For example... Figure 1 As shown, server 104 is connected to terminal device 102 via a network and can be used to provide services (such as application services) to the terminal or clients installed on the terminal. A database can be set up on the server or independently of the server to provide data storage services for server 104. Cloud computing and / or edge computing services can be configured on the server or independently of the server to provide data processing services for server 104.

[0033] The aforementioned networks may include, but are not limited to, wired networks and wireless networks. Wired networks may include, but are not limited to, wide area networks (WANs), metropolitan area networks (MANs), and local area networks (LANs). Wireless networks may include, but are not limited to, Wi-Fi (Wireless Fidelity) and Bluetooth. Terminal device 102 may not be limited to PCs, mobile phones, tablets, smart air conditioners, smart range hoods, smart refrigerators, smart ovens, smart stoves, smart washing machines, smart water heaters, smart washing equipment, smart dishwashers, smart projectors, smart TVs, smart clothes racks, smart curtains, smart audio-visual equipment, smart sockets, smart speakers, smart speakers, smart fresh air systems, smart kitchen and bathroom equipment, smart bathroom fixtures, smart robot vacuums, smart window cleaning robots, smart mopping robots, smart air purifiers, smart steam ovens, smart microwave ovens, smart water heaters, smart air purifiers, smart water dispensers, smart door locks, etc.

[0034] To address the aforementioned issues, this embodiment provides an image processing method, which may, but is not limited to, be applied to the aforementioned terminal device 102 or server 104. Figure 2 This is a flowchart (I) of an image processing method according to an embodiment of the present invention, which includes the following steps S202-S206:

[0035] Step S202: Perform semantic segmentation and line segment detection on the reference image to determine the target line segment set, wherein the target line segment set includes multiple line segments located in the semantic region of the reference image;

[0036] Optionally, the reference image can be a background image, such as a kitchen background image.

[0037] In an exemplary embodiment, step S202 above can be implemented as follows: semantic segmentation of the reference image is performed using a semantic segmentation model to determine multiple semantic regions of the reference image; and line segment detection is performed on the reference image using a line segment detection model, and the target line segment set is determined from the detected line segments, wherein the line segments in the target line segment set include horizontal line segments and vertical line segments, and the slope of the horizontal line segments is less than 45 degrees or greater than 135 degrees.

[0038] It should be noted that in this application, in-depth processing of the reference images is key to achieving accurate embedding of home appliance products. Firstly, by using advanced semantic segmentation models, such as mmsegmentation, a detailed analysis of the kitchen reference images is performed, enabling high-precision differentiation and labeling of semantic regions for large furniture items such as cabinets, drawers, and refrigerators. This process is equivalent to "categorizing" the reference images, clarifying the attributes and functions of each region, and providing a clear framework and guidance for subsequent line segment detection and product positioning.

[0039] Next, line segment detection models, such as M-LSD, are used to detect line segments throughout the reference image. The model not only identifies all line segments in the reference image but also intelligently filters out those located within defined semantic regions, forming a target line segment set. Particular attention is paid to line segments with slopes less than 45 degrees or greater than 135 degrees, as these often represent the horizontal edges of the cabinet and are crucial references for subsequent product placement. This filtering method extracts the complex information from the reference image into a set of target line segments with clear direction and location, laying a solid data foundation for perspective transformation and precise positioning of product images.

[0040] To better understand, Figure 3 This diagram illustrates the effect of semantic segmentation and line segment detection. Figure 3The left side is a reference image, the middle side is a schematic diagram of the effect after semantic segmentation, and the right side is a schematic diagram of the effect after line segment detection.

[0041] Step S204: Determine the initial placement position of the target image in the reference image, and adjust the initial placement position according to the target line segment set to obtain the target placement position, wherein the target image is located above the reference image;

[0042] Optionally, the target image is an image of a kitchen item.

[0043] Figure 4 This diagram illustrates an initial placement position, which is in the form of a matrix, with the four vertices of the rectangle representing the first point (…). Figure 4 P1 in the middle), the second point ( Figure 4 P2 in the middle), the third point ( Figure 4 P3 and the fourth point (in the middle) Figure 4 (P4 in the text).

[0044] Step S206: Process the target image according to the target placement position so as to display the target image at the target placement position in the reference image.

[0045] Optionally, a perspective matrix can be determined based on the target's placement position, and then the target image can be processed based on the perspective matrix.

[0046] The above steps utilize semantic segmentation technology to accurately identify semantic regions such as objects (e.g., cabinets) in the reference image, and combine this with line segment detection to determine key line segments of the cabinet structure. This process ensures the accuracy and completeness of the understanding of the reference image, providing a solid foundation for subsequent localization. Secondly, based on the initial placement of the target image, its specific position in the reference image is intelligently adjusted. This adjustment is based on the target line segment set, taking into account the shape, angle, and occlusion of objects, improving the accuracy and rationality of localization. Finally, the target image is processed based on the perspective matrix of the target placement position. The entire solution, through three key steps—semantic understanding, intelligent adjustment, and precise fusion—solves the problems of inaccurate positioning and unnatural fusion effects in the original technology's image embedding process, achieving highly realistic embedding of the target image into the reference image.

[0047] In an exemplary embodiment, step S204 above includes the following steps S21-S23:

[0048] Step S21: When the initial placement position is in the form of a matrix and the four vertices of the rectangle are the first point, the second point, the third point, and the fourth point, a first reference point is determined on the line segment between the first point and the second point, a second reference point is determined on the line segment between the third point and the fourth point, a third reference point is determined on the line segment between the first point and the third point, and a fourth reference point is determined on the line segment between the second point and the fourth point. The first point is located to the left of the second point, the first point is located above the third point, and the second point is located above the fourth point.

[0049] In an exemplary embodiment, determining a first reference point on the line segment between the first point and the second point, and determining a second reference point on the line segment between the third point and the fourth point, includes: determining the product type corresponding to the target image, and determining a corresponding reference point selection rule based on the product type; determining the first reference point on the line segment between the first point and the second point, and determining the second reference point on the line segment between the third point and the fourth point, based on the reference point selection rule;

[0050] For example, if the target image corresponds to a dishwasher, the first reference point can be selected at the 1 / 6 mark on the left side of line segment P1P5. This is to address the situation where a product image entered at a turn is close to the left cabinet and is mistakenly identified as being placed in the left cabinet. Line segment P3P4 is processed in the same way to find the second reference point.

[0051] In an exemplary embodiment, determining a third reference point on the line segment between the first point and the third point, and determining a fourth reference point on the line segment between the second point and the fourth point, includes: determining a third reference point on the line segment between the first point and the third point, and determining a fourth reference point on the line segment between the second point and the fourth point, according to a reference point selection rule.

[0052] For example, the midpoint of the line segment between the first point and the third point can be determined as the third reference point, and the midpoint of the line segment between the second point and the fourth point can be determined as the fourth reference point.

[0053] Step S22: Determine a first horizontal line segment and a second horizontal line segment in the target line segment set based on the first reference point and the second reference point, and determine a first vertical line segment and a second vertical line segment in the target line segment set based on the third reference point and the fourth reference point;

[0054] In an exemplary embodiment, determining a first horizontal line segment and a second horizontal line segment in the target line segment set based on the first reference point and the second reference point includes: determining the first horizontal line segment closest to the first reference point and the second horizontal line segment closest to the second reference point in the target line segment set;

[0055] In an exemplary embodiment, prior to step S23 below, the method further includes: extending the first horizontal line segment and the second horizontal line segment when they are not parallel, and determining the intersection point as a fifth reference point; updating the first horizontal line segment to the line connecting the first point and the fifth reference point, and updating the second horizontal line segment to the line connecting the third point and the fifth reference point.

[0056] In an exemplary embodiment, determining a first longitudinal line segment and a second longitudinal line segment in the target line segment set based on the third reference point and the fourth reference point includes: determining the first longitudinal line segment closest to the third reference point and the second longitudinal line segment closest to the fourth reference point in the target line segment set.

[0057] Step S23: Adjust the initial placement position according to the four vertices of the rectangle, the first horizontal line segment, the second horizontal line segment, the first vertical line segment, and the second vertical line segment to obtain the target placement position.

[0058] In an exemplary embodiment, step S23 is implemented through the following steps S231-S234:

[0059] Step S231: Determine a second target point on the first horizontal line segment based on the distance between the first point and the second point and the slope of the first horizontal line segment, wherein the second target point is located to the right of the first point;

[0060] Optionally, the distance between the second target point and the first point can be equal to or different from the distance between the second point and the first point, and needs to be finely adjusted according to the slope of the first horizontal line segment.

[0061] Step S232: Determine a third longitudinal line segment based on the first longitudinal line segment and the first point, and determine a fourth longitudinal line segment based on the second longitudinal line segment and the second target point, wherein the slope of the third longitudinal line segment is equal to the slope of the first longitudinal line segment and the third longitudinal line segment passes through the first point; the slope of the fourth longitudinal line segment is equal to the slope of the second longitudinal line segment and the fourth longitudinal line segment passes through the second target point;

[0062] Step S233: Determine the intersection of the third longitudinal line segment and the second transverse line segment as the third target point, and determine the intersection of the fourth longitudinal line segment and the second transverse line segment as the fourth target point;

[0063] Step S234: Determine the target placement position based on the first point, the second target point, the third target point, and the fourth target point.

[0064] It should be noted that the above steps, through a series of precise coordinate transformations and line segment calibrations, enable intelligent embedding of product images in complex backgrounds, solving the problem of poor embedding results caused by complex details in reference images and inaccurate positioning of product images in traditional methods.

[0065] It should be noted that, Figure 5 The diagram illustrates one possible processing result. Figure 5 The image on the left is the original reference image. Figure 5 The image in the middle shows the original location of the product. Figure 5 The right side shows the processed reference image.

[0066] To better understand, Figure 6 This is a flowchart (II) of an image processing method according to an embodiment of the present invention, for better understanding. Figure 6 The relevant steps are illustrated below with examples:

[0067] (1) Using the mature model technology available today, the semantic segmentation model is used to perform semantic segmentation on the kitchen reference image to obtain the semantic regions of large objects such as cabinets, drawers, and refrigerators; the line segment detection model M-LSD is used to perform line segment detection on the kitchen reference image to detect all the line segments in the reference image and appropriately integrate them into usable long straight reference lines.

[0068] (2) Obtain the detected line segments within the semantic region;

[0069] (3) Divide the line segments according to a certain tilt angle, such as 45° or 135°, and divide the line segments into horizontal line segments and vertical line segments;

[0070] (4) The input items will be given an initial placement position, i.e., the four corner points P1P2P3P4, such as... Figure 4 The red dot is at the end.

[0071] (5) Find a suitable position on line segment P1P2 as a reference point. Here, we take 1 / 6 of the distance from the left as P5 to deal with the situation where the product image entered at the turn is close to the left cabinet and is mistakenly judged as being placed in the left cabinet. Do the same for line segment P3P4 to find reference point P6.

[0072] (6) Find the lines closest to points P5 and P6 in the horizontal line segments respectively as horizontal reference line segments;

[0073] (7) Extend the two horizontal reference lines to obtain the intersection point P7. Draw lines from points P1 and P3 to the intersection point P7 as new horizontal reference lines. If the initial two horizontal reference lines are parallel, no update is made. The final reference lines are l1 and l2.

[0074] (8) If the position of the left point in the initial range of the product is relatively accurate, then P1 remains unchanged. Take the length of line segment P1P2 and cut out a new line segment on line segment P1P7. One end of the line segment is P1, and the other end is the new P2 point, i.e., new_P2.

[0075] (9) Take the midpoints of line segments P1P3 and P2P4 as reference points, and obtain points P8 and P9 respectively. Find the vertical line segment closest to points P8 and P9 as the reference line segment.

[0076] (10) Based on the slope of the longitudinal reference line segment, draw parallel lines to P1 and new_P2 respectively to obtain new longitudinal reference lines l3 and l4;

[0077] (11) Reference lines l3 and l2 intersect at point new_P3, and reference lines l4 and l2 intersect at point new_P4. The points P1, new_P2, new_P3, and new_P4 obtained can determine the perspective matrix. Based on the perspective matrix, the product image is rendered using perspective. The perspective result replaces the corresponding position in the background image to obtain the final modified effect image. See Figure 5 As shown.

[0078] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0079] This embodiment also provides an image processing apparatus for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, hardware implementations, or a combination of software and hardware, are also possible and contemplated.

[0080] Figure 7 This is a structural block diagram of an image processing apparatus according to an embodiment of the present invention, the apparatus comprising:

[0081] The determination module 702 is used to perform semantic segmentation and line segment detection on the reference image and determine the target line segment set, wherein the target line segment set includes multiple line segments located in the semantic region of the reference image;

[0082] The adjustment module 704 is used to determine the initial placement position of the target image in the reference image, and adjust the initial placement position according to the target line segment set to obtain the target placement position, wherein the target image is located above the reference image;

[0083] The processing module 706 is used to process the target image according to the target placement position, so as to display the target image at the target placement position of the reference image.

[0084] The aforementioned device utilizes semantic segmentation technology to accurately identify semantic regions such as objects (e.g., cabinets) in reference images. Combined with line segment detection, it identifies key line segments of the cabinet structure. This process ensures the accuracy and completeness of the understanding of the reference image, providing a solid foundation for subsequent localization. Secondly, based on the initial placement of the target image, its specific position within the reference image is intelligently adjusted. This adjustment is based on the target line segment set, taking into account the shape, angle, and occlusion of objects, thus improving the accuracy and rationality of localization. Finally, the target image is processed according to its placement position. The entire solution, through three key steps—semantic understanding, intelligent adjustment, and precise fusion—solves the problems of inaccurate positioning and unnatural fusion effects inherent in previous technologies during image embedding, achieving highly realistic embedding of the target image into the reference image.

[0085] In an exemplary embodiment, the adjustment module 704 is further configured to, when the initial placement position is in the form of a matrix and the four vertices of the rectangle are a first point, a second point, a third point, and a fourth point, determine a first reference point on the line segment between the first point and the second point, a second reference point on the line segment between the third point and the fourth point, a third reference point on the line segment between the first point and the third point, and a fourth reference point on the line segment between the second point and the fourth point, wherein the first point is located to the left of the second point, the first point is located above the third point, and the second point is located above the fourth point; determine a first horizontal line segment and a second horizontal line segment in the target line segment set based on the first reference point and the second reference point, and determine a first vertical line segment and a second vertical line segment in the target line segment set based on the third reference point and the fourth reference point; and adjust the initial placement position according to the four vertices of the rectangle, the first horizontal line segment, the second horizontal line segment, the first vertical line segment, and the second vertical line segment to obtain the target placement position.

[0086] In an exemplary embodiment, the adjustment module 704 is further configured to determine the product type corresponding to the target image, and determine the corresponding reference point selection rule according to the product type; determine a first reference point on the line segment between the first point and the second point and a second reference point on the line segment between the third point and the fourth point according to the reference point selection rule; determine a third reference point on the line segment between the first point and the third point and a fourth reference point on the line segment between the second point and the fourth point according to the reference point selection rule.

[0087] In an exemplary embodiment, the adjustment module 704 is further configured to determine, in the target line segment set, a first horizontal line segment that is closest to the first reference point and a second horizontal line segment that is closest to the second reference point; and to determine, in the target line segment set, a first vertical line segment that is closest to the third reference point and a second vertical line segment that is closest to the fourth reference point.

[0088] In an exemplary embodiment, the adjustment module 704 is further configured to, before adjusting the initial placement position according to the four vertices of the rectangle, the first horizontal line segment, the second horizontal line segment, the first vertical line segment, and the second vertical line segment to obtain the target placement position, extend the first horizontal line segment and the second horizontal line segment if the first horizontal line segment and the second horizontal line segment are not parallel, and determine the intersection point as the fifth reference point; update the first horizontal line segment to the line connecting the first point and the fifth reference point, and update the second horizontal line segment to the line connecting the third point and the fifth reference point.

[0089] In an exemplary embodiment, the adjustment module 704 is further configured to: determine a second target point on the first horizontal line segment based on the distance between the first point and the second point and the slope of the first horizontal line segment, wherein the second target point is located to the right of the first point; determine a third vertical line segment based on the first vertical line segment and the first point, and determine a fourth vertical line segment based on the second vertical line segment and the second target point, wherein the slope of the third vertical line segment is equal to the slope of the first vertical line segment and the third vertical line segment passes through the first point; the slope of the fourth vertical line segment is equal to the slope of the second vertical line segment and the fourth vertical line segment passes through the second target point; determine the intersection of the third vertical line segment and the second horizontal line segment as the third target point, and determine the intersection of the fourth vertical line segment and the second horizontal line segment as the fourth target point; and determine the target placement position based on the first point, the second target point, the third target point, and the fourth target point.

[0090] In an exemplary embodiment, the determining module 702 is further configured to perform semantic segmentation on the reference image using a semantic segmentation model to determine multiple semantic regions of the reference image; and to perform line segment detection on the reference image using a line segment detection model, and determine the target line segment set from the detected line segments, wherein the line segments in the target line segment set include horizontal line segments and vertical line segments, and the slope of the horizontal line segments is less than 45 degrees or greater than 135 degrees.

[0091] Embodiments of the present invention also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to perform the steps in any of the above method embodiments when executed.

[0092] Optionally, in this embodiment, the storage medium may be configured to store a computer program for performing the following steps:

[0093] S1, perform semantic segmentation and line segment detection on the reference image to determine the target line segment set, wherein the target line segment set includes multiple line segments located in the semantic region of the reference image;

[0094] S2, determine the initial placement position of the target image in the reference image, and adjust the initial placement position according to the target line segment set to obtain the target placement position, wherein the target image is located above the reference image;

[0095] S3, process the target image according to the target placement position, so as to display the target image at the target placement position of the reference image.

[0096] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0097] Specific examples in this embodiment can be found in the examples described in the above embodiments and exemplary implementations, and will not be repeated here.

[0098] Embodiments of the present invention also provide an electronic device including a memory and a processor, the memory storing a computer program and the processor being configured to run the computer program to perform the steps in any of the above method embodiments.

[0099] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0100] S1, perform semantic segmentation and line segment detection on the reference image to determine the target line segment set, wherein the target line segment set includes multiple line segments located in the semantic region of the reference image;

[0101] S2, determine the initial placement position of the target image in the reference image, and adjust the initial placement position according to the target line segment set to obtain the target placement position, wherein the target image is located above the reference image;

[0102] S3, process the target image according to the target placement position, so as to display the target image at the target placement position of the reference image.

[0103] In one exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0104] Specific examples in this embodiment can be found in the examples described in the above embodiments and exemplary implementations, and will not be repeated here.

[0105] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above method embodiments.

[0106] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in any of the above method embodiments.

[0107] The embodiments described herein also provide a computer program that includes computer instructions stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps in any of the above method embodiments.

[0108] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those described herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0109] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. An image processing method, characterized in that, include: Semantic segmentation and line segment detection are performed on the reference image to determine the target line segment set, wherein the target line segment set includes multiple line segments located in the semantic region of the reference image; The initial placement position of the target image in the reference image is determined, and the initial placement position is adjusted according to the target line segment set to obtain the target placement position, wherein the target image is located above the reference image; The target image is processed according to the target placement position so as to display the target image at the target placement position in the reference image; The process of adjusting the initial placement position according to the target line segment set to obtain the target placement position includes: when the initial placement position is in the form of a matrix and the four vertices of the rectangle are a first point, a second point, a third point, and a fourth point, respectively, determining a first reference point on the line segment between the first point and the second point, a second reference point on the line segment between the third point and the fourth point, a third reference point on the line segment between the first point and the third point, and a fourth reference point on the line segment between the second point and the fourth point, wherein the first point is located to the left of the second point, the first point is located above the third point, and the second point is located above the fourth point; determining a first horizontal line segment and a second horizontal line segment in the target line segment set based on the first reference point and the second reference point, and determining a first vertical line segment and a second vertical line segment in the target line segment set based on the third reference point and the fourth reference point; and adjusting the initial placement position according to the four vertices of the rectangle, the first horizontal line segment, the second horizontal line segment, the first vertical line segment, and the second vertical line segment to obtain the target placement position. Specifically, determining a first horizontal line segment and a second horizontal line segment in the target line segment set based on the first reference point and the second reference point includes: determining the first horizontal line segment closest to the first reference point and the second horizontal line segment closest to the second reference point in the target line segment set; determining a first vertical line segment and a second vertical line segment in the target line segment set based on the third reference point and the fourth reference point includes: determining the first vertical line segment closest to the third reference point and the second vertical line segment closest to the fourth reference point in the target line segment set; Before adjusting the initial placement position based on the four vertices of the rectangle, the first horizontal line segment, the second horizontal line segment, the first vertical line segment, and the second vertical line segment to obtain the target placement position, the method further includes: extending the first horizontal line segment and the second horizontal line segment when they are not parallel, and determining the intersection point as the fifth reference point; updating the first horizontal line segment to the line connecting the first point and the fifth reference point, and updating the second horizontal line segment to the line connecting the third point and the fifth reference point; The process of adjusting the initial placement position based on the four vertices of the rectangle, the first horizontal line segment, the second horizontal line segment, the first vertical line segment, and the second vertical line segment to obtain the target placement position includes: determining a second target point on the first horizontal line segment based on the distance between the first point and the second point and the slope of the first horizontal line segment, wherein the second target point is located to the right of the first point; determining a third vertical line segment based on the first vertical line segment and the first point, and determining a fourth vertical line segment based on the second vertical line segment and the second target point, wherein the slope of the third vertical line segment is equal to the slope of the first vertical line segment and the third vertical line segment passes through the first point; the slope of the fourth vertical line segment is equal to the slope of the second vertical line segment and the fourth vertical line segment passes through the second target point; determining the intersection of the third vertical line segment and the second horizontal line segment as the third target point, and determining the intersection of the fourth vertical line segment and the second horizontal line segment as the fourth target point; and determining the target placement position based on the first point, the second target point, the third target point, and the fourth target point.

2. The image processing method according to claim 1, characterized in that, Determining a first reference point on the line segment between the first point and the second point, and determining a second reference point on the line segment between the third point and the fourth point, includes: determining the product type corresponding to the target image, and determining a corresponding reference point selection rule based on the product type; determining the first reference point on the line segment between the first point and the second point, and determining the second reference point on the line segment between the third point and the fourth point, based on the reference point selection rule. Determining a third reference point on the line segment between the first point and the third point, and determining a fourth reference point on the line segment between the second point and the fourth point, includes: determining a third reference point on the line segment between the first point and the third point, and determining a fourth reference point on the line segment between the second point and the fourth point, according to the reference point selection rules.

3. The image processing method according to claim 1, characterized in that, Semantic segmentation and line segment detection are performed on the reference image to determine the target line segment set, including: The reference image is semantically segmented using a semantic segmentation model to determine multiple semantic regions of the reference image; and The reference image is used to detect line segments using a line segment detection model, and the target line segment set is determined from the detected line segments. The line segments in the target line segment set include horizontal line segments and vertical line segments, and the slope of the horizontal line segments is less than 45 degrees or greater than 135 degrees.

4. An image processing apparatus, characterized in that, include: The determination module is used to perform semantic segmentation and line segment detection on the reference image and determine the target line segment set, wherein the target line segment set includes multiple line segments located in the semantic region of the reference image; An adjustment module is used to determine the initial placement position of the target image in the reference image, and adjust the initial placement position according to the target line segment set to obtain the target placement position, wherein the target image is located above the reference image; The processing module is used to process the target image according to the target placement position, so as to display the target image at the target placement position in the reference image; The adjustment module is further configured to, when the initial placement position is a matrix and the four vertices of the rectangle are a first point, a second point, a third point, and a fourth point, determine a first reference point on the line segment between the first point and the second point, a second reference point on the line segment between the third point and the fourth point, a third reference point on the line segment between the first point and the third point, and a fourth reference point on the line segment between the second point and the fourth point, wherein the first point is located to the left of the second point, the first point is located above the third point, and the second point is located above the fourth point; determine a first horizontal line segment and a second horizontal line segment in the target line segment set based on the first reference point and the second reference point, and determine a first vertical line segment and a second vertical line segment in the target line segment set based on the third reference point and the fourth reference point; and adjust the initial placement position according to the four vertices of the rectangle, the first horizontal line segment, the second horizontal line segment, the first vertical line segment, and the second vertical line segment to obtain the target placement position; The adjustment module is further configured to determine, in the target line segment set, a first horizontal line segment closest to the first reference point and a second horizontal line segment closest to the second reference point; and to determine, based on the third reference point and the fourth reference point, a first vertical line segment and a second vertical line segment in the target line segment set, including: determining, in the target line segment set, a first vertical line segment closest to the third reference point and a second vertical line segment closest to the fourth reference point; The adjustment module is further configured to extend the first horizontal line segment and the second horizontal line segment when they are not parallel, and determine the intersection point as the fifth reference point; update the first horizontal line segment to the line connecting the first point and the fifth reference point; and update the second horizontal line segment to the line connecting the third point and the fifth reference point. The adjustment module is further configured to: determine a second target point on the first horizontal line segment based on the distance between the first point and the second point and the slope of the first horizontal line segment, wherein the second target point is located to the right of the first point; determine a third vertical line segment based on the first vertical line segment and the first point, and determine a fourth vertical line segment based on the second vertical line segment and the second target point, wherein the slope of the third vertical line segment is equal to the slope of the first vertical line segment and the third vertical line segment passes through the first point; the slope of the fourth vertical line segment is equal to the slope of the second vertical line segment and the fourth vertical line segment passes through the second target point; determine the intersection of the third vertical line segment and the second horizontal line segment as the third target point, and determine the intersection of the fourth vertical line segment and the second horizontal line segment as the fourth target point; and determine the target placement position based on the first point, the second target point, the third target point, and the fourth target point.

5. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program, when executed, performs the method of any one of claims 1 to 3.

6. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to execute the method of any one of claims 1 to 3 through the computer program.

Images