Patterning method, device and electronic device

Abstract

The present application discloses a composition method, device and electronic device, which belongs to the field of communication technology. The method includes: obtaining a background image, and determining N target objects; performing visual saliency prediction on the background image, and determining M first marking points in the background image; selecting T second marking points from the M first marking points, and connecting the T second marking points based on a preset layout graph to obtain a target layout graph; according to the target layout graph, adding the N target objects to the background image, generating and displaying an object layout image; wherein N, M, and T are all positive integers greater than 1.

Description

Technical Field

[0001] The present application belongs to the field of communication technology, and specifically relates to a composition method, device and electronic equipment. Background Art

[0002] The need to place items is very common in people's daily lives, for example, placing furniture, setting up banquet tables, placing items in lockers, etc.

[0003] When placing items, people often place them according to their categories and sizes. This method can only make the items neatly arranged, but the aesthetics of the item layout is poor. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a composition method, device and electronic device, which can solve the problem of poor aesthetics of the layout of objects when placing objects.

[0005] In a first aspect, an embodiment of the present application provides a patterning method, the method comprising:

[0006] Obtain a background image and determine N target objects;

[0007] Performing visual saliency prediction on the background image to determine M first marked points in the background image;

[0008] Selecting T second marking points from the M first marking points, and connecting the T second marking points based on a preset layout graph to obtain a target layout graph;

[0009] According to the target layout graphic, the N target objects are added to the background image to generate and display an object layout image;

[0010] Wherein, N, M, and T are all positive integers greater than 1.

[0011] In a second aspect, an embodiment of the present application provides a patterning device, the device comprising:

[0012] An acquisition module is used to acquire a background image and determine N target objects;

[0013] A first determination module, configured to perform visual saliency prediction on the background image and determine M first marking points in the background image;

[0014] A selection module, configured to select T second marking points from the M first marking points, and connect the T second marking points based on a preset layout graph to obtain a target layout graph;

[0015] A first generating module, configured to add the N target objects to the background image according to the target layout graphic to generate an object layout image;

[0016] A display module, used for displaying an item layout image;

[0017] Wherein, N, M, and T are all positive integers greater than 1.

[0018] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the program or instructions are executed by the processor, the steps of the method described in the first aspect are implemented.

[0019] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0020] In a fifth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the method described in the first aspect.

[0021] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the method described in the first aspect.

[0022] In an embodiment of the present application, based on eye tracking technology, visual significance prediction is performed on the background image, and M first marking points in the background image are determined. Then, in combination with the principle of aesthetic composition, T second marking points are selected from the M first marking points, and the T second marking points are connected to obtain a target layout graphic. In this way, according to the target layout graphic, N target objects are added to the background image, and an object layout image that conforms to the observation laws and aesthetic laws of the human eye can be generated. Users arrange objects based on the object layout image, and the layout is more reasonable, thereby solving the problem of poor layout aesthetics in the existing object placement method. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a schematic diagram of a flow chart of a patterning method provided in an embodiment of the present application;

[0024] Figure 2a is a schematic diagram of a background image provided in an embodiment of the present application;

[0025] Figure 2b is a schematic diagram of another background image provided in an embodiment of the present application;

[0026] Figure 3 is a schematic diagram of a target object provided in an embodiment of the present application;

[0027] Figure 4 is a schematic diagram of dividing a background image provided by an embodiment of the present application;

[0028] Figure 5 is a schematic diagram of a target layout graph provided in an embodiment of the present application;

[0029] Figure 6 is a schematic diagram of another target layout graphic provided in an embodiment of the present application;

[0030] Figure 7 is a structural schematic diagram of a patterning device provided in an embodiment of the present application;

[0031] Figure 8 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application;

[0032] Fig. 9 It is a schematic diagram of the structure of another electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0033] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0034] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0035] The composition method provided in the embodiment of the present application is described in detail below through specific embodiments and their application scenarios in conjunction with the accompanying drawings.

[0036] Please see Figure 1 , which is a flow chart of a composition method provided in an embodiment of the present application. The method can be applied to an electronic device, which can be a mobile phone, a tablet computer, a laptop computer, etc. Figure 1As shown, the method may include steps 1100 to 1400, which are described in detail below.

[0037] Step 1100, obtaining a background image and determining N target objects.

[0038] In this embodiment, the background image may be an image collected by the user including the target background. The target background may be a background object for placing the target object, such as a desk, a dining table, a locker, etc.

[0039] In a specific implementation, the background image can be obtained by selecting one of multiple images taken at different angles as the background image. The multiple images taken at different angles can be images taken from different perspectives of the target background. For example, the target background is a dining table. Figure 2a and Figure 2b As shown, two images can be taken from the front and side of the dining table 201, and one of the two images can be selected as the target image. In this way, the user can generate an item layout image based on actual needs, using the target images of different perspectives as the placement background, which can meet the different needs of users.

[0040] In this embodiment, the target object may be an object that the user wants to place. The target object may be one or more. The target object may be an object selected by the user according to the layout scene. For example, Figure 3 As shown, for the arrangement of the office desk, the target items may include a thermos cup 301, headphones 302, and decorations 303. For another example, for the arrangement of the banquet table, the target items may include dinner plates, wine glasses, decorations, and the like.

[0041] In some embodiments, determining the N target objects may further include: acquiring an object image, wherein the object image includes the N target objects; and determining the N target objects based on the object image.

[0042] In this embodiment, the object image may be an image of N target objects collected by the user, where N is a positive integer greater than 1. In a specific implementation, one object image may be acquired, or multiple object images at different angles may be acquired. The object images at different angles may be images obtained by photographing N target objects at different viewing angles. For example, eight object images at different angles may be acquired. Based on multiple object images at different angles, target objects with different placement angles may be provided. In this way, when arranging the target objects, a suitable placement angle may be selected to improve the aesthetics of the object arrangement.

[0043] After step 1100, step 1200 is executed to perform visual saliency prediction on the background image to determine M first marking points in the background image.

[0044] In this embodiment, performing visual saliency prediction on the background image may be determining the visual saliency of different positions in the background image.

[0045] Exemplarily, the visual saliency prediction may be an eye movement prediction. By performing eye movement prediction on the background image, the eye gaze probabilities at different positions in the background image may be determined. Different eye gaze probabilities correspond to different saliencies of the corresponding positions. The higher the probability value of the eye gaze probability at a certain position in the background image, the higher the visual saliency of the position. The lower the probability value of the eye gaze probability at a certain position in the background image, the lower the visual saliency of the position. Based on this, according to the eye gaze probabilities at different positions in the background image, M first marking points may be determined from the background image.

[0046] The first marking point can be used to mark the visual prominence of different areas in the background image. The first marking point can be determined based on the eye fixation probability of different areas in the background image. Exemplarily, the background image is visually predicted to obtain the eye fixation probability of different areas in the background image, and the position with the highest eye fixation probability in each area is marked as the first marking point, that is, M first marking points are obtained.

[0047] It is understandable that there can be multiple first marking points. The number of first marking points can be set according to the number of target objects to be placed. The more objects to be placed, the more first marking points need to be determined. The fewer objects to be placed, the fewer first marking points need to be determined.

[0048] Taking eye movement prediction as an example, the steps for determining the first marker point are described. Figure 4 , the background image 401 can be divided into several grids by using the grid method, wherein each grid corresponds to a partial area in the background image; eye movement prediction is performed on the background image to determine the eye movement fixation probability of each grid; the position point with the highest eye movement fixation probability in each grid is marked to obtain multiple first marking points 402. In addition, the probability values ​​of the eye movement fixation probabilities corresponding to different areas in the background image are different, and the colors of the first marking points can be different, so that it is easy to identify the first marking points with high eye movement fixation probabilities. It should be noted that when dividing the background image by the grid method, the number of grids can be set according to the number of target objects. The more target objects there are, the denser the grids. In addition, in this embodiment, only one method of dividing the background image is listed, and the background image can also be divided in other ways, which is not limited in the embodiments of the present application.

[0049] After step 1200 , step 1300 is performed to select T second marking points from the M first marking points, and connect the T second marking points based on a preset layout graph to obtain a target layout graph.

[0050] In this embodiment, the preset layout graphic may be a layout graphic that conforms to aesthetic theory. Graphics that are generally more regular or have a symmetrical structure are considered to be graphics that conform to aesthetic theory. Exemplarily, the preset layout graphic may be annular graphics of different shapes, such as polygons, isosceles triangles, circles, isosceles trapezoids, etc. Exemplarily, the preset layout graphic may be other regular graphics, such as S-shape, Z-shape, etc. Exemplarily, the target layout graphic may be any preset layout graphic. The target layout graphic may also be a combination of graphics of multiple different shapes, such as Figure 5 As shown, the target layout pattern 501 is a combination of a pentagon and an isosceles triangle.

[0051] In this embodiment, the second marking point may be a marking point that has a higher visual prominence among the first marking points and meets the preset layout requirements. The marking points that meet the preset layout requirements may be marking points that can form a preset layout graph. In other words, the second marking point may be a marking point that has a higher eye movement fixation probability among the first marking points and can be connected to form a preset layout graph.

[0052] It is understandable that the second marking point can be one or more. The number of the second marking points can be set according to the number of target objects to be placed. For example, if the user has 6 target objects to be placed, 6 second marking points are determined from the multiple first marking points.

[0053] In a specific implementation, each of the M first marking points can be used as a vertex of the constituent graph, and the line between any two of the M first marking points can be used as an edge of the constituent graph. Based on this, according to the preset layout graph, T second marking points are selected from the M first marking points, and the T second marking points are connected to obtain the target layout graph.

[0054] Taking the target layout figure as an isosceles triangle as an example, a first marking point is selected from the M first marking points as the starting point, and two first marking points that may be symmetrical among the other first marking points are determined as the other two vertices of the isosceles triangle according to the distance and angle between the starting point and the other first marking points among the M first marking points, and the starting point is connected with the two first marking points that may be symmetrical to obtain an isosceles triangle, that is, the target layout figure. The first marking point located at the vertex of the target layout figure is the second marking point.

[0055] It should be noted here that the starting point can be the first marking point with the highest probability value of eye gaze probability among the M first marking points. The starting point can also be a point with a higher probability value of eye gaze probability among the M first marking points. For example, according to the eye gaze probability of the first marking point, the M first marking points are divided into multiple levels. The higher the eye gaze probability of the first marking point, the higher the level of the first marking point. For example, the first marking points are divided into six levels AF, the first marking points of level A have the highest eye gaze probability, the first marking points of level F have the lowest eye gaze probability, and the first marking points of level A or level B are used as the starting point.

[0056] In this embodiment, due to too many second marking points, the items will be placed too densely, affecting the aesthetics of the item layout. Based on this, T second marking points can be selected from M first marking points according to the preset layout pattern and the first distance, and the T second marking points can be connected to obtain the target layout pattern. The following is an explanation with a specific embodiment.

[0057] In some embodiments of the present application, the step of selecting T second marking points from the M first marking points and connecting the T second marking points based on a preset layout graphic to obtain a target layout graphic may further include: when the number of target items is greater than a first threshold, based on a preset layout graphic and a preset first distance, selecting T second marking points from the M first marking points and connecting the T second marking points to obtain a target layout graphic.

[0058] In this embodiment, the first threshold value can be used to measure whether there are too many target items to be placed. The first threshold value can be, for example, 20. It should be noted that the first threshold value can be set according to actual conditions, and the embodiment of the present application does not limit this.

[0059] In this embodiment, the preset first distance is used to limit the distance between two second marking points that constitute the target layout figure. When the distance between the two second marking points that constitute the target layout figure is less than the first distance, it means that the distance between the two adjacent second marking points is close, which easily causes the items to be placed too densely. When the distance between the two second marking points that constitute the target layout figure is greater than or equal to the first distance, it means that the two adjacent second marking points are separated by a certain distance, which is not easy to cause dense placement of items. Based on this, by setting the first distance, it is possible to avoid the items being arranged too densely, which affects the layout effect. It should be noted that the first distance can be set according to actual conditions, and the embodiment of the present application does not limit this.

[0060] by Figure 6Taking the target layout graph shown in the figure as an example, a first marking point is selected from the M first marking points as the starting point, for example, the first marking point 601 located at the center of the background image is selected as the starting point. Then, the next first marking point that meets the requirements is searched from the other first marking points in the M first marking points, that is, the first marking point whose distance from the starting point is greater than the first distance is selected as the next marking point. Then, according to the preset layout graph, the selected multiple second marking points are connected to obtain the target layout graph. For example, Figure 6 As shown, the five selected second marking points are connected to obtain a target layout graph 602.

[0061] It should be noted here that the starting point can be the first marking point with the highest probability value of eye gaze probability among the M first marking points. The starting point can also be a point with a higher probability value of eye gaze probability among the M first marking points. For example, according to the eye gaze probability of the first marking point, the M first marking points are divided into multiple levels. The higher the eye gaze probability of the first marking point, the higher the level of the first marking point. For example, the first marking points are divided into six levels AF, the first marking points of level A have the highest eye gaze probability, the first marking points of level F have the lowest eye gaze probability, and the first marking points of level A or level B are used as the starting point.

[0062] In this embodiment, when there are too many target objects to be placed, T second marking points are selected from the M first marking points according to the preset layout figure and the preset first distance, and the T second marking points are connected to obtain the target layout figure. In this way, the multiple second marking points constituting the target layout figure are arranged relatively sparsely, which can avoid the target objects being placed too densely, thereby improving the aesthetics of the layout.

[0063] After step 1300, step 1400 is executed to add the N target objects to the background image according to the target layout graphic, and generate and display an object layout image.

[0064] In a specific implementation, after determining the target layout graph, N second marking points are selected from the T second marking points constituting the target layout graph, and based on the N second marking points, the N target objects are added to the background image to generate an object layout image. For example, assuming that there are three target objects and the target layout graph is a pentagon, the three target objects can be added to any three vertices of the pentagon to obtain an object layout image.

[0065] In some embodiments of the present application, adding the N target items to the background image according to the target layout graphic to generate and display an item layout image may further include: determining, in sequence, the target position of each of the N target items in the target layout graphic according to the aesthetic scores of the N target items; adding the N target items to the background image according to the target position corresponding to each of the target items to generate and display an item layout image.

[0066] In this embodiment, the aesthetic score can be a score that measures the aesthetic quality of a target object. The aesthetic score can be determined based on the characteristic information of the target object. The characteristic information of the target object may include shape information, size information, color information, and texture information of the target object. In other words, the aesthetic score of the target object can be determined based on the shape information, size information, color information, and texture information of the target object. For example, colorful flowers have a higher aesthetic score. For another example, handicrafts with unique shapes have a higher aesthetic score than ordinary daily necessities. In this embodiment, the target objects are arranged based on the aesthetic scores of the target objects, and objects with higher scores can be placed in more prominent positions, which is more in line with the user's aesthetic rules and has a better arrangement effect.

[0067] The target position in the target layout graph may be a second marking point corresponding to a vertex of the target layout graph. The target position of each of the N target items in the target layout graph is, that is, a second marking point corresponding to each of the N target items.

[0068] In this embodiment, according to the aesthetic scores of the N target items, the target position of each target item in the target layout graph is determined in sequence. The N target items are sorted according to the aesthetic scores, and the target position of each target item in the target layout graph is determined in sequence according to the aesthetic scores from high to low. Exemplarily, the target item with the highest aesthetic score among the N target items is used as the target item currently being searched, and the second mark point with the highest probability of eye movement fixation is selected from the T second mark points, and the position corresponding to the second mark point is used as the target position of the target item currently being searched. Afterwards, according to the aesthetic score of the target item, the next target item to be searched is selected from the N target items, and the second mark point with the highest probability of eye movement fixation is selected from the remaining T second mark points, and the position corresponding to the second mark point is used as the target position of the next target item to be searched. Repeat the above determination steps until the target position of each target item in the target layout graph is determined. Afterwards, according to the target position corresponding to each target item, the N target items are added to the background image to generate an item layout image. In specific implementation, the background image including the object images of N target objects can be input into the preset model, and each second marking point of the target layout graphic in the background image can be traversed to determine the target position of the target object in the target layout graphic.

[0069] It should be noted here that, when the probability values ​​of the eye fixation probability of at least two second marking points among the T second marking points are the same, the position corresponding to any one of the at least two second marking points with the same probability values ​​of the eye fixation probability is taken as the target position of the target object in the target layout graphic.

[0070] In this embodiment, according to the aesthetic scores of N target objects, the target position of each target object in the target layout graphic is determined in turn, and according to the target position corresponding to each target object, N target objects are added to the background image to generate an object layout image. In this way, combined with the aesthetic scores of the target objects to be placed and the prominence of different positions of the background image, target objects with higher aesthetic scores can be arranged at more prominent positions in the background image, so that the arrangement of the target objects is more in line with the user's observation habits, which can improve the aesthetics of the layout.

[0071] In some embodiments of the present application, determining the target position of each of the N target items in the target layout graphic may further include: determining the eye fixation probability of the target item based on the eye fixation probability of the second marking point and the contrast information between the target item and the background image; determining the target position of the target item in the target layout graphic based on the eye fixation probability of the target item.

[0072] In this embodiment, the probability value of the eye movement fixation probability of the second mark point can reflect the prominence of the position corresponding to the background image and the second mark point. In other words, the eye movement fixation probability of the second mark point can reflect the visual prominence of the position where the target object is to be placed.

[0073] The contrast information between the target object and the background image may include: the contrast information between the target object and the part of the background image where the target object is located, and the contrast information between the target object and the entire background image. It should be noted here that the contrast information between the target object and the background image may be the color contrast between the target object and the background image. The higher the color contrast between the target object and the background image, the greater the color difference between the target object and the background, and the more prominent the target object. The lower the color contrast between the target object and the background image, the smaller the color difference between the target object and the background, and the less prominent the target object.

[0074] The eye movement fixation probability of the target object may reflect the visual prominence of the target object arranged at the target position when the target object is placed at the target position of the background image.

[0075] In this embodiment, the eye gaze probability of the target object is not only related to the aesthetic score of the target object, but also to the background environment in which the target object is located. That is to say, the eye gaze probability of the target object is not only related to the aesthetic score of the target object, but also to the eye gaze probability of the second marking point and the contrast between the target object and the background image. Based on this, the eye gaze probability of the target object can be determined according to the eye gaze probability of the second marking point and the contrast information between the target object and the background image, so that the target position of the target object in the target layout graph can be determined according to the eye gaze probability of the target object.

[0076] In the specific implementation, the N target items are sorted according to the aesthetic score, and the target position of each target item in the target layout graph is determined in order from high to low aesthetic scores. Specifically, the target item with the highest aesthetic score among the N target items is used as the target item currently being searched, and the object image of the target item currently being searched, including the background image of the target layout graph, is input into the preset model to obtain the eye movement fixation probability of the target item currently being searched corresponding to each of the T second marking points, and the second marking point with the highest probability value of the eye movement fixation probability of the target item currently being searched is determined as the target position of the target item currently being searched. Repeat the above determination steps until the target position of each of the N target items in the target layout graph is determined.

[0077] In this embodiment, when the target position of each of the N target objects is determined in turn according to the aesthetic scores of the N target objects, the eye gaze probability of the target object is determined according to the eye gaze probability of the second mark point and the contrast information between the target object and the background image, and the target position of the target object in the target layout graph is determined according to the eye gaze probability of the target object. In this way, on the basis of considering the aesthetic score of the target object, the contrast information of the target object and the surrounding background image is combined to improve the accuracy of the determination, thereby ensuring that the more aesthetically pleasing target objects are arranged in more prominent positions to obtain a better layout effect. In addition, by combining the contrast information of the target object and the surrounding background image for arrangement, each object to be placed can be paid attention to, thereby solving the problem that the existing method of arranging according to the size of the target object is insensitive to smaller objects, and improving the user experience.

[0078] In some embodiments of the present application, adding the N target items to the background image according to the target layout graphic to generate and display an item layout image may further include: when the number of target items is greater than a first threshold, selecting T first items from the N target items according to the aesthetic scores of the N target items; adding the T first items to T second marking points in the background image, and adding the second items to the target area corresponding to each of the T second marking points according to contrast information between the first items and the second items, to generate and display an item layout image; wherein the second items are items other than the T first items among the N target items.

[0079] In this embodiment, the first threshold value can be used to measure whether there are too many target items to be placed. The first threshold value can be, for example, 20. It should be noted that the first threshold value can be set according to actual conditions, and the embodiment of the present application does not limit this.

[0080] The first item may be an item with a higher aesthetic score among the N target items. The number of the first items may be determined according to the number of the second marking points. For example, the N target items are sorted in descending order of aesthetic scores, and the target items with the first T aesthetic scores are taken as the first items. The second item may be other items among the N target items except the T first items.

[0081] The contrast information between the first object and the second object may be the color contrast between the first object and the second object. The higher the color contrast between the first object and the second object, the greater the color difference between the first object and the second object, and the easier it is to notice the second object. The lower the color contrast between the first object and the second object, the smaller the color difference between the first object and the second object, and the harder it is to notice the second object.

[0082] The target area corresponding to the second marking point may be an area near the second marking point. Figure 6 , with the second marking point as the center and the second distance as the radius, the obtained circular area is the target area 603 corresponding to the second marking point.

[0083] In specific implementation, Figure 6 As shown, first, N target items are sorted in order of aesthetic scores from high to low, and 5 first items with higher aesthetic scores are selected from the N target items, and the 5 first items are added to the 5 second marking points. Specifically, the 5 first items can be added to the 5 second marking points according to the rule that the items with higher aesthetic scores are arranged in more prominent positions. After that, according to the aesthetic scores of the second items and the contrast information between the second items and the first items, the remaining second items are added to the target area corresponding to each of the 5 second marking points. Taking the addition of a second item as an example, the contrast information of the second item and the first item at each of the 5 second marking points is obtained, and the target area of ​​the second marking point where the first item with the largest contrast is located is used as the target position of the second item, and the second item is added to the target area of ​​the second marking point. It should be noted that after the target area of ​​the second marking point is arranged, the remaining items can be arranged between the target areas of two second marking points.

[0084] In this embodiment, when there are too many target items to be placed, T second marking points are selected from the M first marking points according to the preset layout figure and the preset first distance, and the T second marking points are connected to obtain the target layout figure, and T first items with higher aesthetic scores are added to the T second marking points in the background image, and the remaining second items are added to the target area corresponding to the second marking points according to the contrast information between the first items and the second items. In this way, the multiple second marking points constituting the target layout figure are arranged more sparsely, which can avoid the target items from being placed too densely and make the layout more reasonable. In addition, it can also avoid the layout picture having a clear sense of fragmentation due to the gathering of items with similar aesthetic scores in the same area, further improving the aesthetics of the layout. In addition, according to the contrast information between the first item and the second item, adding the second item to the background image can highlight the first item with a higher aesthetic score, avoid the layout being too messy, and further improve the layout effect.

[0085] In some embodiments of the present application, after adding the N target items to the background image according to the target layout graphic and generating and displaying the item layout image, the method may further include: generating a first eye movement trajectory according to the eye movement fixation probability of each of the N target items; and generating a visual guide image according to the first eye movement trajectory and the item layout image.

[0086] In this embodiment, the first eye movement trajectory may be an eye movement trajectory obtained according to the order of probability values ​​of the eye movement fixation probability of the target object.

[0087] The visual guidance image can be used to show the user the order and position of the target items. The visual guidance image can be a thermal image. The higher the probability value of the eye movement fixation probability of the target item, the higher the color temperature of the corresponding position in the visual guidance image, and the lower the probability value of the eye movement fixation probability of the target item, the lower the color temperature of the corresponding position in the visual guidance image.

[0088] In a specific implementation, after the object layout image is generated, the point with the highest eye gaze probability of the target object moves along a connected route with a probability from high to low to form a first eye movement trajectory. Afterwards, the first eye movement trajectory is updated from high to low according to the color temperature to form a visual guidance image.

[0089] In this embodiment, a first eye movement trajectory is generated according to the eye movement fixation probability of each target object among N target objects, and a visual guidance image is generated according to the first eye movement trajectory and the object layout image, so as to prompt the user to arrange the objects through the visual guidance image, facilitate user operation, and provide a better user experience.

[0090] In some embodiments of the present application, generating a visual guidance image based on the first eye movement trajectory and the object layout image may further include: when the first eye movement trajectory is different from the second eye movement trajectory, adjusting the first eye movement trajectory according to the second eye movement trajectory to obtain an adjusted first eye movement trajectory; generating a visual guidance image based on the adjusted first eye movement trajectory and the object layout image; wherein the second eye movement trajectory is an eye movement trajectory generated based on the aesthetic scores of the N target objects.

[0091] The second eye movement trajectory is an eye movement trajectory generated according to the aesthetic scores of the N target objects, that is, the eye movement trajectory is generated in the order of the aesthetic scores of the N target objects from high to low.

[0092] In this embodiment, when the first eye movement trajectory is inconsistent with the second eye movement trajectory, the first eye movement trajectory is adjusted according to the second eye movement trajectory to obtain the adjusted first eye movement trajectory; a visual guidance image is generated according to the adjusted first eye movement trajectory and the object layout image, so that the generated visual guidance image is consistent with the order in which the user places the target objects, better meets the requirement of continuity of the user's eye movement trajectory changes, and provides a better user experience.

[0093] In an embodiment of the present application, based on eye tracking technology, visual significance prediction is performed on the background image, and M first marking points in the background image are determined. Then, in combination with the principle of aesthetic composition, T second marking points are selected from the M first marking points, and the T second marking points are connected to obtain a target layout graphic. In this way, according to the target layout graphic, N target objects are added to the background image, and an object layout image that conforms to the observation laws and aesthetic laws of the human eye can be generated. Users arrange objects based on the object layout image, and the layout is more reasonable, thereby solving the problem of poor layout aesthetics in the existing object placement method.

[0094] The patterning method provided in the embodiment of the present application can be executed by a patterning device. In the embodiment of the present application, the patterning method performed by the patterning device is taken as an example to illustrate the patterning device provided in the embodiment of the present application.

[0095] Corresponding to the above embodiment, see Figure 7 The embodiment of the present application further provides a composition device 700 , which includes an acquisition module 701 , a first determination module 702 , a selection module 703 , a first generation module 704 and a display module 705 .

[0096] The acquisition module 701 is used to acquire a background image and determine N target objects;

[0097] A first determination module 702, configured to perform visual saliency prediction on the background image and determine M first marking points in the background image;

[0098] A selection module 703 is used to select T second marking points from the M first marking points, and connect the T second marking points based on a preset layout graph to obtain a target layout graph;

[0099] A first generating module 704 is used to add the N target objects to the background image according to the target layout pattern to generate an object layout image;

[0100] Display module 705, used to display the item layout image;

[0101] Wherein, N, M, and T are all positive integers greater than 1.

[0102] Optionally, the first generation module 704 includes: a first determination unit, used to determine the target position of each of the N target objects in the target layout graphic in turn according to the aesthetic scores of the N target objects; a first generation unit, used to add the N target objects to the background image according to the target position corresponding to each of the target objects, to generate an object layout image.

[0103] Optionally, the first determination unit is specifically used to: determine the eye fixation probability of the target object based on the eye fixation probability of the second marking point and contrast information between the target object and the background image; determine the target position of the target object in the target layout graphic based on the eye fixation probability of the target object.

[0104] Optionally, the device also includes: a second generating module, used to generate a first eye movement trajectory according to the eye movement fixation probability of each of the N target objects; and a third generating module, used to generate a visual guidance image according to the first eye movement trajectory and the object layout image.

[0105] Optionally, the third generating module includes: an adjustment unit, used to adjust the first eye movement trajectory according to the second eye movement trajectory to obtain an adjusted first eye movement trajectory when the first eye movement trajectory is different from the second eye movement trajectory; a second generating unit, used to generate a visual guidance image according to the adjusted first eye movement trajectory and the object layout image; wherein the second eye movement trajectory is an eye movement trajectory generated according to the aesthetic scores of the N target objects.

[0106] Optionally, the selection module 703 is specifically used to select T second marking points from the M first marking points based on a preset layout graphic and a preset first distance to obtain a target layout graphic when the number of target objects is greater than a first threshold, and fit the T second marking points to obtain a target layout graphic; the first generation module includes: a selection unit, used to select T first objects from the N target objects according to the aesthetic scores of the N target objects when the number of target objects is greater than the first threshold; a second generation unit, used to add the T first objects to the T second marking points in the background image, and add the second objects to the target area corresponding to each of the T second marking points according to the contrast information between the first object and the second object, to generate an object layout image; wherein the second objects are other objects among the N target objects except the T first objects.

[0107] In an embodiment of the present application, based on eye tracking technology, visual significance prediction is performed on the background image, and M first marking points in the background image are determined. Then, in combination with the principle of aesthetic composition, T second marking points are selected from the M first marking points, and the T second marking points are connected to obtain a target layout graphic. In this way, according to the target layout graphic, N target objects are added to the background image, and an object layout image that conforms to the observation laws and aesthetic laws of the human eye can be generated. Users arrange objects based on the object layout image, and the layout is more reasonable, thereby solving the problem of poor layout aesthetics in the existing object placement method.

[0108] The patterning device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0109] The patterning device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0110] The patterning device provided in the embodiment of the present application can achieve Figure 1 To avoid repetition, the various processes implemented by the method embodiment are not described here.

[0111] Alternatively, if Figure 8As shown, an embodiment of the present application also provides an electronic device 800, including a processor 801 and a memory 802, wherein the memory 802 stores programs or instructions that can be executed on the processor 801, and when the program or instructions are executed by the processor 801, the various steps of the above-mentioned composition method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, they are not described here.

[0112] Fig. 9 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.

[0113] The electronic device 900 includes but is not limited to: a radio frequency unit 901, a network module 902, an audio output unit 903, an input unit 904, a sensor 905, a display unit 906, a user input unit 807, an interface unit 908, a memory 909, and a processor 910 and other components.

[0114] Those skilled in the art will appreciate that the electronic device 900 may also include a power source (such as a battery) for supplying power to each component, and the power source may be logically connected to the processor 910 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Fig. 9 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0115] Among them, the processor 910 is used to: obtain a background image and determine N target objects; perform visual saliency prediction on the background image to determine M first marking points in the background image; select T second marking points from the M first marking points, and connect the T second marking points based on a preset layout graphic to obtain a target layout graphic; according to the target layout graphic, add the N target objects to the background image to generate an object layout image; the display unit 906 is used to display the object layout image; wherein N, M, and T are all positive integers greater than 1.

[0116] Optionally, when adding the N target items to the background image according to the target layout graphic to generate an item layout image, the processor 910 is used to: determine, in sequence according to the aesthetic scores of the N target items, a target position of each of the N target items in the target layout graphic; and add the N target items to the background image according to the target position corresponding to each of the target items to generate an item layout image.

[0117] Optionally, when determining the target position of each of the N target items in the target layout graphic, the processor 910 is used to: determine the eye fixation probability of the target item based on the eye fixation probability of the second mark point and contrast information between the target item and the background image; and determine the target position of the target item in the target layout graphic based on the eye fixation probability of the target item.

[0118] Optionally, after adding the N target objects to the background image according to the target layout graphic to generate an object layout image, the processor 910 is further used to: generate a first eye movement trajectory according to the eye movement fixation probability of each of the N target objects; and generate a visual guidance image according to the first eye movement trajectory and the object layout image.

[0119] Optionally, when generating a visual guidance image according to the first eye movement trajectory and the object layout image, the processor 910 is used to: when the first eye movement trajectory is different from the second eye movement trajectory, adjust the first eye movement trajectory according to the second eye movement trajectory to obtain an adjusted first eye movement trajectory; generate a visual guidance image according to the adjusted first eye movement trajectory and the object layout image; wherein the second eye movement trajectory is an eye movement trajectory generated according to the aesthetic scores of the N target objects.

[0120] Optionally, when the processor 910 selects T second marking points from the M first marking points and connects the T second marking points based on a preset layout graphic to obtain a target layout graphic, it is used to: when the number of target items is greater than a first threshold, based on the preset layout graphic and a preset first distance, select T second marking points from the M first marking points, and connect the T second marking points to obtain a target layout graphic; Optionally, when the processor 910 adds the N target items to the background image according to the target layout graphic to generate an item layout image, it is used to: when the number of target items is greater than the first threshold, select T first items from the N target items according to the aesthetic scores of the N target items; add the T first items to the T second marking points in the background image, and add the second items to the target area corresponding to each of the T second marking points according to the contrast information between the first items and the second items, to generate and display the item layout image; wherein the second items are items other than the T first items among the N target items.

[0121] In an embodiment of the present application, based on eye tracking technology, visual significance prediction is performed on the background image, and M first marking points in the background image are determined. Then, in combination with the principle of aesthetic composition, T second marking points are selected from the M first marking points, and the T second marking points are connected to obtain a target layout graphic. In this way, according to the target layout graphic, N target objects are added to the background image, and an object layout image that conforms to the observation laws and aesthetic laws of the human eye can be generated. Users arrange objects based on the object layout image, and the layout is more reasonable, thereby solving the problem of poor layout aesthetics in the existing object placement method.

[0122] It should be understood that in the embodiment of the present application, the input unit 904 may include a graphics processor (Graphics Processing Unit, GPU) 9041 and a microphone 9042, and the graphics processor 9041 processes the image data of the static picture or video obtained by the image capture device (such as a camera) in the video capture mode or the image capture mode. The display unit 906 may include a display panel 9061, and the display panel 9061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 907 includes a touch panel 9071 and at least one of other input devices 9072. The touch panel 9071 is also called a touch screen. The touch panel 9071 may include two parts: a touch detection device and a touch controller. Other input devices 9072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0123] The memory 909 can be used to store software programs and various data. The memory 909 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 909 may include a volatile memory or a non-volatile memory, or the memory 909 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 909 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0124] The processor 910 may include one or more processing units; optionally, the processor 910 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 910.

[0125] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned composition method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0126] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0127] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned composition method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0128] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0129] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned composition method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0130] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0131] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0132] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A composition method, characterized in that: The method comprises: Obtain a background image and determine N target objects; Performing visual saliency prediction on the background image to determine M first marked points in the background image; Selecting T second marking points from the M first marking points, and connecting the T second marking points based on a preset layout graph to obtain a target layout graph; According to the target layout graphic, the N target objects are added to the background image to generate and display an object layout image; Wherein, N, M, and T are all positive integers greater than 1; The step of adding the N target objects to the background image according to the target layout graphic to generate and display an object layout image includes: Determining, in sequence, a target position of each of the N target objects in the target layout graph according to the aesthetic scores of the N target objects; According to the target position corresponding to each of the target objects, the N target objects are added to the background image to generate and display an object layout image.

2. The method according to claim 1, characterized in that: The determining of the target position of each of the N target objects in the target layout graph includes: Determining the eye movement fixation probability of the target object according to the eye movement fixation probability of the second marking point and contrast information between the target object and the background image; According to the eye movement fixation probability of the target object, a target position of the target object in the target layout graph is determined.

3. The method according to claim 1, characterized in that After adding the N target objects to the background image according to the target layout graphic and generating and displaying the object layout image, the method further includes: Generate a first eye movement trajectory according to the eye movement fixation probability of each of the N target objects; A visual guidance image is generated according to the first eye movement trajectory and the object layout image.

4. The method according to claim 3, characterized in that: The step of generating a visual guidance image according to the first eye movement trajectory and the object layout image includes: When the first eye movement trajectory is different from the second eye movement trajectory, adjusting the first eye movement trajectory according to the second eye movement trajectory to obtain an adjusted first eye movement trajectory; generating a visual guidance image according to the adjusted first eye movement trajectory and the object layout image; The second eye movement trajectory is an eye movement trajectory generated according to the aesthetic scores of the N target objects.

5. The method according to claim 1, characterized in that The step of selecting T second marking points from the M first marking points and connecting the T second marking points based on a preset layout graph to obtain a target layout graph includes: When the number of target items is greater than the first threshold, based on the preset layout graph and the preset first distance, T second marking points are selected from the M first marking points, and the T second marking points are connected to obtain a target layout graph; The step of adding the N target objects to the background image according to the target layout graphic to generate and display an object layout image includes: When the number of target objects is greater than a first threshold, selecting T first objects from the N target objects according to the aesthetic scores of the N target objects; Adding the T first objects to the T second marking points in the background image, and adding the second objects to the target area corresponding to each of the T second marking points according to the contrast information between the first objects and the second objects, to generate and display an object layout image; The second items are other items among the N target items except the T first items.

6. A patterning device, characterized in that: The device comprises: An acquisition module is used to acquire a background image and determine N target objects; A first determination module, configured to perform visual saliency prediction on the background image and determine M first marking points in the background image; A selection module, configured to select T second marking points from the M first marking points, and connect the T second marking points based on a preset layout graph to obtain a target layout graph; A first generating module, configured to add the N target objects to the background image according to the target layout graphic to generate an object layout image; A display module, used for displaying an item layout image; Wherein, N, M, and T are all positive integers greater than 1; The first generating module comprises: A first determining unit, configured to sequentially determine a target position of each of the N target objects in the target layout graph according to the aesthetic scores of the N target objects; The first generating unit is used to add the N target objects to the background image according to the target position corresponding to each of the target objects to generate an object layout image.

7. The device according to claim 6, characterized in that The device also includes: A second generating module, configured to generate a first eye movement trajectory according to an eye movement fixation probability of each of the N target objects; The third generating module is used to generate a visual guidance image according to the first eye movement trajectory and the object layout image.

8. The device according to claim 6, characterized in that The selection module is specifically configured to select T second marking points from the M first marking points based on a preset layout graph and a preset first distance, and fit the T second marking points to obtain a target layout graph when the number of target objects is greater than a first threshold; The first generating module comprises: a selection unit, configured to select T first objects from the N target objects according to the aesthetic scores of the N target objects when the number of the target objects is greater than a first threshold; A second generating unit is used to add the T first objects to the T second marking points in the background image, and add the second object to a target area corresponding to each of the T second marking points according to contrast information between the first object and the second object, so as to generate an object layout image; The second items are other items among the N target items except the T first items.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the composition method according to any one of claims 1 to 5 are implemented.

Images