Image processing method and device and electronic equipment

By collecting videos from multiple angles and generating 3D models or animations, combining AR technology to display the target object and environment, it solves the problem that merchants find it difficult to obtain high-quality display effects at low cost, and achieves efficient and low-cost store and product display.

CN119007059APending Publication Date: 2024-11-22RAJAX NETWORK &TECHNOLOGY (SHANGHAI) CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202310581581.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-22
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

In the prior art, it is difficult for merchants to obtain high-quality store and product display effects at low technical costs, and they require professional equipment and knowledge, and it is difficult to perceive the product through pictures when matching with the user environment.

Method used

By obtaining the initial video of the target object collected from multiple angles, extracting the video frame to generate the target model, combining the description information to generate a 3D panoramic model or animation, providing low-cost high-quality display effect, and integrating the model with real scenes through AR technology , display the matching of the target object and the environment.

Benefits of technology

It realizes the generation of high-reduction target models without relying on professional equipment and knowledge, providing high-quality store and product display effects, helping merchants to decorate at low cost and improve user experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119007059A_ABST
    Figure CN119007059A_ABST
Patent Text Reader

Abstract

The invention discloses an image processing method and device and electronic equipment. The method comprises the following steps: in response to a first instruction for guiding a merchant to provide an initial video, obtaining the initial video of a target object collected from multiple angles; and in response to a second instruction for generating an object model, extracting at least part of video frames from the initial video, and generating a target model of the target object according to the at least part of video frames. By adopting the method, the problem of providing a target object display effect with relatively high restoration degree at low technical cost is solved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, device, electronic device and storage medium. Background Art

[0002] With the development of online shopping, how to present stores and products on the Internet, as important assets of merchants, is becoming more and more important for merchant operations. Good presentation of stores and products can attract users and improve user retention rate.

[0003] In the prior art, in order to optimize the presentation of stores and goods, it is often necessary to use more professional equipment to take multiple photos from different angles during the operation of store decoration and product display, such as using an automatic rotating platform, a pan / tilt head and other equipment to generate a model of the corresponding store or product. The operator needs to have certain professional knowledge to obtain a usable model. Otherwise, the obtained model will be of poor quality and the effect will be poor. In addition, it is difficult to perceive whether the product matches the user environment simply through the product picture. Therefore, it is difficult for merchants to obtain a good presentation effect of stores and products with low technical costs.

[0004] Therefore, how to provide a display effect with high quality expression and high degree of restoration at a low technical cost is a problem that needs to be solved.

[0005] The above information disclosed in the Background section is only for enhancement of understanding of the background of the present application and therefore it may contain information that does not constitute the prior art that is already known to a person of ordinary skill in the art. Summary of the invention

[0006] The image processing method provided in the embodiment of the present application solves the problem of providing store and product display effects with high quality expression and high degree of restoration at a low technical cost.

[0007] An embodiment of the present application provides an image processing method, comprising: in response to a first instruction for guiding a merchant to provide an initial video, obtaining an initial video of a target object captured from multiple angles; in response to a second instruction for generating an object model, extracting at least a portion of video frames from the initial video, and generating a target model of the target object based on the at least portion of the video frames.

[0008] Optionally, in response to a first instruction for guiding a merchant to provide an initial video, obtaining an initial video of a target object captured from multiple angles includes: displaying first guidance information for guiding a merchant to operate a first terminal to shoot around the target object to obtain the initial video; wherein the first terminal is provided with a camera device.

[0009] Optionally, it also includes: detecting the shooting angle of the camera device to the target object; updating the first guidance information according to the change of the shooting angle to guide the merchant to operate the first terminal to adjust the shooting angle and capture a full-range video around the target object as the initial video.

[0010] Optionally, the first guidance information further includes guidance information for guiding the merchant to capture the initial video from multiple viewing angles including the front, side, top, and top directions of the target object.

[0011] Optionally, extracting at least part of the video frames from the initial video includes: extracting at least part of the video frames from the initial video according to a preset interval frame number or interval duration.

[0012] Optionally, it also includes: obtaining description information provided by the merchant for the initial video; generating a target model of the target object based on at least a portion of the video frames, including: generating the target model based on at least a portion of the video frames and the description information; wherein the target model includes images of the target object from multiple perspectives, and the images from different perspectives are associated with corresponding information in the description information.

[0013] Optionally, generating the target model based on the at least part of the video frame and the description information includes: determining visual effect characteristics of the target object based on the description information of the target object; generating a target model of the target object based on the visual effect characteristics of the target object and the at least part of the video frame, and providing corresponding visual effects after the target model is displayed.

[0014] Optionally, the target model includes at least one form of information including images, text, and voice representing visual effects corresponding to different viewing angles of the target object.

[0015] Optionally, the target object is the store and / or goods of the merchant; the initial video includes a store video collected for the store of the merchant, and / or a product video collected for the product of the merchant; accordingly, the target model includes a store 3D panoramic model generated corresponding to the store video and corresponding description information, and / or a product animated image or product 3D model generated corresponding to the product video and corresponding description information.

[0016] Optionally, the target object is a dish displayed by the merchant to the user; generating the target model based on the at least partial video frame and the description information includes: obtaining one or more dish attributes of the dish based on the description information of the dish; the dish attributes include at least one of dish category attributes, ingredient attributes, taste attributes, and atmosphere attributes; determining the visual effect characteristics of the dish based on the dish attributes; generating the target model of the dish based on the visual effect characteristics of the dish and the at least partial video frame.

[0017] Optionally, it also includes: using the model preliminarily generated according to at least part of the video frames as the first model; synthesizing audio according to the description information, fusing the audio with the first model, and using the audio and video data obtained after the fusion as the target model, and the target model is used for display on the second terminal.

[0018] Optionally, it also includes: sending the target model to the server; when the target object is the product of the merchant, in response to a first page setting instruction, sending first page editing information to the server to indicate that the first page is set using the target model corresponding to the product and the product description information of the product; and / or, when the target object is the store of the merchant, in response to a second page setting instruction, sending second page editing information to the server to indicate that the second page is set using the target model corresponding to the store and the store description information of the store.

[0019] Optionally, it also includes: receiving merchant operation behavior information of the merchant, and performing at least one of the following operations on the target model according to the merchant operation behavior information: rotating, flipping, scaling, displaying corresponding description information corresponding to the current viewing angle, and displaying the image after the operation corresponding to the operation instruction.

[0020] Optionally, extracting at least a portion of the video frames from the initial video and generating a target model of the target object based on the at least a portion of the video frames includes: extracting at least a portion of the video frames from the initial video based on a model intelligent generation algorithm that is built-in or requested from a server, and generating a target model of the target object based on the at least a portion of the video frames; or, sending the initial video to a server, and the server extracting at least a portion of the video frames from the initial video based on a model intelligent generation algorithm, and generating a target model of the target object based on the at least a portion of the video frames.

[0021] An embodiment of the present application also provides an image processing method, applied to a second terminal, including: obtaining a target model of a target object; the target model is generated based on an initial video of the target object captured from multiple angles; in response to a third instruction for displaying the object in a real scene, guiding a user to capture a real scene image of the real scene, and displaying the target model in a fusion with the real scene image.

[0022] Optionally, obtaining the target model of the target object includes: responding to a loading instruction of a first page, loading the first page, wherein the target model included in the first page is a product animated image or a product 3D model; or, responding to a loading instruction of a second page, loading the second page, wherein the target model included in the second page is a 3D panoramic model of a store; or, responding to a download instruction of the target model, downloading the target model from a server according to information of the target object.

[0023] Optionally, the method further includes: displaying one or more objects of a merchant; determining an object having a 3D model as the target object; displaying an operable element for acquiring a corresponding 3D model in association with the target object; acquiring the 3D model after the operable element is triggered, and using the acquired 3D model as the target model.

[0024] Optionally, when the target object is a dish, the target model is a product animated image or a product 3D model of the dish; the method also includes: after obtaining the target model, displaying the target model with any form of visual effects including hot air effect, ice effect, bubble effect, heart-shaped effect, picture switching display effect, motion lens effect, and lighting effect.

[0025] Optionally, guiding the user to capture a real scene image of the real scene and fusing the target model with the real scene image includes: guiding the user to adjust the shooting posture of the camera device of the second terminal to capture the real scene image; taking the perspective of the camera device as the main perspective to display the placement position of the target model in the real scene image; and projecting the target model into the real scene image according to the placement position.

[0026] Optionally, the method further includes: receiving any of the following data of the user adjusting the target model in the display scene image: placement position, orientation, and model size.

[0027] Optionally, it also includes: receiving user operation behavior information on the target model, and performing at least one of the following operations on the target model according to the user operation behavior information: rotating, flipping, scaling, displaying corresponding descriptive information corresponding to the user's current viewing angle, and displaying the image after the operation corresponding to the operation instruction.

[0028] An embodiment of the present application also provides an image processing method, including: a first terminal responds to a first instruction for guiding a merchant to provide an initial video, and obtains an initial video of a target object captured from multiple angles; the first terminal responds to a second instruction for generating an object model, generates a target model of the target object according to the initial video, and sends the target model to a server; the server responds to a model download request for obtaining the target model of the target object sent by a second terminal, and provides the target model to the second terminal; the second terminal responds to a third instruction for displaying an object in a real scene, guides a user to capture a real scene image of the real scene, and displays the target model in a fusion with the real scene image.

[0029] Optionally, it also includes: the server receives the target model sent by the first terminal, and the CDN service node associated with the server pulls the target model from the server and caches it; the server responds to the model download request sent by the second terminal to obtain the target model of the target object, and provides the target model to the second terminal, including: the CDN service node receives the model download request sent by the second terminal, and provides the target model to the second terminal.

[0030] An embodiment of the present application also provides an image processing device, including: a video acquisition unit, used to respond to a first instruction for guiding a merchant to provide an initial video, and obtain an initial video of a target object acquired from multiple angles; a model generation unit, used to respond to a second instruction for generating an object model, and extract at least a portion of the video frames from the initial video, and generate a target model of the target object based on the at least portion of the video frames.

[0031] An embodiment of the present application also provides an image processing device, including: a model downloading unit, used to obtain a target model of a target object; the target model is generated based on an initial video of the target object collected from multiple angles; a model and real scene fusion unit, used to respond to a third instruction for displaying the object in a real scene, guide a user to collect a real scene image of the real scene, and display the target model by fusing it with the real scene image.

[0032] An embodiment of the present application also provides an electronic device, comprising: a memory and a processor; the memory is used to store a computer program, and after the computer program is run by the processor, the method provided in the embodiment of the present application is executed.

[0033] The embodiment of the present application also provides a computer storage medium storing computer execution instructions, which are used to implement the method provided in the embodiment of the present application when executed by a processor.

[0034] Compared with the prior art, this application has the following advantages:

[0035] An image processing method, device, electronic device and storage medium provided in an embodiment of the present application, the method obtains an initial video of a target object captured from multiple angles by responding to a first instruction for guiding a merchant to provide an initial video; in response to a second instruction for generating an object model, extracts at least part of the video frame from the initial video, and generates a target model of the target object based on the at least part of the video frame. Guide merchants to obtain a video of a target object, and the initial video of the target object can be shot from multiple angles without the need for additional professional equipment and more professional shooting knowledge. Generate a target model of the target object based on the video frame of the initial video, so that a target model with a high degree of restoration can be obtained at a low technical cost, which is convenient for presenting the target object with high-quality expression on the terminal used by the user. Furthermore, the target model is generated based on the description information and the video frame, and is included in the product page or the store page, so that merchants can decorate the store and present the product at a low technical cost, and provide users with a good presentation effect of the target object (such as the product and the store).

[0036] An image processing method, device, electronic device and storage medium provided in an embodiment of the present application are applied to a second terminal, by obtaining a target model of a target object; the target model is generated based on the initial video of the target object collected from multiple angles; in response to a third instruction for displaying the object in a real scene, the user is guided to collect a real scene image of the real scene, and the target model is fused with the real scene image for display. The target model is generated based on the video frame of the initial video of the target object, so that merchants can shoot the initial video of the target object from multiple angles without additional professional equipment and more professional shooting knowledge, and obtain a target model with a high degree of restoration at a low technical cost. The target model is fused with the real scene image captured by the second terminal and displayed, which can show the matching effect of the target object and the actual environment, so that the target object is presented with high-quality expression and high restoration on the second terminal, providing users with a good presentation effect of the target object (such as goods and stores).

[0037] The embodiment of the present application provides an image processing method, device, electronic device and storage medium, wherein the first terminal responds to a first instruction for guiding a merchant to provide an initial video, and obtains an initial video of a target object collected from multiple angles; the first terminal responds to a second instruction for generating an object model, generates a target model of the target object according to the initial video, and sends the target model to a server; the target model is a 3D model. The server responds to a model download request for obtaining the target model of the target object sent by the second terminal, and provides the target model to the second terminal; the second terminal responds to a third instruction for displaying the object in a real scene, guides the user to collect a real scene image of the real scene, and displays the target model by fusing it with the real scene image. From guiding the merchant to collect the initial video of the target object, generating the corresponding target model, and transmitting it to the second terminal used by the user to display the effect of the fusion of the model and the actual scene image, a complete target object display solution and display link are provided, which can help merchants to operate and assist users in deciding whether to select the target object for transaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 This is a schematic diagram of an application scenario of the image processing method provided by this application;

[0039] Figure 2 is a processing flow chart of an image processing method provided in the first embodiment of the present application;

[0040] Figure 3 This is a schematic diagram of device interaction of an image processing method provided by the first embodiment of the present application;

[0041] Figure 4 is a processing flow chart of an image processing method provided in the second embodiment of the present application;

[0042] Figure 5 is a processing flow chart of an image processing method provided in the third embodiment of the present application;

[0043] Figure 6 is a schematic diagram of an image processing device provided in the fourth embodiment of the present application;

[0044] Figure 7 is a schematic diagram of an image processing device provided in the fifth embodiment of the present application;

[0045] Figure 8 It is a schematic diagram of an electronic device provided by this application. DETAILED DESCRIPTION

[0046] Many specific details are described in the following description to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present application, so the present application is not limited by the specific implementation disclosed below.

[0047] The embodiments of the present application provide several image processing methods, devices, electronic devices and storage media, which are described one by one in the following embodiments.

[0048] For ease of understanding, an application scenario of the image processing method provided in this application is first given. This scenario is only an embodiment, and the purpose of providing this scenario embodiment is to facilitate understanding of the application of the image processing method provided in this application, and is not used to limit the image processing method of this application.

[0049] Please refer to Figure 1, the application scenario shown in the figure includes: a first terminal 101, a server 102, and a second terminal 103. Among them, the first terminal collects an initial video or multiple photos of the target object, generates a corresponding target model based on the video frame or multiple photos, and uploads the target model to the server. This application does not specifically limit the method of obtaining the initial video or photo. The initial video is a video shot from multiple angles for the target object, and the multiple photos are photos taken from multiple angles for the target object. The first terminal includes a terminal used by a merchant. For example, in one implementation, the merchant's store and / or goods are taken as the target object, and the merchant uses the first terminal to shoot the target object from multiple angles to obtain an initial video, which may specifically include shooting around the target object to obtain the initial video. For another example, in one implementation, the first terminal obtains (locally or online) stored initial videos shot by other camera devices such as cameras around the target object. The first terminal generates a corresponding target model based on the initial video, and the generating of the corresponding target model based on the initial video includes generating a corresponding target model based on the initial video and a model intelligent generation algorithm. The target model can be used to generate the merchant's product page and / or store page. The second terminal obtains the target model, shoots the real scene image, and projects the target model on the real scene image. The specific way in which the second terminal obtains the target model is not limited. For example, the second terminal loads a page generated based on the target model (such as a product page and / or a store page) to obtain the target model. In actual applications, the user uses the second terminal to browse the product page and / or the store page, and the second terminal displays the product page and / or the store page. The target model is then displayed on the second terminal to present it to the user, or the second terminal receives the user operation behavior and obtains the target model associated with these pages according to the user operation behavior. Of course, the second terminal can also download and project the target model into the captured real scene image in an interactive manner through the operable elements of the APP. The second terminal includes a terminal used by the user. Of course, the second terminal can also obtain the target model through other pages or containers, and project the target into the captured real scene image. The first terminal and the second terminal include but are not limited to mobile phones, computers, intelligent interactive devices, etc. Preferably, the first and second terminals are provided with camera devices. The server can be an independent physical server or a cluster composed of multiple physical servers.

[0050] It should be noted that the above disclosed information is only used to help understand the present application and does not mean to constitute the prior art known to ordinary technicians in the field.

[0051] In the image processing method provided in the first embodiment of the present application, the merchant is guided to obtain an initial video of the target object and send it, and the initial video is a video captured from multiple angles of the target object; video frames are extracted from the initial video, and a target model of the target object is generated based on the extracted video frames. Thus, the image material (i.e., the extracted video frame) for synthesizing the target model can be obtained without relying on more professional video shooting equipment such as an automatic rotation platform and a pan / tilt, and a target model associated with the original captured information (i.e., the initial video) can be synthesized based on these image materials, and the target model has a high degree of restoration, so as to quickly generate a 3D model that is relatively close to the real vision, improve the processing efficiency and provide a target model with a high degree of restoration.

[0052] The following combination Figure 2 and Figure 3 The image processing method provided in the first embodiment of the present application is described.

[0053] Figure 2 The image processing method shown comprises:

[0054] Step S201, in response to a first instruction for guiding a merchant to provide an initial video, obtaining an initial video of a target object captured from multiple angles;

[0055] Step S202: In response to a second instruction for generating an object model, extract at least a portion of video frames from the initial video, and generate an object model of the target object according to the at least a portion of the video frames.

[0056] The image processing method provided in this embodiment is executed by the first terminal; or, it is implemented through the interaction between the first terminal and the server, and some steps are executed by the server. For example, the first terminal obtains an initial video of the target object collected from multiple angles; based on the model intelligent generation algorithm built into the first terminal, at least part of the video frames are extracted from the initial video, and the target model of the target object is generated according to the at least part of the video frames, and the generated target model is uploaded to the server. For another example, the first terminal obtains an initial video of the target object collected from multiple angles, and provides the initial video to the server; the server extracts at least part of the video frames from the initial video based on the model intelligent generation algorithm, and generates the target model of the target object based on the at least part of the video frames. For another example, the first terminal obtains an initial video of the target object collected from multiple angles; the first terminal requests the server to obtain the model intelligent generation algorithm, extracts at least part of the video frames from the initial video based on the obtained intelligent generation algorithm, and generates the target model of the target object according to the at least part of the video frames.

[0057] The first terminal includes a terminal used by a merchant. The target object includes a store and / or goods of the merchant, and of course, it can also be other items or services that the merchant hopes to display. The initial video is a video shot for the target object, which can be a video shot using a first terminal equipped with a camera, or a video shot by other camera devices with a camera function such as a camera. Of course, the initial video can also be replaced by multiple photos taken from multiple angles. Subsequently, multiple photos are used to generate a target model, and the information collection method of taking photos is simpler and easier to operate. The multiple angles refer to multiple shooting angles, and the so-called shooting angles include the angles between the camera device that captures the initial video and the target object being photographed, which can be distinguished from different aspects, for example, according to the shooting height, including but not limited to the horizontal direction, the downward direction and the upward direction, according to the shooting direction, including but not limited to the front, side, oblique side, back, etc., according to the shooting distance, including but not limited to the distant view, the medium view, the close view, the close-up, etc. The multiple angles include different shooting angles and / or a combination of various shooting angles.

[0058] As described in step S201, the first terminal guides the merchant to obtain an initial video of the target object, and the initial video is a video collected from multiple angles. The first terminal guides the merchant, including displaying different guidance information to instruct the merchant to obtain the initial video. The so-called guidance information includes at least one of the following forms of guidance information: voice guidance information, text guidance information, and image guidance information. The present application does not limit the specific implementation method of obtaining the initial video. In a preferred implementation, the first terminal displays the first guidance information to guide the merchant to operate the first terminal to shoot the initial video for the target object, or guides the merchant to select the stored initial video in the local storage device of the first terminal, or guides the merchant to download the initial video according to the storage address information of the initial video. The storage address information is the storage location and / or storage file name of the store video and / or product video included in the merchant data stored online. Further, it also includes: if it is detected that the first terminal is equipped with a camera device, the first guidance information includes guidance information for guiding the merchant to operate the first terminal to shoot the initial video for the target object. In one implementation, it also includes displaying an interface element for receiving the merchant to input the storage address information.

[0059] In this step, the video shot from multiple angles includes: a video captured by shooting around the target object, such as a video captured around 360 degrees. When guiding the merchant to operate the first terminal to shoot and obtain the initial video, the first instruction for guiding the merchant to provide the initial video is used to obtain the initial video of the target object captured from multiple angles, specifically including: displaying the first guidance information for guiding the merchant to operate the first terminal to shoot around the target object to obtain the initial video; wherein the first terminal is provided with a camera device. Further, it also includes: detecting the shooting angle of the camera device to the target object; updating the first guidance information according to the change of the shooting angle, so as to guide the merchant to operate the first terminal to adjust the shooting angle, and shoot a full-range video around the target object as the initial video. Preferably, the first guidance information also includes guidance information for guiding the merchant to capture the initial video from multiple perspectives of the front, side, top view, and top view of the target object, so as to obtain a 360-degree panoramic video. The so-called multiple perspectives refer to the perspective of the observer with the lens of the camera device, which can be understood as shooting angles. When the first guidance information is information in the form of an image, the image includes any of the following images: an arrow, a progress bar, and the like. For example, a dynamic arrow and / or text is used to indicate a change in the shooting angle, and the dynamic arrow and / or text is updated as the shooting angle changes. The dynamic arrow is preferably a dynamic surround arrow. For another example, the progress bar can be a progress bar in various forms, which updates the progress as the shooting angle changes. Shooting starts from the first terminal until 360 degrees around the target object is completed, and the initial video is saved. Furthermore, the first guidance information also includes guidance information for guiding the merchant to capture the initial video from multiple perspectives including the front, side, top view, and top view of the target object. Of course, if it is determined that the target object is an object that can display an internal structure, the merchant is prompted to shoot the internal structure, such as in an application scenario where the target object is a refrigerator, a washing machine, or the like.

[0060] In a preferred embodiment, the initial video is a video collected within a preset shooting time, and accordingly, the method specifically includes: displaying second guidance information for guiding the merchant to operate the first terminal to shoot around the target object within the preset shooting time, so as to obtain the initial video within the preset shooting time. For example, guiding the merchant to shoot around the target object within 10 seconds to obtain the initial video.

[0061] As described in step S202, according to the second instruction, a target model of the target object is generated based on the video frames of the initial video. Preferably, the target model is generated by extracting video frames from the initial video according to a preset number of interval frames or interval duration, and using the extracted video frames. For example, for the initial video shot around the target object within 10 seconds, a video frame is extracted every 10 frames, and the extracted video frames are used as materials to generate the corresponding target model. During implementation, the target model generated in this embodiment includes any of the following types of images: 3D model, animated image. The so-called 3D model includes a three-dimensional panoramic image of the target object obtained by splicing video frames extracted from the initial video, wherein each extracted video frame is a photo, and extracting video frames from the initial video can be understood as a group or multiple groups of photos taken from different shooting angles around 360 degrees. Wherein, extracting at least part of the video frames from the initial video includes: extracting at least part of the video frames from the initial video according to a preset number of interval frames or interval duration. The animated image refers to a dynamic picture of the target object generated by extracting video frames from the initial video, such as GIF. Preferably, the target object is the store and / or goods of the merchant; the initial video includes a store video collected for the store of the merchant, and / or a product video collected for the goods of the merchant; accordingly, the target model includes a store 3D panoramic model generated corresponding to the store video and corresponding description information, and / or a product animated image or a product 3D model generated corresponding to the product video and corresponding description information. The so-called 3D panoramic model refers to a model with a stereoscopic visual effect that presents the target object captured by the initial video from the perspective of the surroundings with the camera device as the center. For example, a model of a store. The target model generated by using the initial video shot by the merchant for the goods and / or store is closer to the actual situation of the goods and stores, and can present the goods and / or stores more realistically.

[0062] Further, the target model is generated according to the visual effect characteristics of the target object and the at least part of the video frame, and the target model provides corresponding visual effects to the user after being displayed; wherein the visual effect characteristics of the target object are determined based on the visual effect specified by the merchant or based on the description information provided by the merchant. Preferably, the first terminal performs the following processing: obtaining the description information provided by the merchant for the initial video; the generating the target model of the target object according to the at least part of the video frame includes: generating the target model according to the at least part of the video frame and the description information; wherein the target model includes images of the target object from multiple perspectives, and the images from different perspectives are associated with corresponding information in the description information. wherein, the generating the target model according to the at least part of the video frame and the description information includes: determining the visual effect characteristics of the target object according to the description information of the target object; generating the target model of the target object according to the visual effect characteristics of the target object and the at least part of the video frame, and providing corresponding visual effects after the target model is displayed. further, the target model includes at least one form of information in the form of image, text, and voice that characterizes the visual effects corresponding to different perspectives of the target object. During implementation, it also includes: performing word segmentation processing on the description information, matching each word segment obtained by the word segmentation processing with a reference word representing a visual effect, and using the visual effect associated with the matched reference word as a visual effect feature of the target object.

[0063] Of course, the generated target model can also include voice information. The target model generated in this embodiment includes a story video of the target object. The story video refers to a video containing voice information generated by extracting video frames from the initial video and the extracted video frames and the corresponding description information. This embodiment includes: taking the model initially generated according to at least part of the video frames as the first model; synthesizing audio according to the description information, fusing the audio with the first model, and taking the audio and video data obtained after the fusion as the target model, and the target model is used to display on the second terminal. In practical applications, the first terminal specifically includes the following processing: preliminarily generating the first model of the target object according to at least part of the video frames of the initial video; taking the text obtained based on the description information (such as text) of the initial video as input, and synthesizing the input text into the first audio (waveform) based on the text-to-speech synthesis technology and the deep learning method; synthesizing the first model and the first audio into a story video as the target model. When the target object is a store, the generated story video is a store story video; when the target object is a commodity, the generated story video is a commodity story video.

[0064] One application scenario of the method is to provide a solution for catering merchants to quickly generate a target model of dishes and / or stores close to the merchant at a low technical cost. In this application scenario, the target object is the dish displayed by the merchant to the user; the generating the target model according to the at least part of the video frame and the description information includes: obtaining one or more dish attributes of the dish according to the description information of the dish; the dish attribute includes at least one of the dish category attribute, the ingredient attribute, the taste attribute, and the atmosphere attribute; determining the visual effect characteristics of the dish according to the dish attribute; generating the target model of the dish according to the visual effect characteristics of the dish and the at least part of the video frame. The target model of the dish includes any form of information in images, texts, and voices that characterize the visual effects corresponding to the dish attributes. The visual effects of the target model of the dish include any of the following visual effects: hot air effect, ice effect, bubble effect, heart-shaped effect, picture switching display dynamic effect, motion lens dynamic effect, and lighting dynamic effect. For example, the dish is beef noodles, and the dish description information includes hot dish and spicy taste. The server extracts the dish category attribute as hot dish and the taste attribute as spicy from the dish description information, and determines that the visual effect characteristics of the dish include heat effect and specific effect indicating spiciness. Accordingly, the generated target model is an initial 3D model (i.e., a first model) initially generated by extracting video frames based on the initial video of beef noodles. The initial 3D model is then superimposed with synthetic motion effects to obtain a 3D model with heat effect and spicy information as the target model of beef noodles, and the spicy information is information in at least one form of pictures (such as flames and / or peppers), text, and voice.

[0065] In this embodiment, at least one of the following dynamic image production methods can be used to generate dynamic images: a motion lens dynamic effect production method, a lighting dynamic effect production method, a synthetic dynamic effect production method, and a generated dynamic effect production method. Among them, the motion lens dynamic effect production method includes separating the foreground and background of the picture (such as a video frame) containing the target object, and taking at least one method such as translation, scaling, and rotation to process the foreground to obtain a dynamic image of the dynamic effect of the lens motion shooting. In the lighting dynamic effect production method, edge special effects, sweeping light, etc. are added to the foreground to achieve a dynamic image that highlights the foreground effect. In the synthetic dynamic effect production method, the first model preliminarily generated based on the video frame is superimposed on the corresponding image to synthesize different effects, such as a heart-shaped effect, a bubble effect, an ice effect, a hot air effect, a picture switching display effect, and at least one other effect. In the generated dynamic effect production method, a text generation video model is applied, and text information (such as description information or other text content provided by the merchant) is used as input, and multi-frame rate layered training is used to obtain a high frame rate dynamic effect video. The text-generated video model consists of two modules. The first module generates several frames of images from text to synthesize a low-frame-rate video with a lower frame rate. The second module interpolates several frames of the low-frame-rate video based on the bidirectional attention model to generate a high-frame-rate animated image. An example of a text-generated video model is the CogVideo model. If there is no conflict, the above-mentioned animated image production methods can be combined with each other to generate the corresponding target model.

[0066] In this embodiment, extracting at least part of the video frames from the initial video and generating the target model of the target object according to the at least part of the video frames include: extracting at least part of the video frames from the initial video based on a model intelligent generation algorithm built in or requested from a server, and generating the target model of the target object according to the at least part of the video frames; or, sending the initial video to a server, and the server extracting at least part of the video frames from the initial video based on a model intelligent generation algorithm, and generating the target model of the target object according to the at least part of the video frames. For example, in one embodiment, the first terminal generates the target model based on the initial video. In this embodiment, the first terminal converts the extracted video frames (each frame is equivalent to a photo) into a 3D model of the target object based on a 3D model generation algorithm or function of AI technology such as an open source nerf algorithm and iOS's RealityKit (an open kit that supports the ability to generate 3D models from pictures) as a target model, and can also further generate animated images and / or story videos as the target model. In another implementation, the first terminal provides the initial video to the server, and the server generates the target model based on the initial video. In this implementation, the server is a server of a platform capable of generating multimedia visual capabilities such as 3D models, animated images, and story videos.

[0067] Therefore, this embodiment helps merchants to quickly generate any of the following models without relying on professional equipment by shooting videos: 3D models, animated images, story videos. The cost of generating models with 3D scanners is lower and there is no technical threshold, so it is easy to promote and popularize. Compared with the model made by 3D modeling tools based on art construction, the generated model has a higher correlation with the information of the target object initially collected, such as the initial video, and the target model has a higher degree of restoration of the target object, which can more realistically reflect the actual product, especially for complex models with high modeling difficulty. For example, dishes have many model details and the generation of models is difficult. The method provided by this embodiment can help merchants quickly generate target models of real dishes.

[0068] The target model is provided to the server, provided to the second terminal by the server, and displayed on the second terminal, which can assist the user in deciding whether to select the target object for transaction. In this embodiment, the specific way in which the second terminal obtains the target model is not specifically limited. For example, the target model is used to set the first page and / or the second page, and the second terminal requests to load the first page and / or the second page, thereby displaying the target model; or, the second terminal obtains the target model from the server in an interactive manner and displays it in fusion with the real scene photographed by the second terminal. In a preferred embodiment, the first terminal also performs the following processing to set the first page and / or the second page: sending the target model to the server; when the target object is the commodity of the merchant, in response to the first page setting instruction, sending the first page editing information to the server to indicate that the target model corresponding to the commodity and the commodity description information of the commodity are used to set the first page; and / or, when the target object is the store of the merchant, in response to the second page setting instruction, sending the second page editing information to the server to indicate that the target model corresponding to the store and the store description information of the store are used to set the second page. For example, the first page includes a product page, which includes but is not limited to a product details page, an event venue page, etc. The second page includes a store page, which includes but is not limited to a store basic page, a store list page, a channel advertising page, etc. In another embodiment, the second terminal obtains the target model from the server in an interactive manner and displays it in fusion with the real scene captured by the second terminal, including: the second terminal interacts with the user using the second terminal in all directions through computer technology, instructs the user to capture the real scene image, and projects the target model into the real scene image, so that the user can watch the real scene to restore and display the target model.

[0069] In this embodiment, it also includes: the second terminal projects the target model to the user's current environment through AR technology, so that the user can experience a more concrete target object. The so-called AR (Augmented Reality, referred to as AR), that is, augmented reality technology, realizes the integration of the user's real environment and the object virtual information (such as the target model) by calculating the position and angle of the captured image in real time and superimposing other images, so as to integrate the virtual and the real on the screen and interact. By fusing the target model with the real scene image reflecting the user's real environment, the target object can be presented to the user in a real and three-dimensional manner. Further, since the target model is generated based on the real initial video shot from multiple angles, it can ensure the relevance with the collected original information (i.e., the initial video), achieve sufficient restoration, and truly reflect the target object. Even if the target object is a non-standard commodity (such as a dish) with high modeling difficulty, the model details can be presented. In addition, the collection of original information can be completed using a camera or a mobile phone (first terminal) camera, and the information collection is simple and the cost is low. Specifically, the second terminal obtains a target model of the target object; the target model is generated according to the initial video of the target object collected from multiple angles; in response to a third instruction for displaying the object in a real scene, the user is guided to collect a real scene image of the real scene, and the target model is fused and displayed with the real scene image. Wherein, the acquisition of the target model of the target object includes: the second terminal responds to the loading instruction of the first page to load the first page, and the target model included in the first page is a product animation or a product 3D model; or, the second terminal responds to the loading instruction of the second page to load the second page, and the target model included in the second page is a store 3D panoramic model; or, the second terminal responds to the download instruction of the target model, and downloads the target model from the server according to the information of the target object. Further, the second terminal also includes: displaying one or more objects of the merchant; determining an object with a 3D model as the target object; displaying an operable element for obtaining the corresponding 3D model in association with the target object; obtaining the 3D model after the operable element is triggered, and the obtained 3D model is used as the target model. In the application scenario of the catering industry, when the target object is a dish, the target model is a product animated image or a product 3D model of the dish; the second terminal also includes the execution: after obtaining the target model, the target model is displayed with any form of visual effects including hot air effect, ice effect, bubble effect, heart-shaped effect, picture switching display effect, motion lens effect, and lighting effect.Among them, guiding the user to collect a real scene image of a real scene and fusing the target model with the real scene image includes: guiding the user to adjust the shooting posture of the camera device of the second terminal to collect the real scene image; using the perspective of the camera device as the main perspective to display the placement position of the target model in the real scene image; projecting the target model into the real scene image according to the placement position. Furthermore, it also includes: the second terminal receives the following arbitrary data of the user adjusting the target model in the displayed scene image: placement position, orientation, model size.

[0070] In this embodiment, the user can operate the target model through the second terminal, specifically including: the second terminal receives the user operation behavior information of the user on the target model, and performs at least one of the following operations on the target model according to the user operation behavior information: rotation, flipping, scaling, displaying corresponding descriptive information corresponding to the user's current perspective, and displaying the image after the operation corresponding to the operation instruction.

[0071] In this embodiment, the merchant can operate the target model displayed on the first terminal, specifically including: receiving the merchant operation behavior information of the merchant, and performing at least one of the following operations on the target model according to the merchant operation behavior information: rotation, flipping, scaling, displaying corresponding description information corresponding to the current perspective, and displaying the image after the operation corresponding to the operation instruction. This enables the merchant to preview the target model in an all-round interactive manner, helping the merchant to decide whether to present the target model to the user of the second terminal. The first terminal also includes the following processing: in response to a model preview instruction for previewing the object model, displaying the target model. The first terminal can also receive an operation of the merchant submitting a compliance review for the target model. Specifically, the method includes the following processing: in response to a compliance review instruction for the target model, sending the target model to a compliance device for compliance review to perform a compliance review on the target model. The target model after the compliance review can be used to be displayed on the second terminal.

[0072] The method provided by this embodiment can help merchants shoot multi-angle videos of target objects (such as goods or stores), use AI technology (specifically 3D model generation algorithm) to generate target models, and upload them to the server; it is convenient for users to obtain the target model through the second terminal (such as downloading the model through the client APP), shoot the user's current real scene through the camera, and use AR technology to project the target model into the captured real scene image, so as to realize the fusion of the model and the real scene to display the target object. In this way, the product model can be collected and generated at a low cost, and the model can be displayed through AR to realize the three-dimensional display of the product, providing users with a higher degree of restoration and a better immersive experience.

[0073] Please refer to Figure 3, the figure shows the interaction process between the first and second terminals and the server, including: S301, guiding the merchant using the first terminal to shoot an initial video of the target object from multiple angles. The target object is the merchant's goods and / or stores. The goods can be a single product or a package product. S302, obtaining the description information corresponding to the initial video. For example, guiding the merchant to enter the product description information of the product or package, and / or the store description information of the store. S303, using AI technology (i.e., 3D model generation algorithm) to intelligently generate a target model, including a 3D model, a dynamic image, and a story video. For the product, generate at least one of a 3D model of the product, a dynamic image of the product, and a product story video; for the store, generate at least one of a 3D model of the store, a dynamic image of the store, and a story video of the store. Extract several video frames from the initial video, use the video frames to synthesize the target model, and if the corresponding description information is obtained, generate a product story video and / or a store story video according to the description information. Of course, the visual effect features can also be determined according to the description information, so that the server can process the target model to have the corresponding visual effect. S304, upload the target model to the server. S305, the merchant operates the target model. For example, conduct compliance audit, flip, rotate, zoom, display corresponding description information corresponding to the merchant's current perspective, display the image after the operation corresponding to the operation instruction, etc. S306, generate product pages and / or store pages based on the target model; or, carry the target model in other ways. S307, download the target model. The second terminal requests the server to load the product page and / or store page through the APP, and requests the server to download the target model through these pages; or, the second terminal triggers the download of the target model through the operable elements related to the other methods associated with the APP. If the CDN cache function is used, the second terminal initiates a download request to the CDN service node through the APP, and the CDN service node provides the cached target model to the second terminal; if there is no cached target model, the CDN service node searches step by step until the source station, pulls the target model and provides it to the second terminal. S308, shoot the user's current real scene image. For example, trigger the camera device of the second terminal to start, and guide the user to shoot the image of the actual scene. S309, the target model is projected on the real scene image, and the model obtained by projection and the real scene fusion image are displayed on the second terminal. S310, receiving the user's operation target model. Among them, steps S301 to S304, that is, guiding the merchant to shoot a video and send the target model to the server, can be implemented based on the native image SDK of the first terminal. S302 and S305 do not have to be executed, that is, the merchant may not provide description information or preview the target model, and the visual effect features are determined according to the keywords in the target object identifier or the image recognition result of the target object in the initial video.The execution of S310 is independent of S308 and S309, that is, after downloading the target model, the second terminal can display the target model and receive user operation behavior information to operate the target model, so that the user can browse the target model from different perspectives.

[0074] The image processing method provided in this embodiment can provide a solution to help merchants quickly generate at least one of the target models of 3D models, animated images, and story videos at low or no cost, thereby reducing the merchant's cost of art model construction, allowing merchants to simply and effectively decorate their important assets (stores and goods), and provide high-quality target models for stores and / or goods without professional knowledge reserves and professional equipment, which is helpful for merchants' brand expression and operation. It should be understood that models of other forms different from 3D models, other types of videos different from the story video, and other forms of animated images different from GIF animated images generated based on partial video frames of the initial video are also within the scope of protection of this application.

[0075] It should be noted that, in the absence of conflict, the features given in this embodiment and other embodiments of the present application can be combined with each other, and steps S201 and S202 or similar terms do not limit the steps to be executed sequentially.

[0076] So far, the method provided by this embodiment has been described. The method guides merchants to obtain a video of a target object, and can shoot an initial video of the target object from multiple angles without the need for additional professional equipment and more professional shooting knowledge. A target model of the target object is generated based on the video frame of the initial video, so that a target model with a high degree of restoration can be obtained at a low technical cost, which is convenient for presenting the target object with high-quality expression on the terminal used by the user. Furthermore, the target model is generated based on the description information and the video frame, and is included in the product page or store page, so that merchants can decorate stores and present products at a low technical cost, and provide users with a good presentation effect of the target object (such as products and stores).

[0077] Based on the above embodiment, the second embodiment of the present application provides another image processing method. Figure 4 The method is described below. For the same parts, please refer to the description of the corresponding parts of the above embodiments and will not be repeated here. Figure 4 The image processing method shown is applied to the second terminal, including:

[0078] Step S401, obtaining a target model of a target object; the target model is generated based on initial videos of the target object collected from multiple angles;

[0079] Step S402, in response to a third instruction for displaying an object in a real scene, guide the user to capture a real scene image of the real scene, and display the target model by fusing it with the real scene image.

[0080] In this embodiment, the target model is generated based on the initial video of the target object captured from multiple angles. After the second terminal obtains the target model, it projects the target model to the user's current environment through AR technology. By fusing the target model with the real scene image that reflects the user's real environment, the target object can be presented to the user in a three-dimensional manner, so that the user can experience a more concrete target object to assist the user in deciding whether to select the target object for trading. Since the target model is generated based on the real initial video shot from multiple angles, it can ensure the relevance with the collected original information (i.e., the initial video), achieve sufficient restoration, and truly reflect the target object. Even when the target object is a non-standard commodity (such as a dish) that is difficult to model, the model details can be presented. In addition, the collection of original information can be completed using a camera or a mobile phone (first terminal) camera, and the information collection is simple and the cost is low.

[0081] Preferably, the target object is the store and / or product of the merchant; the initial video includes a store video collected for the store of the merchant, and / or a product video collected for the product of the merchant; accordingly, the target model includes a store story video or a store 3D panoramic model generated corresponding to the store video, and / or a product animated image or a product 3D model generated corresponding to the product video.

[0082] As described in step S401, the second terminal obtains the target model of the target object. Wherein, the obtaining of the target model of the target object includes: responding to the loading instruction of the first page, loading the first page, the target model included in the first page is a product animation or a product 3D model; or, responding to the loading instruction of the second page, loading the second page, the target model included in the second page is a store 3D panoramic model; or, responding to the download instruction of the target model, downloading the target model from the server according to the information of the target object. In this application, the specific manner in which the second terminal obtains the target model is not specifically limited. In one embodiment, after the second terminal loads the first page and / or the second page, the target model is naturally displayed with the display of the page. In one embodiment, after the second terminal loads the first page and / or the second page, the information of the object is displayed, and the object includes the target object; receiving the user trigger to download the target model of the target object, requesting the server to download the target model and display it. In this embodiment, the second terminal includes the following processing: displaying one or more objects of the merchant; determining an object with a 3D model as the target object; displaying an operable element for obtaining the corresponding 3D model in association with the target object; obtaining the 3D model after the operable element is triggered, and the obtained 3D model is used as the target model. In one embodiment, the second terminal triggers a download instruction of the target model through an APP, and initiates a request to the server to download the target model. In one embodiment, the server can be a social application server, and the first terminal and the second terminal communicate through the corresponding social application. The target model generated by the first terminal can be shared with the second terminal through a link or a file. The second terminal downloads the target model through the shared link or downloads the file to obtain the target model.

[0083] Taking the application scenario in the catering industry as an example, when the target object is a dish, the target model is a product animated image or a product 3D model of the dish; the second terminal also includes the execution: after obtaining the target model, the target model is displayed with any form of visual effects including hot air effect, ice effect, bubble effect, heart-shaped effect, picture switching display effect, motion lens effect, and lighting effect.

[0084] In this embodiment, the user can operate the target model through the second terminal, specifically including: receiving user operation behavior information on the target model, and performing at least one of the following operations on the target model according to the user operation behavior information: rotating, flipping, scaling, displaying corresponding descriptive information corresponding to the user's current perspective, and displaying the image after the operation corresponding to the operation instruction.

[0085] As described in step S402, the second terminal guides the user to collect a real scene image of the real scene, and fuses the target model with the real scene image for display. Wherein, guiding the user to collect a real scene image of the real scene, and fusing the target model with the real scene image, includes: guiding the user to adjust the shooting posture of the camera device of the second terminal to collect the real scene image; using the perspective of the camera device as the main perspective to display the placement position of the target model in the real scene image; and projecting the target model into the real scene image according to the placement position.

[0086] Furthermore, it also includes: receiving any of the following data of the user adjusting the target model in the displayed scene image: placement position, orientation, model size. So far, the method provided in this embodiment has been described. The method fuses the target model with the real scene image captured by the second terminal and displays it, which can show the matching effect of the target object and the actual environment, so as to present the target object with high-quality expression and high restoration degree on the second terminal, and provide the user with a good presentation effect of the target object (such as goods and shops). Since the target model is generated based on the video frame of the initial video of the target object, the information collection is simple, and the merchant does not need additional professional equipment and more professional shooting knowledge to shoot the initial video of the target object from multiple angles, thereby obtaining a target model with a high degree of restoration at a low technical cost.

[0087] Based on the above embodiment, the third embodiment of the present application provides another image processing method. Figure 5 The method is described below. For the same parts, please refer to the description of the corresponding parts of the above embodiments and will not be repeated here. Figure 5 The image processing method shown comprises:

[0088] Step S501: The first terminal obtains an initial video of a target object captured from multiple angles in response to a first instruction for guiding a merchant to provide an initial video;

[0089] Step S502, the first terminal generates a target model of the target object according to the initial video in response to the second instruction for generating an object model, and sends the target model to the server;

[0090] Step S503, the server provides the target model to the second terminal in response to the model download request for obtaining the target model of the target object sent by the second terminal;

[0091] Step S504: the second terminal responds to the third instruction for displaying the object in the real scene, guides the user to collect a real scene image of the real scene, and displays the target model by fusing it with the real scene image.

[0092] In this embodiment, the first terminal generates the target model based on the initial video and sends it to the server, specifically including: extracting at least part of the video frame from the initial video; based on the 3D model generation algorithm or function of AI technology, such as the open source nerf algorithm, iOS's RealityKit (an open kit that supports the ability to generate 3D models from pictures), converting the extracted video frames (each frame is equivalent to a photo) to obtain the target model of the target object, the target model includes at least one of a 3D model, a dynamic image, and a story video. The second terminal obtains the target model from the server, and projects the target model to the user's current environment through AR technology, so that the user can experience a more concrete target object. Among them, the fusion display of the target model and the real scene image includes projecting the target model into the real scene image in an interactive manner with the user, and displaying it on the screen of the second terminal, for example: the second terminal receives the following data of the user adjusting the target model in the display scene image: placement position, orientation, model size.

[0093] In addition, since the target model is generated based on the original video shot from multiple angles, it can ensure the relevance to the original information (i.e., the original video) collected, achieve sufficient restoration, and truly reflect the target object. Even if the target object is a non-standard product (such as a dish) that is difficult to model, the model details can be presented. In addition, the original information can be collected using a camera or a mobile phone (first terminal) camera, which is simple to collect information and has low cost.

[0094] In this embodiment, when the CDN cache function is used, the method further includes: the server receives the target model sent by the first terminal, and the CDN service node associated with the server pulls the target model from the server and caches it; the server responds to the model download request sent by the second terminal to obtain the target model of the target object, and provides the target model to the second terminal, including: the CDN service node receives the model download request sent by the second terminal, and provides the target model to the second terminal. This allows the second terminal to quickly download the target model.

[0095] So far, the method provided in this embodiment has been explained. The method includes guiding the merchant to collect the initial video of the target object through the first terminal, generating the corresponding target model, and transmitting it to the second terminal used by the user to display the effect of the fusion of the model and the actual scene image. A complete target object display solution and display link are provided, which can help merchants to operate and assist users in deciding whether to select the target object for transaction.

[0096] Corresponding to the first embodiment, the fourth embodiment of the present application provides an image processing device. For the relevant parts, please refer to the description of the corresponding method embodiment. Figure 6 , the image processing device shown in the figure comprises:

[0097] The video acquisition unit 601 is used to acquire an initial video of a target object acquired from multiple angles in response to a first instruction for guiding a merchant to provide an initial video;

[0098] The model generating unit 602 is configured to extract at least a portion of video frames from the initial video in response to a second instruction for generating an object model, and to generate a target model of the target object according to the at least a portion of the video frames.

[0099] Optionally, the video acquisition unit 601 is specifically used to: display first guidance information for guiding the merchant to operate the first terminal to shoot around the target object to obtain the initial video; wherein the first terminal is provided with a camera device.

[0100] Optionally, the video acquisition unit 601 is specifically used to: detect the shooting angle of the camera device to the target object; update the first guidance information according to the change of the shooting angle to guide the merchant to operate the first terminal to adjust the shooting angle, and capture a full-range video around the target object as the initial video.

[0101] Optionally, the first guidance information further includes guidance information for guiding the merchant to capture the initial video from multiple viewing angles including the front, side, top, and top directions of the target object.

[0102] Optionally, the model generation unit 602 is specifically configured to extract at least part of the video frames from the initial video according to a preset interval frame number or interval duration.

[0103] Optionally, the model generation unit 602 is specifically used to obtain description information provided by the merchant for the initial video; generate the target model based on at least a portion of the video frame and the description information; wherein the target model includes images of the target object from multiple perspectives, and the images from different perspectives are associated with corresponding information in the description information.

[0104] Optionally, the model generation unit 602 is specifically used to determine the visual effect characteristics of the target object based on the description information of the target object; generate a target model of the target object based on the visual effect characteristics of the target object and at least part of the video frame, and provide corresponding visual effects after the target model is displayed.

[0105] Optionally, the target model includes at least one form of information including images, text, and voice representing visual effects corresponding to different viewing angles of the target object.

[0106] Optionally, the target object is the store and / or goods of the merchant; the initial video includes a store video collected for the store of the merchant, and / or a product video collected for the product of the merchant; accordingly, the target model includes a store 3D panoramic model generated corresponding to the store video and corresponding description information, and / or a product animated image or product 3D model generated corresponding to the product video and corresponding description information.

[0107] Optionally, the target object is a dish displayed by the merchant to the user; the model generation unit 602 is specifically used to obtain one or more dish attributes of the dish based on the description information of the dish; the dish attributes include at least one of dish category attributes, ingredient attributes, taste attributes, and atmosphere attributes; determine the visual effect characteristics of the dish based on the dish attributes; and generate a target model of the dish based on the visual effect characteristics of the dish and at least part of the video frame.

[0108] Optionally, the model generation unit 602 is specifically used to use the model preliminarily generated according to at least part of the video frames as the first model; synthesize audio according to the description information, fuse the audio with the first model, and use the audio and video data obtained after the fusion as the target model, and the target model is used to be displayed on the second terminal.

[0109] Optionally, the device also includes a configuration unit, which is used to: send the target model to the server; when the target object is a product of the merchant, in response to a first page setting instruction, send first page editing information to the server to indicate that the first page is set using the target model corresponding to the product and the product description information of the product; and / or, when the target object is a store of the merchant, in response to a second page setting instruction, send second page editing information to the server to indicate that the second page is set using the target model corresponding to the store and the store description information of the store.

[0110] Optionally, the model generation unit 602 is specifically used to: receive merchant operation behavior information of the merchant, and perform at least one of the following operations on the target model according to the merchant operation behavior information: rotate, flip, scale, display corresponding description information corresponding to the current viewing angle, and display the image after the operation corresponding to the operation instruction.

[0111] Optionally, the model generation unit 602 is specifically used to: extract at least part of the video frames from the initial video based on a built-in model intelligent generation algorithm or a model intelligent generation algorithm requested from a server, and generate a target model of the target object based on the at least part of the video frames; or, send the initial video to the server, and the server extracts at least part of the video frames from the initial video based on a model intelligent generation algorithm, and generates a target model of the target object based on the at least part of the video frames.

[0112] Corresponding to the second embodiment, the fifth embodiment of the present application provides another image processing device. For the relevant parts, please refer to the description of the corresponding method embodiment. Figure 7 , the image processing device shown in the figure comprises:

[0113] The model downloading unit 701 is used to obtain a target model of a target object; the target model is generated according to initial videos of the target object collected from multiple angles;

[0114] The model and real scene fusion unit 702 is used to respond to the third instruction for displaying the object in the real scene, guide the user to collect the real scene image of the real scene, and display the target model by fusing it with the real scene image.

[0115] Optionally, the model downloading unit 701 is specifically used to: load the first page in response to a loading instruction of the first page, wherein the target model included in the first page is a product animated image or a product 3D model; or, load the second page in response to a loading instruction of the second page, wherein the target model included in the second page is a 3D panoramic model of the store; or, download the target model from the server according to the information of the target object in response to a download instruction of the target model.

[0116] Optionally, the model downloading unit 701 is specifically used to: display one or more objects of the merchant; determine an object with a 3D model as the target object; associate an operable element for obtaining the corresponding 3D model with the target object; obtain the 3D model after the operable element is triggered, and use the obtained 3D model as the target model.

[0117] Optionally, when the target object is a dish, the target model is a product animated image or a product 3D model of the dish; the model downloading unit 701 is specifically used to: after obtaining the target model, display the target model with any form of visual effects such as hot steam effect, ice effect, bubble effect, heart-shaped effect, picture switching display effect, motion lens effect, and lighting effect.

[0118] Optionally, the model and real scene fusion unit 702 is specifically used to: guide the user to adjust the shooting posture of the camera device of the second terminal to capture the real scene image; use the perspective of the camera device as the main perspective to display the placement position of the target model in the real scene image; project the target model into the real scene image according to the placement position.

[0119] Optionally, the model and real scene fusion unit 702 is specifically used to: receive any of the following data of the user adjusting the target model in the display scene image: placement position, orientation, and model size.

[0120] Optionally, the model downloading unit 701 is specifically used to: receive user operation behavior information on the target model, and perform at least one of the following operations on the target model according to the user operation behavior information: rotate, flip, scale, display corresponding description information corresponding to the user's current viewing angle, and display the image after the operation corresponding to the operation instruction.

[0121] Based on the above embodiments, the sixth embodiment of the present application provides an electronic device. For the relevant parts, please refer to the corresponding description of the above embodiments. Figure 8 The electronic device shown in the figure includes: a memory 801 and a processor 802; the memory is used to store a computer program, and after the computer program is run by the processor, the method provided in the embodiment of the present application is executed.

[0122] Based on the above embodiments, the seventh embodiment of the present application provides a computer storage medium. For the relevant parts, please refer to the corresponding description of the above embodiments. Figure 8 The memory in the figure can be understood as the storage medium. The computer storage medium stores computer execution instructions, which are used to implement the method provided in the embodiment of the present application when executed by the processor.

[0123] In a typical configuration, an electronic device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory. The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0124] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0125] Although the present application is disclosed as above in the form of a preferred embodiment, it is not intended to limit the present application. Any technical personnel in this field may make possible changes and modifications without departing from the spirit and scope of the present application. Therefore, the scope of protection of the present application shall be based on the scope defined by the claims of the present application.

Claims

1. An image processing method, characterized in that: include: In response to a first instruction for guiding a merchant to provide an initial video, obtaining an initial video of a target object captured from multiple angles; In response to a second instruction for generating an object model, at least a portion of video frames are extracted from the initial video, and an object model of the target object is generated according to the at least a portion of the video frames.

2. The method according to claim 1, characterized in that The step of obtaining, in response to a first instruction for guiding a merchant to provide an initial video, an initial video of a target object captured from multiple angles includes: The first guiding information for guiding the merchant to operate the first terminal to shoot around the target object is displayed to obtain the initial video; wherein the first terminal is provided with a camera device.

3. The method according to claim 2, characterized in that Also includes: Detecting a shooting angle of the camera device to the target object; The first guidance information is updated according to the change of the shooting angle to guide the merchant to operate the first terminal to adjust the shooting angle and capture an all-round video around the target object as the initial video.

4. The method according to claim 2, characterized in that: The first guidance information also includes guidance information for guiding the merchant to collect the initial video from multiple viewing angles of the target object, including the front, side, top, and top directions.

5. The method according to claim 1, characterized in that: The extracting at least part of the video frames from the initial video comprises: The at least part of the video frames are extracted from the initial video according to a preset interval frame number or interval duration.

6. The method according to claim 1, characterized in that Also includes: Obtaining description information provided by the merchant for the initial video; The step of generating a target model of the target object according to at least part of the video frames comprises: The target model is generated according to at least part of the video frame and the description information; wherein the target model includes images of the target object from multiple perspectives, and the images from different perspectives are associated with corresponding information in the description information.

7. The method according to claim 6, characterized in that The generating the target model according to the at least part of the video frame and the description information comprises: Determining the visual effect characteristics of the target object according to the description information of the target object; A target model of the target object is generated according to the visual effect features of the target object and at least part of the video frame, and the target model provides a corresponding visual effect after being displayed.

8. The method according to claim 7, characterized in that The target model includes at least one form of information including images, text, and voice that represents the visual effects corresponding to different viewing angles of the target object.

9. An image processing method, characterized in that: Applied to the second terminal, including: Acquire a target model of a target object; the target model is generated based on initial videos of the target object collected from multiple angles; In response to a third instruction for displaying an object in a real scene, the user is guided to capture a real scene image of the real scene, and the target model is fused with the real scene image for display.

10. An image processing method, characterized in that: include: The first terminal obtains the initial video of the target object captured from multiple angles in response to the first instruction for guiding the merchant to provide the initial video; The first terminal generates a target model of the target object according to the initial video in response to a second instruction for generating an object model, and sends the target model to the server; The server provides the target model to the second terminal in response to a model download request for acquiring the target model of the target object sent by the second terminal; The second terminal responds to the third instruction for displaying the object in the real scene, guides the user to collect a real scene image of the real scene, and displays the target model by fusing it with the real scene image.

Citation Information

Patent Citations

  • Information generation method and device, computer equipment and storage medium

    CN113141498A

  • Image processing method and device, equipment, storage medium and program product

    CN113793255A

  • Marketing poster generation method and electronic equipment

    CN115271808A

  • Method and device for generating fused reality scene

    CN115984156A