Simulation camera imaging method and device, equipment and medium

Abstract

The invention relates to a simulation camera imaging method and device, equipment and medium in the field of image processing, and the method comprises the steps: responding to a simulation camera starting event, and displaying a simulation camera operation interface which comprises a shooting scene preview window, the shooting control and the shooting parameter adjusting control are arranged in a control area on the periphery of the shooting scene preview window; according to a customized shooting parameter set correspondingly limited by the shooting parameter adjustment control, rendering each image frame in a video stream captured by a shooting unit, and forming a preview video stream to be displayed in the shot scene preview window; and in response to an imaging control event corresponding to the operation of the shooting control, extracting at least one image frame from the preview video stream as shooting imaging data. According to the invention, real camera shooting can be simulated and the same imaging effect can be obtained.

Description

Technical Field

[0001] This application relates to the field of image processing, and in particular, to a method for simulating camera imaging, a corresponding device, a computer device, and a computer-readable storage medium. Background Art

[0002] With the continuous development of technology, users' diverse demands for shooting effects have become increasingly prominent. On the one hand, users expect to retain the shooting convenience of smart devices such as mobile phones and tablets. On the other hand, they are eager to obtain shooting imaging effects that are approximately similar to those of using a real camera to a certain extent. For this reason, in traditional technologies, it usually focuses on filter design, and the shooting imaging effects of a real camera are approximated by applying filters to photos. However, filters are fixed after all and cannot adapt to the diverse demands of users in actual shooting scenarios.

[0003] In view of the deficiencies of traditional technologies, the applicant has made corresponding explorations. Summary of the Invention

[0004] The primary objective of this application is to solve at least one of the above problems and provide a method for simulating camera imaging, a corresponding device, a computer device, and a computer-readable storage medium.

[0005] To meet the various objectives of this application, the following technical solutions are adopted:

[0006] A method for simulating camera imaging provided to meet one of the objectives of this application includes the following steps:

[0007] Respond to a simulated camera startup event and display a simulated camera operation interface, where the simulated camera operation interface includes a preview window for the shooting scene and shooting control controls and shooting parameter adjustment controls arranged in a control area surrounding the preview window for the shooting scene;

[0008] Render each image frame in the video stream captured by the imaging unit according to a customized shooting parameter set defined by the shooting parameter adjustment control, and form a preview video stream to be displayed in the preview window for the shooting scene;

[0009] Respond to an imaging control event corresponding to operating the shooting control control, and extract at least one image frame from the preview video stream as shooting imaging data.

[0010] On the other hand, a simulated camera imaging device provided to meet one of the purposes of the present application includes an event response module, a preview display module, and a shooting imaging module. Among them, the event response module is used to respond to the simulated camera startup event and display the simulated camera operation interface. The simulated camera operation interface includes a shooting scene preview window, and shooting control controls and shooting parameter adjustment controls arranged in the control area around the shooting scene preview window; the preview display module is used to render each image frame in the video stream captured by the camera unit according to the customized shooting parameter set defined corresponding to the shooting parameter adjustment control, and form a preview video stream to be displayed in the shooting scene preview window; the shooting imaging module is used to respond to the imaging control event corresponding to operating the shooting control control, and extract at least one image frame from the preview video stream as shooting imaging data.

[0011] On the other hand, a computer device provided to meet one of the purposes of the present application includes a central processing unit and a memory. The central processing unit is used to call and run a computer program stored in the memory to execute the steps of the simulated camera imaging method described in the present application.

[0012] On the other hand, a computer-readable storage medium provided to meet another purpose of the present application stores a computer program implemented according to the simulated camera imaging method in the form of computer-readable instructions. When the computer program is called and run by a computer, it executes the steps included in the method.

[0013] The technical solution of the present application has many advantages, including but not limited to the following aspects:

[0014] When the present application starts a simulated camera for a user on their terminal device, it provides a simulated camera operation interface for the user to operate the simulated camera for personalized shooting operations. And based on the corresponding customized shooting parameter set, it renders the video stream captured by the camera unit in the device into a preview video stream and displays it in the interface for the user to view the corresponding shooting imaging effect in real time, so that the user can obtain a what-you-see-is-what-you-get shooting experience. Furthermore, when the user makes a shooting imaging operation, it retrieves the corresponding shooting imaging data from the preview video stream. It can be seen that it meets the user's experience of simulating the use of a real camera for shooting, can perform personalized shooting in a real shooting scene, and can obtain imaging effects equivalent to a certain extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The above and / or additional aspects and advantages of the present application will become apparent and easy to understand from the following description of the embodiments in conjunction with the drawings, where:

[0016] Figure 1 It is a schematic flowchart of a typical embodiment of the simulated camera imaging method of the present application;

[0017] Figure 2 A schematic diagram of a simulated camera operation interface for illustrative purposes of this application;

[0018] Figure 3 A schematic diagram of a skin details interface for illustrative purposes of this application;

[0019] Figure 4 A schematic diagram of a skin effect display diagram for illustrative purposes of this application;

[0020] Figure 5 A principle block diagram of the simulated camera imaging device of this application;

[0021] Figure 6 A structural schematic diagram of a computer device adopted by this application. Detailed implementation manners

[0022] The embodiments of the present application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application and should not be construed as a limitation of the present application.

[0023] Those skilled in the art of this technology can understand that, unless specifically stated otherwise, the singular forms "a", "an", "the" and "said" used herein may also include the plural forms. It should be further understood that the term "including" used in the specification of the present application means the presence of the described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or their groups. It should be understood that when we say that an element is "connected" or "coupled" to another element, it can be directly connected or coupled to other elements, or there may also be intermediate elements. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The phrase "and / or" used herein includes all or any unit and all combinations of one or more related listed items.

[0024] Those skilled in the art of this technology can understand that, unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meaning as the general understanding of those of ordinary skill in the art to which this application belongs. It should also be understood that terms such as those defined in a general dictionary should be understood to have a meaning consistent with the meaning in the context of the prior art and will not be interpreted with an idealized or overly formal meaning unless specifically defined as here.

[0025] Those skilled in the art can understand that the "client", "terminal", and "terminal device" used herein include both devices with wireless signal receivers that only have the ability to receive and no ability to transmit, and devices with both receiving and transmitting hardware that can perform two-way communication on a two-way communication link. Such devices can include: cellular or other communication devices such as personal computers, tablets, etc., which have a single-line display or a multi-line display or a cellular or other communication device without a multi-line display; PCS (Personal Communications Service), which can combine voice, data processing, fax, and / or data communication capabilities; PDA (Personal Digital Assistant), which can include a radio frequency receiver, pager, Internet / intranet access, web browser, notepad, calendar, and / or GPS (Global Positioning System) receiver; conventional laptop and / or palm computers or other devices, which are conventional laptop and / or palm computers or other devices with and / or including a radio frequency receiver. The "client", "terminal", and "terminal device" used herein can be portable, transportable, installed in a vehicle (air, sea, and / or land), or suitable for and / or configured to run locally, and / or run in a distributed form at any other location on the earth and / or in space. The "client", "terminal", and "terminal device" used herein can also be a communication terminal, an Internet access terminal, a music / video playback terminal, such as a PDA, MID (Mobile Internet Device), and / or a mobile phone with music / video playback function, or can also be devices such as a smart TV, a set-top box, etc.

[0026] The hardware referred to by names such as "server", "client", and "service node" in this application is essentially an electronic device with the equivalent capabilities of a personal computer, and is a hardware device with the necessary components revealed by the von Neumann principle, including a central processing unit (including an arithmetic unit and a controller), a memory, an input device, and an output device. The computer program is stored in its memory, and the central processing unit loads the program stored in the external memory into the memory for execution, executes the instructions in the program, and interacts with the input / output devices to complete specific functions.

[0027] It should be noted that the concept of "server" referred to in this application can similarly be extended to the case applicable to a server cluster. According to the network deployment principle understood by those skilled in the art, the servers should be logically divided. Physically, these servers can either be independent of each other but can be invoked through an interface, or be integrated into a physical computer or a set of computer clusters. Those skilled in the art should understand this flexibility and should not be restricted by this when implementing the network deployment method of this application.

[0028] One or several technical features of this application, unless expressly specified, can either be deployed on the server and accessed by the client remotely invoking the online service interface provided by the server, or be directly deployed and run on the client for access.

[0029] The neural network models cited or possibly cited in this application, unless expressly specified, can either be deployed on a remote server and remotely invoked by the client, or be directly invoked on the client with sufficient device capabilities. In some embodiments, when it runs on the client, its corresponding intelligence can be obtained through transfer learning to reduce the requirements for the client's hardware operation resources and avoid excessive occupation of the client's hardware operation resources.

[0030] All kinds of data involved in this application, unless expressly specified, can either be remotely stored on the server or stored on the local terminal device, as long as it is suitable for being invoked by the technical solution of this application.

[0031] Those skilled in the art should be aware that although the various methods of this application are described based on the same concept and thus show commonality with each other, unless otherwise specified, these methods can be executed independently. Similarly, for the various embodiments disclosed in this application, they are all proposed based on the same inventive concept. Therefore, for the same expressed concepts, as well as concepts that are only appropriately transformed for convenience although the concept expressions are different, they should be equivalently understood.

[0032] For the various embodiments to be disclosed in this application, unless expressly pointed out that there is a mutually exclusive relationship between them, the relevant technical features involved in each embodiment can be cross-combined to flexibly construct new embodiments, as long as this combination does not deviate from the creative spirit of this application and can meet the requirements in the prior art or solve certain deficiencies in the prior art. Those skilled in the art should be aware of this flexibility.

[0033] A method for simulating camera imaging according to the present application can be programmed as a computer program product and implemented by running on a client or a server. For example, in an exemplary application scenario of the present application, it can be implemented by deploying on the server of an e-commerce platform. Thereby, by accessing the interface opened after the computer program product runs, human-computer interaction can be performed with the process of the computer program product through a graphical user interface to execute the method.

[0034] Please refer to Figure 1 , in a typical embodiment of the method for simulating camera imaging according to the present application, the following steps are included:

[0035] Step S1100: Respond to the event of starting the simulated camera, and display the operation interface of the simulated camera. The operation interface of the simulated camera includes a preview window for the shooting scene, and shooting control controls and shooting parameter adjustment controls arranged in the control area surrounding the preview window of the shooting scene.

[0036] The present application provides a user with an experience of simulating the shooting and imaging of a real camera on a terminal device, and builds a corresponding application program running on the terminal device. When the terminal device is controlled by the user to run the application program, in response to the event of starting the simulated camera, after rendering the operation interface of the simulated camera on the terminal device, it is displayed on the screen. The simulated real camera can be a digital camera, a disposable imaging camera, a film camera, etc.

[0037] An exemplary simulated camera operation interface is as Figure 2 shown. In this interface, it includes the shooting mirror preview window 200, which can be used to preview the image in real time before shooting an image; the shooting control control 201, which can be used to control the operation of shooting an image, and can be any one or any combination of operations such as shutter, continuous shooting, video recording, etc.; the shooting parameter control can be used to configure various shooting parameters, and can be any one or any combination of exposure 202, focal length 203, saturation 204, etc.

[0038] Step S1200: Render each image frame in the video stream captured by the imaging unit according to the customized shooting parameter set defined by the shooting parameter adjustment control, and form a preview video stream to be displayed in the preview window of the shooting scene;

[0039] The shooting parameter adjustment control can configure various shooting parameters correspondingly, and thus the configured corresponding shooting parameters can form a customized shooting parameter set. The configuration can be realized by the user according to his own shooting needs, or can be realized in advance according to shooting experience and / or shooting practice. In one embodiment, the configurable range of saturation is [0, 2]. When the saturation is 0, the colors in the corresponding image will be completely gray; when the saturation is 1, the colors in the corresponding image will remain the original colors; when the saturation is closer to 2 starting from 1, the colors in the corresponding image will be more vivid and intense. The configurable range of the focal length is [0.5x, 4x]. When the focal length is in the range of [0.5x, 1x), the ultra-wide-angle lens in the camera unit configured in the terminal device is enabled, providing a wide field of view and a strong perspective effect, which is more suitable for shooting large scenes and creating a sense of space and visual impact; when the focal length is in the range of [1x, 2x), the wide-angle lens in the camera unit configured in the terminal device is enabled, expanding the shooting range and enhancing the depth of field of the picture, which is more suitable for shooting more scenes in a limited space and highlighting the foreground; when the focal length is in the range of [2x, 4x), the telephoto lens in the camera unit configured in the terminal device is enabled, bringing the distant subject closer, compressing the spatial perspective, highlighting the subject and blurring the background, which is more suitable for shooting distant details and hierarchical pictures. In addition, the focal length can be configured finely and quickly by scrolling Figure 2 the focal length in Figure 2 .

[0040] In a further embodiment, the following steps are included: Step S1210, for each image frame in the video stream captured by the camera unit, convert the image frame into texture data in the image rendering environment, and configure the shader according to the customized shooting parameter set;

[0041] The frame extraction algorithm can be used to extract each image frame from the video stream captured by the camera unit. The frame extraction algorithm can be FFmpeg, MediaCodec combined with MediaExtractor, Android-VideoToImages, etc. For each of the image frames, the OpenGL ES 3.0 algorithm can be used to create an image rendering environment, in which texture processing is performed on the image frame. First, the RGB values of each pixel point in the image frame are sampled, and then the gray formula edited based on the perceptual differences of the human eye for different colors: Gray = xR + yG + zB is used to calculate the gray value of each of the pixel points, constituting grayscale image data. The glGenTextures function is used to create a texture object and set the corresponding texture parameters, and the glTexImage2D function is used to upload the image data to the texture object to obtain texture data. The x, y, and z are the weights of R, G, and B respectively, which can be set by those skilled in the art according to prior knowledge or experimental data. In one embodiment, the x, y, and z are 0.299, 0.587, and 0.114 respectively. In addition, according to the customized shooting parameter set, the fragment shader is configured, and the adjustment functions corresponding to various shooting parameters in the set are edited in its code, which can be implemented based on the linear interpolation algorithm.

[0042] Step S1220: Call the shader to adjust the texture data into target texture data, submit it to the rendering pipeline, and obtain a preview image frame;

[0043] Call the vertex shader to act on the texture data first, pass the resulting data to the fragment shader for further processing, calculate the corresponding color value of each pixel, and combine texture mapping to generate the final output color value, and then hand it over to the rendering pipeline for rendering to obtain a preview image frame.

[0044] Step S1230: Compose the preview video stream from each of the preview image frames.

[0045] The preview image frames are spliced in the order from first to last according to the time sequence to obtain a preview video stream, and the preview video stream is played in the shooting scene preview window.

[0046] Step S1300: Respond to the imaging control event corresponding to operating the shooting control control, and extract at least one image frame from the preview video stream as shooting imaging data.

[0047] It is not difficult to understand that the user can operate the shooting control control, and can choose to take corresponding images in the shooting mode of single-shot or continuous shooting or video recording. Thus, when the user touches the control, the imaging control event is triggered. In one embodiment, if the user adopts the single-shot mode: at the moment when the user touches the shooting control control, the current image frame is extracted from the preview video stream as the shooting imaging data; if the user adopts the continuous shooting mode: at the moment when the user presses the shooting control control, first the current image frame is extracted from the preview video stream, and then the next image frame is extracted every once in a while until a specified number of image frames are obtained to form the shooting imaging data, and the specified number and the interval time can be set by the user himself; if the user adopts the video recording mode: after the user touches the shooting control control and touches the shooting control control again, the respective image frames within the time period between the two touches are extracted from the preview video stream, and they are spliced into an output video stream in the order from the first to the last according to the time sequence as the shooting imaging data.

[0048] Based on the above embodiments, it is not difficult to understand that compared with the prior art, the present application has many advantages, including at least:

[0049] When the present application starts a simulated camera on the user's terminal device, it provides a simulated camera operation interface for the user to operate the simulated camera for personalized shooting operations, and based on the corresponding customized shooting parameter set, renders the video stream captured by the camera unit in the device into a preview video stream and displays it in the interface for the user to view the corresponding shooting imaging effect in real time, so that the user can obtain a shooting experience of what you see is what you get. Furthermore, when the user makes a shooting imaging operation, the corresponding shooting imaging data is retrieved from the preview video stream. It can be seen that it meets the user's experience of simulating the use of a real camera for shooting, conducts personalized shooting in a real shooting scenario, and obtains an imaging effect equivalent to a certain extent.

[0050] In other embodiments, after step S1200, the step of displaying the simulated camera operation interface, the following steps are included:

[0051] Step S2200: Obtain each image frame in the video stream captured by the camera unit at the first resolution, adjust the customized shooting parameter set corresponding to the shooting parameter adjustment control, and render each image frame to form a preview video stream and display it in the shooting scene preview window;

[0052] It can be understood that, in order to avoid poor viewing experiences such as frame drops and lags in the playback of the subsequently constructed preview video stream, the camera unit configured in the user's terminal device can be controlled to perform the capture at a preset first resolution when capturing the video stream. Thus, the resolution of each captured image frame is the first resolution. For each of the image frames, the Openg l ES 3.0 algorithm can be used to perform rendering according to the customized shooting parameter set, and the corresponding preview image frames are rendered to form a preview video stream and played in the shooting scene preview window. The first resolution can be specifically set in terms of numerical value by those skilled in the art according to the purpose of constructing a smoothly played preview video stream disclosed herein and the actual business requirements, for example, 1280*720.

[0053] Step S2300, in response to the imaging control event corresponding to the operation of the shooting control control, obtain at least one image frame captured by the camera unit at a second resolution for rendering to obtain shooting imaging data, where the second resolution is greater than the first resolution.

[0054] It can be understood that, in order to ensure a good viewing experience for the user in the subsequent shooting imaging, when the user touches the shooting control control, the imaging control event is triggered in response. In one embodiment, if the user adopts the single-shot mode: at the moment when the user touches the shooting control control, obtain the current image frame captured by the camera unit at the second resolution, and use the Openg l ES 3.0 algorithm to perform rendering on this image frame according to the customized shooting parameter set, and use the rendered image frame as the shooting imaging data; if the user adopts the continuous shooting mode: at the moment when the user touches the shooting control control, obtain the current image frame captured by the camera unit at the second resolution, and then obtain the next image frame at regular intervals until a specified number of image frames are obtained. Then, for each of the image frames, use the Openg l ES 3.0 algorithm to perform rendering on this image frame according to the customized shooting parameter set, and use the corresponding rendered image frames to form the shooting imaging data. The specified number and the interval time can be set by the user himself; if the user adopts the video recording mode: after the user touches the shooting control control and touches the shooting control control again, obtain each image frame within the time interval between the two touches captured by the camera unit at the second resolution. For each of the image frames, the Openg l ES 3.0 algorithm can be used to perform rendering according to the customized shooting parameter set, and the corresponding rendered image frames are formed into an output video stream as the shooting imaging data. The second resolution can be specifically set in terms of numerical value by those skilled in the art according to the purpose of constructing a beautiful and delicate shooting imaging disclosed herein and the actual business requirements, for example, 1920*1080.

[0055] In other embodiments, image processing controls may be pre-set in the simulated camera operation interface. The user touches the image processing controls to select an image or video from the gallery stored in the user's terminal device according to their own needs. Then, the application uses the Openg l ES 3.0 algorithm to render each image frame in the selected image or video according to the customized shooting parameter set, and will render the corresponding image or video and display it to the user. The user can choose to store the rendered image or video in the gallery or not store them.

[0056] In a further embodiment, after step S1100, the step of displaying the simulated camera operation interface, the following steps are included:

[0057] Step S2100, in response to a simulated camera skin change event, obtain the target customized skin template pointed to by the event from the customized skin library. The target customized skin template includes the camera coating style, its supporting control styles, and the supporting control layout.

[0058] The user can touch Figure 2 the skin change control 205 in it to change the current skin of the simulated camera operation interface to the skin they want. For this purpose, when the user touches this control, a skin details interface is displayed on top of the layer of the simulated camera operation interface, as Figure 3 shown.

[0059] In one embodiment, the customized skin library includes multiple customized skin templates, their corresponding designed user identifiers, skin effect display diagrams of the simulated camera skin, various user evaluations, various user evaluation behavior values, and preferred evaluations. Each customized skin template includes the corresponding camera coating style, its supporting control styles, and the supporting control layout. The skin effect display diagram is a screenshot of the simulated camera operation interface after the style is changed according to the corresponding customized skin template. The user evaluation is a text evaluation made by the user using the application on the simulated camera skin after using the simulated camera operation interface with the changed style and / or viewing the interface effect display diagram. The preferred evaluation is the user evaluation selected from various user evaluations. The user evaluation behavior value corresponds to the cumulative number of times of a single user behavior, and the user behavior can be any one of like, favorite, share, etc. The camera coating style includes the painted image on the camera operation shell surface. The supporting control styles include the trigger icons and un-triggered icons corresponding to each control on the operation surface of the camera that match the camera coating style, as well as the sizes of the icons. The supporting control layout includes the position information of each control on the operation surface of the camera on the simulated camera operation interface.

[0060] The user's terminal device can initiate a request to the server that provides services for the application to obtain the skin effect display diagrams of each simulated camera skin in the customized skin library, each user evaluation, each user evaluation behavior value, and the preferred evaluation, and construct and display the skin detail diagram, as Figure 3 shown. This diagram contains options for each optional simulated camera skin, and preferentially displays the options for the simulated camera skin designed by the user himself / herself. Secondly, it displays the options for any one or more of the evaluation behaviors of the user's like, collection, and sharing. Furthermore, it displays the options for the simulated camera skin designed by users with relatively better evaluations. The user can slide left and right in the skin detail diagram to view each of the options. In addition, the user can touch the preview skin effect 300 in the option to display the corresponding skin effect display diagram above the layer of the skin detail diagram, as Figure 4 shown. In one embodiment, when the user selects an option in the skin detail diagram, the border of this option is displayed as the selected orange color. For example, when selecting an option 301 for a simulated camera skin in Figure 3 , the border of the unselected option is white. Those skilled in the art can flexibly evaluate the pros and cons of each simulated camera skin, so as to preferentially display the simulated camera skin with relatively better evaluations. For example, the one with a relatively larger number in any one of the like count, forward count, collection count, and good review count is relatively better, or the one with better comprehensive performance in any combination of multiple items is better.

[0061] When the user selects an option in the skin detail diagram according to his / her own needs, the corresponding customized skin template of this option is used as the target customized skin template, and the trigger response simulated camera skin change event will be triggered to obtain this target customized skin template from the customized skin library.

[0062] Step S2110: Refresh and color the simulated camera operation interface accordingly according to the camera coating style, and adjust the shooting control control and shooting parameter adjustment control in this interface according to the supporting control style and supporting control layout, and display the obtained simulated camera operation interface with the changed style.

[0063] In one embodiment, first, clear all the controls in the simulated camera operation interface and restore the initial camera operation shell surface of the camera in this interface. The initial camera operation shell surface has not been dyed or colored at all and maintains the camera operation shell surface when leaving the factory. Then, superimpose the colored image in the camera coating style in the target customized skin template on the layer of the initial camera operation shell surface, and, superimpose the untriggered icons corresponding to each control in the supporting control style in this template on the layer of the superimposed camera operation shell surface according to the position information corresponding to each control in the supporting control layout in this template. Thus, the simulated camera operation interface with the changed style is obtained.

[0064] In this embodiment, a one - click skin - changing function for the simulated camera operation interface is disclosed. Using this function, users can quickly and conveniently change the skin of the simulated camera operation interface according to their preferences and needs, enhancing the user experience and the personalization level of the interface. At the same time, the display of user evaluations and user behavior values provides a reference for other users, helping them choose skins that better meet their needs.

[0065] In a further embodiment, before step S2100, the step of responding to the simulated camera skin - changing event, the following steps are included:

[0066] Step S2000: Respond to the camera skin customization event, display the simulated camera editing interface, and convert the simulated camera operation interface into a canvas object that can be edited and display it in the canvas of the simulated camera editing interface;

[0067] Users can customize the skin of the simulated camera they want through the skin customization control 206 Figure 2 in it. For this purpose, when the user touches this control, it triggers the response to the camera skin customization event, displays the simulated camera editing interface including the canvas and the editing tool area. In the canvas of this interface, the converted editable canvas object for the simulated camera operation interface is displayed, and the camera operation shell surface therein is initialized and reset to the initial camera operation shell surface.

[0068] Step S2010: Respond to the editing operation triggered in the editing tool area of the simulated camera editing interface and acting on the canvas object, and display the edited canvas object on the canvas;

[0069] Users can, by using the corresponding tools in the editing tool area, after selecting the canvas object in the canvas, zoom in or out on the canvas object as needed for easy editing; in addition, after selecting any control in the canvas object, they can zoom in, zoom out, and move the control within the canvas object area to the desired position as needed. They can also provide various trigger icons and un - triggered icons corresponding to the control in the editing tool area and select a required trigger icon and un - triggered icon to replace the current trigger icon and un - triggered icon of the control; furthermore, in the canvas object, they can select the initial camera operation shell surface in the canvas object, select the image superimposed on its layer. Further, users can also move, zoom in, and zoom out the image so that the superimposed effect meets their own needs, and after the user confirms the completion of the superposition, crop the part of the image that exceeds the size of the simulated camera editing interface. The superimposed image can be an image stored in the user's terminal device uploaded in real - time, or the user can use the shooting unit in the terminal device to take a real - time photo and upload the obtained image.

[0070] It can be understood that when the user uses the corresponding tool in the editing tool area to perform any of the above editing operations on the canvas object, the editing operation is responded to, and the corresponding edited canvas object is displayed on the canvas for the user to check the corresponding editing effect.

[0071] Those skilled in the art can flexibly implement this interface according to the disclosure of the above-mentioned simulated camera editing interface.

[0072] Step S2020: Respond to the editing completion event, determine the corresponding camera coating style, its supporting control style, and supporting control layout according to the canvas object pointed to by this event, use them as a customized skin template, and store it in the customized skin library.

[0073] It can be understood that a submission control can be preset in the simulation editing interface to submit the corresponding customized skin template to cloud storage. For this reason, when the user finishes editing, touch the submission control to trigger the response to the editing completion event, obtain the canvas object submitted by the user, extract the position information of each control in the simulation camera operation interface in this interface from this canvas object to form the supporting control layout; extract the image superimposed on the initial camera operation shell surface in the simulation camera operation interface from this canvas object as the camera coating style; extract the trigger icons and untriggered icons corresponding to each control in the simulation camera operation interface from this canvas object to form the supporting control style. Combine the camera coating style, its supporting control style, and supporting control layout to form a customized skin template, associate it with the unique identifier of this user as the designed user identifier, and store it in the customized skin library. Further, after configuring the simulation camera operation interface according to this customized skin template, obtain the simulation camera operation interface with the style changed, take a screenshot of it, and associate the obtained skin effect display image with this customized skin template and store it in the customized skin library. The unique identifier of the user uniquely refers to this user to distinguish it from other users.

[0074] It can be understood that the user can see the simulated camera skins designed by himself and other users in the skin details interface, and can also perform text evaluations, likes, collections, sharing, selection, and preview of skin effects on each simulated camera skin.

[0075] In this embodiment, the customization function of the simulated camera skin is disclosed, which enhances the freedom of the user's independent design of the simulated camera operation interface. It also realizes the sharing and dissemination of the user's creations by storing the customized skin template in the cloud and associating it with the user's unique identifier, enriching the content of the customized skin library. At the same time, the user can perform interactive operations such as evaluating, liking, collecting, and sharing the skins designed by other users, further improving the user participation and community activity.

[0076] In a further embodiment, step S2010, the step of responding to an editing operation triggered on the canvas object in the editing tool area of the simulation camera editing interface and displaying the edited canvas object on the canvas includes the following steps:

[0077] Step S2011, use the selection tool in the editing tool area of the simulation camera editing interface to select the camera coating in the canvas object as the editing object, and determine the target camera coating style according to the coloring image acting on the editing object;

[0078] The user can use the selection tool in the editing workspace of the simulation camera editing interface to select the camera coating in the canvas object, that is, the current camera operation shell surface of the camera in this interface, as the editing object. Furthermore, the user can upload a coloring image for painting this camera coating in the editing workspace. Thus, first restore the initial camera operation shell surface of the camera in this interface, superimpose this coloring image on the layer of the initial camera operation shell surface, and display the superimposed effect to the user. Further, the user can also move, zoom in, and zoom out this image so that the superimposed effect meets their needs. Moreover, after the user confirms that the superimposition is completed, crop the part of the image that exceeds the size of the simulation camera editing interface, and use the obtained coloring image as the target camera coating style.

[0079] Step S2012, replace the camera coating styles in each customized skin template in the customized skin library with the target camera coating style to obtain the corresponding customized skin templates to be evaluated;

[0080] Obtain each customized skin template in the customized skin library, replace the camera coating style in each customized skin template with the target camera layer style, and use the obtained customized skin templates as the customized skin templates to be evaluated respectively for subsequent evaluation of the corresponding simulation camera skins.

[0081] Step S2013, perform corresponding style changes on the simulation camera operation interface according to each of the customized skin templates to be evaluated to obtain the corresponding skin effect display diagrams after the changes;

[0082] It can be understood that each of the customized skin templates to be evaluated can correspondingly adjust the style and layout of the controls in any simulation camera operation interface and the corresponding initial camera operation shell surface according to this template. In this way, the effect images corresponding to the adjusted simulation camera skins can be used as the skin effect display diagrams.

[0083] Step S2014, use a preset design evaluation model to determine the skin evaluation information corresponding to each skin effect display diagram;

[0084] The design evaluation model has pre-learned the ability to determine the skin evaluation information of the skin effect diagram.

[0085] Input each skin effect display image into the design evaluation model. The image feature extraction layer in the model extracts the image features in the skin effect display image and outputs the corresponding image feature vector. The multi-classifier in the model performs multi-classification mapping on the image feature vector to determine the positive class probability of each classification as the evaluation score corresponding to a single dimension. The obtained rating scores of each dimension constitute the skin evaluation information. The dimension can be any one or any combination of shooting operation experience, overall coordination degree, overall color matching beauty degree, and overall pattern beauty degree.

[0086] Step S2015: Recommend the skin effect display image to the user who triggers the camera skin customization event according to the skin evaluation information;

[0087] For each skin evaluation information, the weighted sum can be obtained according to the evaluation scores of each dimension in the skin evaluation information to obtain a comprehensive score. Then, sort the skin effect display images in descending order according to the comprehensive score, and display the skin effect display images under the sorting to the user who triggers the camera skin customization event, so that the user can quickly select the skin effect display image with better evaluation.

[0088] The weights corresponding to the evaluation scores of each dimension can be set by those skilled in the art as needed.

[0089] Step S2016: Update the canvas object accordingly according to the custom skin template corresponding to the selected skin effect display image by the user, and then display it on the canvas.

[0090] First, determine the supporting control styles and supporting control layouts in the custom skin template corresponding to the selected skin effect display image by the user. Then, clear all the controls in the canvas object, and overlay the untriggered icons corresponding to each control in the supporting control styles on the layer of the superimposed camera operation shell surface according to the position information corresponding to each control in the supporting control layout. The updated canvas object obtained in this way is displayed on the canvas to show the user the effect of the simulated camera skin corresponding to the selected custom skin template.

[0091] In this embodiment, based on the ability of the user to independently design the camera coating style and real-time match and recommend the corresponding custom skin to the user for previewing the corresponding skin effect, the user can quickly make a decision on the desired skin without having to design the position and style of the controls from scratch, which greatly improves the user's participation and autonomy in the simulation camera skin customization, can provide the user with more diverse choices, and significantly improves the user experience and the flexibility of the simulation camera skin customization.

[0092] In a further embodiment, before step S2014, the step of determining the skin evaluation information corresponding to each skin effect display diagram by using a preset design evaluation model, the following steps are included:

[0093] Step S3000: Obtain a preset training set, where the training set includes multiple training samples and their supervision labels. The training samples are skin effect display diagrams, and the supervision labels are the skin evaluation information of the skin effect display diagrams.

[0094] In one embodiment, the operators of the application program can be users with administrator status, create multiple customized skin templates and their skin effect display diagrams, and store them in the customized skin library. Thus, each skin effect display diagram in the customized skin library is respectively used as a training sample in advance, and the skin evaluation information of each skin effect display diagram is marked, corresponding to the supervision label of each training sample. Combine each training sample and its supervision label to form a training set. It can be understood that all users of the application program can be opened for evaluation respectively, and then these evaluations are integrated, and the obtained comprehensive evaluation information is used as the skin evaluation information for the marking.

[0095] The skin evaluation information includes scores corresponding to multiple dimensions of the simulated camera skin shown in the skin effect display diagram. The multiple dimensions include shooting operation experience, overall coordination degree, overall color matching beauty degree, and overall pattern beauty degree. It is not difficult to understand that users can rate higher scores according to the performance of each dimension, and the better the performance, the higher the rating.

[0096] Step S3010: Use the training set to train a design evaluation model so that it learns to determine the skin evaluation information of the simulated camera operation interface.

[0097] The structure of the design evaluation model is an image feature extraction layer followed by a multi-classifier. The image feature extraction layer is used to vectorize and represent the features of the input image. It is recommended to select the Resnet model, and other image processing models such as EfficientNet, DenseNet, MobileNet, LeNet, ViT, VGG, etc. can also be used. The multi-classifier is suitable for multi-classification tasks, and the selection can be MLP (feedforward neural network) or FC (fully connected layer).

[0098] Obtain a single training sample and its supervision label in the training set. Extract the image features of the training sample by the image feature extraction layer in the design evaluation model, and output its vectorized representation, i.e., the image feature vector. Perform multi-class classification mapping on the image feature vector by the multi-classifier in the model, and output the positive class classification probabilities corresponding to each class as the prediction scores corresponding to a single dimension. Construct a prediction result from the obtained prediction scores. Invoke a preset cross-entropy loss function, and calculate the cross-entropy loss value corresponding to the prediction result according to the supervision label of the training sample. When the cross-entropy loss values reach a preset threshold, it indicates that the design evaluation model has been trained to a convergence state, and thus the training of the design evaluation model can be terminated; when the cross-entropy loss value does not reach the preset threshold, it indicates that the design evaluation model has not converged. Then, perform gradient update on the model according to the cross-entropy loss value, usually by backpropagation to correct the weight parameters of each link of the model to make the model closer to convergence. Then, continue to invoke other training samples and their supervision labels to perform iterative training on the design evaluation model until the model is trained to a convergence state. The preset threshold can be preset by those skilled in the art according to the disclosure here as needed.

[0099] In this embodiment, the training process of the design evaluation model is disclosed, so that the design evaluation model trained to a convergence state can accurately and efficiently determine the skin evaluation information of the skin effect diagram.

[0100] In a further embodiment, in step S1100, the step of displaying the simulated camera operation interface includes the following steps:

[0101] Step S1110: Obtain the center of gravity parameters of the device including the camera unit, match the center of gravity parameters to play the simulated camera movement animation, and display the full image of the simulated camera at the tilt angle corresponding to the center of gravity parameters in the animation;

[0102] In addition to the camera unit in the user's terminal device, an IMU (Inertial Measurement Unit) is usually built in. The IMU can measure the acceleration, angular velocity and direction of the device, so as to calculate the center of gravity position and attitude of the device. Thus, the center of gravity parameters measured by the inertial measurement unit can be read.

[0103] A simulated camera movement animation can be pre-constructed, which is realized by 3D modeling. A physical camera simulated in this application can be displayed in the animation. The tilt angle of the camera can be controlled by the read center of gravity parameters, so that the tilt of the camera is consistent with the tilt of the terminal device, and thus the shooting viewfinder of the camera is also consistent with the shooting viewfinder of the camera unit in the terminal device. In addition, the size of the camera displayed in the animation can be freely controlled by the user, and the style of the operation table side of the camera is the same as the style corresponding to the simulated camera operation interface.

[0104] Step S1120: When the center of gravity parameter of the device remains unchanged within a preset time period, stop playing the simulated camera movement animation, and gradually fade from the overall image finally displayed in the animation to display the simulated camera operation interface.

[0105] It is not difficult to understand that when the center of gravity parameter of the device remains unchanged within a predetermined time, it means that the device is in a stationary state at this time. Usually, the user has determined the shooting angle and is ready to take the corresponding scene. Accordingly, stop playing the camera movement animation, and gradually enlarge the image of the operation console side in the camera in the overall image finally displayed in the animation until the simulated camera operation interface is finally presented, so as to present a gradually changing transition effect.

[0106] In this embodiment, the process of a user using a terminal device to simulate the shooting of a physical camera is disclosed, enabling the user to obtain a visual feedback extremely similar to that of a real camera when operating the simulated camera, significantly enhancing the intuitiveness and immersion of the simulated camera operation.

[0107] Please refer to Figure 5 , a simulated camera imaging device provided to meet one of the purposes of the present application, which is a functional embodiment of the simulated camera imaging method of the present application. The device includes an event response module 1100, a preview display module 1200, and a shooting imaging module 1300. Among them, the event response module 1100 is used to respond to the simulated camera startup event and display the simulated camera operation interface. The simulated camera operation interface includes a shooting scene preview window, and shooting control controls and shooting parameter adjustment controls arranged in the control area surrounding the shooting scene preview window; the preview display module 1200 is used to render each image frame in the video stream captured by the imaging unit according to the customized shooting parameter set defined by the shooting parameter adjustment controls, and form a preview video stream to be displayed in the shooting scene preview window; the shooting imaging module 1300 is used to respond to the imaging control event corresponding to the operation of the shooting control control, and extract at least one image frame from the preview video stream as shooting imaging data.

[0108] In a further embodiment, the event response module 1100 includes: a first event response sub-module, configured to respond to a camera skin customization event, display a simulated camera editing interface, and convert the simulated camera operation interface into a canvas object that can be edited, and display it in the canvas of the simulated camera editing interface; a canvas display sub-module, configured to respond to an editing operation triggered in the editing tool area of the simulated camera editing interface and acting on the canvas object, and display the edited canvas object in the canvas; a second event response sub-module, configured to respond to an editing completion event, determine a corresponding camera coating style, its supporting control style, and supporting control layout according to the canvas object pointed to by the event, as a customized skin template, and store it in the customized skin library.

[0109] In a further embodiment, the canvas display sub-module includes: a coating determination unit, configured to use the selection tool in the editing tool area of the simulated camera editing interface to select the camera coating in the canvas object as the editing object, and determine the target camera coating style according to the coloring image acting on the editing object; a template construction unit, configured to replace the camera coating style in each customized skin template in the customized skin library with the target camera coating style to obtain corresponding to-be-evaluated customized skin templates; a rendering unit, configured to perform corresponding style changes on the simulated camera operation interface according to each to-be-evaluated customized skin template to obtain corresponding skin effect display diagrams after change; a beauty evaluation unit, configured to use a preset design evaluation model to determine the skin evaluation information corresponding to each skin effect display diagram; a user recommendation unit, configured to recommend the skin effect display diagrams to the user who triggers the camera skin customization event according to the skin evaluation information; a canvas display unit, configured to update the canvas object according to the customized skin template corresponding to the user-selected skin effect display diagram and display it in the canvas.

[0110] In a further embodiment, the beauty evaluation unit includes: a training set acquisition sub-unit, configured to acquire a preset training set, the training set including a plurality of training samples and their supervision labels, the training samples being skin effect display diagrams, and the supervision labels being the skin evaluation information of the skin effect display diagrams; a model training sub-unit, configured to train a design evaluation model using the training set so that it learns to determine the skin evaluation information of the simulated camera operation interface.

[0111] In a further embodiment, the event response module 1100 includes: an animation playback unit configured to obtain the center of gravity parameters of a device including a camera unit, and play an animation simulating camera movement by matching the center of gravity parameters, and display the entire image of the simulated camera at a tilt angle corresponding to the center of gravity parameters in the animation; an interface display unit configured to end the playback of the animation simulating camera movement and gradually transition from the finally displayed entire image of the animation to display the operation interface of the simulated camera when the center of gravity parameters of the device remain unchanged within a preset duration.

[0112] In a further embodiment, the preview display module 1200 includes: a shader configuration unit configured to convert each image frame in the video stream captured by the camera unit into texture data in an image rendering environment and configure a shader according to the customized shooting parameter set; a preview frame determination unit configured to call the shader to adjust the texture data into target texture data, submit it to a rendering pipeline, and obtain a preview image frame; a streaming frame assembly unit configured to form a preview video stream from each of the preview image frames.

[0113] To solve the above technical problems, an embodiment of the present application further provides a computer device. As Figure 6 shown, it is a schematic internal structure diagram of the computer device. The computer device includes a processor, a computer-readable storage medium, a memory, and a network interface connected through a system bus. Among them, the computer-readable storage medium of the computer device stores an operating system, a database, and computer-readable instructions. Control information sequences may be stored in the database. When the computer-readable instructions are executed by the processor, the processor can implement a method for simulating camera imaging. The processor of the computer device is used to provide computing and control capabilities to support the operation of the entire computer device. Computer-readable instructions may be stored in the memory of the computer device. When the computer-readable instructions are executed by the processor, the processor can execute the method for simulating camera imaging of the present application. The network interface of the computer device is used to connect and communicate with a terminal. Those skilled in the art can understand that Figure 6 the structure shown in

[0114] In this embodiment, the processor is used to execute Figure 5For the specific functions of each module and its sub-modules, the memory stores the program codes and various types of data required to execute the above-mentioned modules or sub-modules. The network interface is used for data transmission between user terminals or servers. In this embodiment, the memory stores the program codes and data required to execute all modules / sub-modules in the analog camera imaging device of the present application, and the server can call the program codes and data of the server to execute the functions of all sub-modules.

[0115] The present application also provides a storage medium storing computer-readable instructions, which when executed by one or more processors, cause the one or more processors to execute the steps of the analog camera imaging method according to any embodiment of the present application.

[0116] Those of ordinary skill in the art can understand that all or part of the processes of implementing the methods in the above embodiments of the present application can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. Among them, the aforementioned storage medium can be a computer-readable storage medium such as a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM), etc.

[0117] In summary, the present application can simulate the shooting of a real camera and achieve the same imaging effect.

[0118] Those skilled in the art of this technology can understand that the steps, measures, and solutions in the various operations, methods, and processes discussed in the present application can be alternated, changed, combined, or deleted. Further, the other steps, measures, and solutions in the various operations, methods, and processes discussed in the present application can also be alternated, changed, rearranged, decomposed, combined, or deleted. Further, those in the prior art having the steps, measures, and solutions in the various operations, methods, and processes disclosed in the present application can also be alternated, changed, rearranged, decomposed, combined, or deleted.

[0119] The above are only some embodiments of the present application. It should be noted that for those of ordinary skill in the art of this technology, without departing from the principle of the present application, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present application.

Claims

1. A method for analog camera imaging, characterized in that: The steps include: In response to the simulated camera start event, displaying the simulated camera operation interface, the simulated camera operation interface includes a shooting scene preview window, and shooting control controls and shooting parameter adjustment controls arranged in a control area outside the shooting scene preview window; Rendering each image frame in the video stream captured by the camera unit according to the customized shooting parameter set corresponding to the shooting parameter adjustment control to form a preview video stream displayed in the shooting scene preview window; In response to an imaging control event corresponding to the operation of the shooting control widget, at least one image frame is extracted from the preview video stream as shooting imaging data.

2. The analog camera imaging method according to claim 1, characterized in that: After the step of displaying the simulated camera operation interface, the following steps are included: In response to a simulated camera skin-changing event, a target customized skin template pointed to by the event is obtained from a customized skin library, wherein the target customized skin template includes a camera coating style and its supporting control style and supporting control layout; The simulated camera operation interface is refreshed and colored accordingly according to the camera coating style, and the shooting control controls and shooting parameter adjustment controls in the interface are adjusted according to the supporting control style and supporting control layout to display the simulated camera operation interface after the style change.

3. The analog camera imaging method according to claim 2, characterized in that: Before the step of responding to the simulated camera skinning event, the following steps are included: In response to the camera skin customization event, the simulated camera editing interface is displayed, and the simulated camera operation interface is converted into a canvas object that can be edited, and displayed in the canvas of the simulated camera editing interface; In response to an editing operation triggered on the canvas object in an editing tool area in the simulated camera editing interface, displaying the edited canvas object on the canvas; In response to the edit completion event, the corresponding camera coating style and its supporting control style and supporting control layout are determined according to the canvas object pointed to by the event as a custom skin template, and stored in the custom skin library.

4. The analog camera imaging method according to claim 3, characterized in that: The step of displaying the edited canvas object on the canvas in response to the editing operation triggered in the editing tool area of ​​the simulated camera editing interface comprises the following steps: Using a selection tool in an editing tool area in the simulated camera editing interface, a camera coating in the canvas object is selected as an editing object, and a target camera coating style is determined according to a colored image applied to the editing object; The target camera coating pattern is used to replace the camera coating pattern in each customized skin template in the customized skin library to obtain the corresponding customized skin templates to be evaluated; According to each customized skin template to be evaluated, the style of the simulated camera operation interface is changed accordingly to obtain a display image of the skin effect after each change; Using a preset design evaluation model to determine the skin evaluation information corresponding to each skin effect display diagram; recommending the skin effect display image to the user who triggers the camera skin customization event according to the skin evaluation information; According to the customized skin template corresponding to the skin effect display image selected by the user, the canvas object is updated accordingly and then displayed on the canvas.

5. The analog camera imaging method according to claim 4, characterized in that: Before the step of using a preset design evaluation model to determine the skin evaluation information corresponding to each skin effect display image, the following steps are included: Obtain a preset training set, wherein the training set includes a plurality of training samples and supervisory labels thereof, wherein the training sample is a skin effect display image, and the supervisory label is skin evaluation information of the skin effect display image; The training set is used to train the designed evaluation model so that the model can learn and determine the skin evaluation information of the simulated camera operation interface.

6. The analog camera imaging method according to claim 1, characterized in that: The step of displaying the simulated camera operation interface includes the following steps: Obtaining the center of gravity parameter of the device including the camera unit, playing a simulated camera maneuvering animation matching the center of gravity parameter, and displaying the full image of the simulated camera at the tilting angle of view corresponding to the center of gravity parameter in the animation; When the center of gravity parameter of the device remains unchanged within a preset time, the playing of the simulated camera adjustment animation ends, and the final displayed image of the animation gradually transitions to displaying the simulated camera operation interface.

7. The analog camera imaging method according to any one of claims 1 to 6, characterized in that: The step of rendering each image frame in the video stream captured by the camera unit according to the customized shooting parameter set corresponding to the shooting parameter adjustment control to form a preview video stream displayed in the shooting scene preview window includes the following steps: For each image frame in the video stream captured by the camera unit, convert the image frame into texture data in an image rendering environment, and configure a shader according to the customized shooting parameter set; Calling the shader to adjust the texture data to target texture data, submitting it to the rendering pipeline, and obtaining a preview image frame; The preview image frames form a preview video stream.

8. An analog camera imaging device, characterized in that: include: An event response module, used to respond to a simulated camera startup event and display a simulated camera operation interface, wherein the simulated camera operation interface includes a shooting scene preview window, and shooting control controls and shooting parameter adjustment controls arranged in a control area outside the shooting scene preview window; A preview display module, used for rendering each image frame in the video stream captured by the camera unit according to the customized shooting parameter set corresponding to the shooting parameter adjustment control, and forming a preview video stream to be displayed in the shooting scene preview window; The shooting and imaging module is used to respond to the imaging control event corresponding to the operation of the shooting control widget and extract at least one image frame from the preview video stream as shooting and imaging data.

9. A computer device comprising a central processing unit and a memory, characterized in that: The central processing unit is used to call and run the computer program stored in the memory to execute the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: It stores a computer program implemented according to the method described in any one of claims 1 to 7 in the form of computer-readable instructions, and when the computer program is called and executed by a computer, the steps included in the corresponding method are executed.

Images