Method and device for guiding user to shoot, and electronic device
By collecting preview images on the terminal device and using the server to determine the movement direction of overexposure, users can quickly capture images without obvious overexposure, solving the problem of image abnormalities caused by overexposure in the prior art, improving efficiency and users Experience.
Patent Information
- Application Number
- CN202211347638.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-10-31
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2042-10-31
AI Technical Summary
In the prior art, the user has an inefficient image failure caused by overexposed images when shooting images, and the image is unusable, and the efficiency of acquiring images without significant overexposed conditions is inefficient.
By receiving the target input of the user on the terminal device, collecting preview images and sending them to the server, the server determines the movement direction to improve the overexposure based on the exposure information. After receiving the terminal device, the terminal device outputs the guidance information, and the user guides the mobile device to reduce the overexposure area according to the guidance of the user to reduce the overexposure area .
Improve the efficiency of obtaining images without obvious overexposure, reduce the number of user operations, and reduce user requirements.
Smart Images

Figure CN115835005B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data processing technology, and in particular to a method and device for guiding a user to take photos, and an electronic device. Background Art
[0002] Overexposure usually refers to image distortion caused by an aperture that is too large, a film with too high a sensitivity, or an exposure time that is too long. Sometimes overexposure can also occur because the light reflected from the subject is too strong. For example, when shooting with a mobile phone, if the light reflected from a certain position of the subject is too strong, the image area corresponding to that position in the image will be distorted. This distorted area is also called an overexposed area.
[0003] When an image is obviously overexposed, the image in the overexposed area is distorted, making the image unusable in many businesses and requiring the user to re-submit the image. The user can only rely on shooting experience to continuously try to shoot new images to eventually obtain an image without obvious overexposure.
[0004] However, this method of obtaining an image without obvious overexposure not only places high demands on the user, but is also very inefficient. Summary of the invention
[0005] The embodiments of the present invention provide a method and device for guiding a user to take photos, and an electronic device, so as to solve the problem of low efficiency in obtaining images without obvious overexposure in the prior art.
[0006] In a first aspect, an embodiment of the present invention provides a method for guiding a user to take photos, which is applied to a terminal device, and the method includes:
[0007] Receive target input from the user;
[0008] In response to the target input, when the terminal device is in a stationary state, a scene within a current shooting range is captured to obtain a preview image at at least one moment;
[0009] Sending the preview image at the at least one moment to the server;
[0010] When first guidance information including a first moving direction is received from the server, the first guidance information is output; wherein the first moving direction is opposite to the direction of the overexposed area in the target preview image relative to the center point, and the target preview image is any preview image in which the area of the overexposed area is larger than a preset area.
[0011] Optionally, after sending the preview image at the at least one moment to the server, the method further includes:
[0012] outputting the second guidance information upon receiving the second guidance information sent by the server and used to guide the user to keep the terminal device in a stationary state;
[0013] When the terminal device is continuously in a stationary state within a target time period, a target image is captured, wherein the target time period is a time period with a target duration starting from outputting the second guidance information.
[0014] Optionally, before receiving the target input from the user, the method further includes:
[0015] outputting third guide information for guiding the user to aim at the first photographing object;
[0016] After the target image is obtained by photographing, the method further includes:
[0017] The target image is sent to the server, so that the server determines the target image as a business picture of the first photographed object in the first business.
[0018] Optionally, before outputting third guidance information for guiding the user to align with the first photographing object, the method further includes:
[0019] Displaying a service list including service information of multiple services, wherein in the service list, an identifier for indicating whether a photographing object corresponding to the service has been photographed is set corresponding to each service, each service corresponds to a photographing object, and the first photographing object is a photographing object corresponding to the first service among the multiple services;
[0020] After the target image is obtained by photographing, the method further includes:
[0021] The identifier in the service list corresponding to the first service setting is set as a second identifier indicating that the shooting of the shooting object is completed.
[0022] Optionally, in response to the target input, when the terminal device is in a stationary state, capturing a scene within a current shooting range to obtain a preview image at at least one moment includes:
[0023] In response to the target input, monitoring acceleration information of the terminal device in three different directions;
[0024] Based on the acceleration information, when the acceleration in each of the three different directions within a preset time period is less than an acceleration threshold, determining that the terminal device is in a stationary state;
[0025] The scene within the current shooting range is captured to obtain a preview image at at least one moment.
[0026] Optionally, the capturing of the scene within the current shooting range to obtain a preview image at at least one moment includes:
[0027] Continuously collecting scene information within a current shooting range through a camera unit of the terminal device to obtain video data;
[0028] Video frames at different moments in the video data are acquired to obtain a preview image at at least one moment.
[0029] In a second aspect, an embodiment of the present invention provides a method for guiding a user to take photos, which is applied to a server, and the method includes:
[0030] Receiving a preview image at at least one moment sent by a terminal device, wherein the preview image at at least one moment is an image acquired by the terminal device in response to a user's target input and when the terminal device is in a stationary state and acquires a scene within a current shooting range;
[0031] determining exposure information for each of the preview images;
[0032] In a case where the number of elements in the image set is greater than a target threshold, determining first guidance information including a first moving direction based on exposure information of a target preview image, wherein the image set is a set of preview images having an overexposed area greater than a preset area, the target preview image is any element in the image set, and the first moving direction is opposite to a direction of the overexposed area in the target preview image relative to a center point;
[0033] The first guide information is sent to the terminal device, so that the terminal device outputs the first guide information.
[0034] Optionally, determining the exposure information of each preview image includes:
[0035] For each of the preview images, input the preview image into a pre-trained neural network model to obtain a detection result output by the neural network model, wherein the detection result includes a first result indicating that there is an overexposed area larger than a preset area and recording the position of the overexposed area, or a second result indicating that there is no overexposed area larger than the preset area;
[0036] For each of the preview images, the detection result output by the neural network model corresponding to the preview image is determined as the exposure information of the preview image.
[0037] Optionally, the step of pre-training the neural network model includes:
[0038] Build an initial neural network model;
[0039] Adjusting the anchor box parameters of the initial neural network model to target parameters;
[0040] Acquire sample data containing an overexposed area larger than a preset area, wherein the label of each sample in the sample data is the coordinates of the minimum circumscribed rectangular frame of the overexposed area in the sample;
[0041] The sample data is used to train the initial neural network model.
[0042] Optionally, the determining, based on the exposure information of the target preview image, first guiding information including the first moving direction includes:
[0043] Based on the exposure information, determining the exposure area type of the target preview image; wherein the exposure area type includes: a first type characterizing that the overexposed area is located within the central area and a second type characterizing that the overexposed area is located outside the central area, and the central area is a preset range centered on the central point in the target preview image;
[0044] In a case where the exposure area type of the target preview image is a first type, determining first guide information including a translation direction opposite to a direction of the overexposed area in the target preview image relative to a center point;
[0045] Alternatively, when the exposure area type of the target preview image is the second type, first guide information including a rotation direction opposite to a direction of the overexposed area in the target preview image relative to a center point is determined.
[0046] Optionally, after determining the exposure information of each preview image, the method further includes:
[0047] In a case where the number of elements in the image set is less than or equal to the target threshold, determining second guidance information for guiding the user to keep the terminal device in a stationary state;
[0048] The second guide information is sent to the terminal device, so that the terminal device outputs the second guide information.
[0049] In a third aspect, an embodiment of the present invention further provides a device for guiding a user to shoot, which is applied to a terminal device, and the device includes:
[0050] An input module, used for receiving a user's target input;
[0051] A response module, configured to, in response to the target input, capture a scene within a current shooting range when the terminal device is in a stationary state, and obtain a preview image at at least one moment;
[0052] A first sending module, used for sending the preview image at the at least one moment to a server;
[0053] A first guidance module is used to output the first guidance information when receiving the first guidance information including the first moving direction sent by the server; wherein the first moving direction is opposite to the direction of the overexposed area in the target preview image relative to the center point, and the target preview image is any preview image in which the area of the overexposed area is larger than the preset area.
[0054] In a fourth aspect, an embodiment of the present invention further provides a device for guiding a user to shoot, which is applied to a server, and the device includes:
[0055] A first receiving module is configured to receive a preview image at at least one moment sent by a terminal device, wherein the preview image at at least one moment is an image acquired by the terminal device in response to a user's target input and when the terminal device is in a stationary state and acquires a scene within a current shooting range;
[0056] An exposure module, used for determining exposure information of each of the preview images;
[0057] a second guiding module, configured to determine, when the number of elements in the image set is greater than a target threshold, first guiding information including a first moving direction based on exposure information of a target preview image, wherein the image set is a set of preview images having an overexposed area greater than a preset area, the target preview image is any element in the image set, and the first moving direction is opposite to a direction of the overexposed area in the target preview image relative to a center point;
[0058] The second sending module is used to send the first guiding information to the terminal device, so that the terminal device outputs the first guiding information.
[0059] In a fifth aspect, an embodiment of the present application provides an electronic device, comprising a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the programs or instructions are executed by the processor, the steps of the method described in the first aspect or the second aspect are implemented.
[0060] In a sixth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the method described in the first aspect or the second aspect are implemented.
[0061] The method for guiding a user to shoot provided in an embodiment of the present invention is applied to a terminal device. Based on the user's target input, when the terminal device is in a stationary state, the scene within the current shooting range is collected to obtain a preview image at at least one moment. Since it is not certain that an image without obvious overexposure can be shot in the current situation, a preview image is used here, and the preview image is the image to be obtained by shooting at this time. The obtained preview image is sent to the server, and the server determines the exposure information of the preview image, and then determines the first moving direction to improve the overexposure of the preview image when the preview image is obviously overexposed. When the terminal device receives the first guiding information containing the first moving direction sent by the server, the terminal device displays the first guiding information, which can guide the user to move the terminal device in the first moving direction, so that an image with improved overexposure can be shot. Finally, an image without obvious overexposure is shot by continuously moving the terminal device. Since each movement is targeted, compared with blindly moving the terminal device, the present invention can undoubtedly reduce the number of times the terminal device is moved, thereby improving the efficiency of obtaining an image without obvious overexposure. At the same time, the user only needs to move the terminal device according to the first guiding information, which reduces the requirements for the user. BRIEF DESCRIPTION OF THE DRAWINGS
[0062] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.
[0063] Figure 1 One of the flowcharts to guide users to complete each shooting item;
[0064] Figure 2 The second flowchart to guide users to complete the steps of each shooting item;
[0065] Figure 3 Schematic diagram showing nine shooting items in a car insurance damage assessment scenario in an embodiment of the present invention;
[0066] Figure 4 One of the step flow charts of the method for guiding a user to take photos provided by an embodiment of the present invention;
[0067] Figure 5 A second flowchart of the method for guiding a user to take photos provided by an embodiment of the present invention;
[0068] Figure 6One of the structural block diagrams of the device for guiding a user to take photos provided by an embodiment of the present invention;
[0069] Figure 7 A second structural block diagram of the device for guiding a user to take photos provided by an embodiment of the present invention;
[0070] Figure 8 This is a structural block diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0071] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0072] It should be understood that the references to "one embodiment" or "an embodiment" throughout the specification mean that the specific features, structures, or characteristics associated with the embodiment are included in at least one embodiment of the present invention. Therefore, the references to "in one embodiment" or "in an embodiment" appearing throughout the specification do not necessarily refer to the same embodiment. In addition, these specific features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
[0073] In various embodiments of the present invention, it should be understood that the sequence numbers of the following processes do not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.
[0074] like Figure 1 As shown, it is a practical application flow chart of a method for guiding a user to shoot provided by an embodiment of the present invention. In the process of guiding a user to shoot, there are N different shooting items to be shot by the user. N is a fixed value, and its specific value is not limited here. Specifically, it includes:
[0075] Step 101: Display the home page, which includes N shooting items. Each shooting item corresponds to a round of shooting process.
[0076] Step 102: Determine whether the shooting process is completely completed. If yes, execute step 106, if not, execute step 103.
[0077] Step 103: Perform an unfinished shooting process and display a prompt page related to the shooting item corresponding to the shooting process. That is, a prompt page is displayed, and the prompt page contains data content related to the shooting item corresponding to the shooting process. When the user performs relevant operations on the prompt page, step 104 will be executed or the home page will be displayed. For example, the prompt page contains two controls, "Confirm Shooting" and "Cancel". If the user operates the "Confirm Shooting" control, step 104 will be executed; if the user operates the "Cancel" control, it will return to the home page.
[0078] Step 104: Display the scanning interface and scan the scene within the field of view. Since taking photos is required for handling various services, the scanning interface can be a preview interface after the camera is turned on. Preferably, a scanning frame can be set in the scanning interface to accurately scan or shoot the shooting target. It is understandable that the pictures taken during the scanning process will be transmitted to the server, and the server will determine whether the pictures meet the requirements.
[0079] Step 105: When receiving confirmation information returned by the server indicating that the image meets the requirements, a data collection success prompt page will be displayed to remind the user that the data of the current shooting item has been successfully collected. Then step 102 is re-executed.
[0080] Step 106: Display the process end page to remind the user that the process has ended. The process end page may include the data captured in each shooting item, and is provided with an editing control, which can delete and re-add the data captured in each shooting item, but is not limited thereto.
[0081] It should be noted that after each execution of step 105, step 102 may be automatically executed, or the successfully collected data may be displayed for user confirmation, and step 102 may be executed after the user confirms. Different designs may be made according to the shooting item targeted by the current step 105.
[0082] In the embodiment of the present invention, in a scenario with multiple shooting items, the user can be guided to complete the relevant operations of each shooting item, thereby reducing the requirements on the user and improving the user experience.
[0083] It is understandable that in Figure 1 In the application scenario shown, the method for guiding users to shoot provided by the embodiment of the present invention will be used in the process of shooting each shooting item. That is, during the shooting process, the user performs relevant operations on the terminal device, and finally collects pictures that meet the business requirements. Figure 2 As shown, in another embodiment of the present invention, when the pictures taken by the user do not meet the business requirements, the user can be guided to make adjustments to take pictures that meet the business requirements as soon as possible. Figure 1Steps 101 to 104 are similar, and steps 208 to 209 are similar. Figure 1 Steps 105 to 106 are similar and will not be described again here.
[0084] Step 205: During the process of the terminal device scanning the scene within the field of view, when the terminal device detects that it is in a stationary state, it will capture the scene within the current shooting range to obtain a preview image at least one time; and send the preview image at at least one time to the server, so that the server determines whether the overexposure of the preview image meets the business requirements. Specifically, when the number of elements in the image set is greater than the target threshold, the server sends first guidance information including a first moving direction based on the exposure information of the target preview image; wherein the image set is a set of preview images with overexposed areas greater than a preset area, the target preview image is any element in the image set, and the first moving direction is opposite to the direction of the overexposed area in the target preview image relative to the center point. When the number of elements in the image set is less than or equal to the target threshold, the server sends confirmation information indicating that there is no overexposed area to the terminal device.
[0085] Step 206: After receiving the information returned by the server, the terminal device determines whether there is an overexposed area in the currently captured image based on the specific content of the returned information. That is, if the returned information is the first guidance information, it is determined that there is an overexposed area in the currently captured image, and step 207 is executed. If the returned information is confirmation information indicating that there is no overexposed area, it is determined that there is no overexposed area in the currently captured image, and step 208 is executed.
[0086] Step 207: Output the first guidance information to remind the user to move the terminal device in the first moving direction. Then re-execute step 204, and during the execution of step 204, if it is detected that the terminal device is in a stationary state again, continue to execute steps 205 and 206 until the server returns the first guidance information.
[0087] In an embodiment of the present invention, in a scenario with multiple shooting items, it is possible to detect whether a captured image has an overexposed area while guiding the user to complete related operations of each shooting item. If the captured image has an overexposed area, corresponding guidance information is output to guide the user to take images without an overexposed area. This improves the user experience and the efficiency of obtaining images that meet business requirements.
[0088] like Figure 3As shown, it is a schematic diagram showing 9 shooting items in the scenario of car insurance damage assessment. The 9 shooting items are the accident scene, the surroundings of the vehicle, the license plate number, the frame number, the damaged part of the vehicle, the driver's license, the vehicle registration certificate, the ID card, and the bank card. Among them, the display interface of the terminal device can be divided into multiple different areas to display different data content. Specifically, the first area 31 is used to display each shooting item, and a smaller area can be demarcated again in the first area 31 to display the current shooting item, that is, the second area 32 displays the current shooting item. The third area 33 is used to display the shooting screen, and a smaller area can be demarcated again in the third area 33 to display the current shooting item, that is, the fourth area 34 displays the current shooting item. Continuing with the 9 shooting items as an example, based on Figure 1The process of shooting is shown in the flowchart. The user can click the "Start" control on the homepage to start the business process. Among them, if it is detected that the terminal device has not turned on the sound, a prompt message is displayed to remind the user to turn on the system sound. When the shooting item is "license plate number", a prompt page related to "license plate number" is displayed, which contains the text and image of the license plate number. The relevant voice can also be broadcast at the same time. A "Start shooting" button is set in the prompt page. After the user clicks the "Start shooting" button, the scanning interface is entered. In the scanning interface, the user is prompted to shoot the license plate number in the form of text + voice. After the captured picture is uploaded to the server, if the confirmation information returned by the server that the representation picture meets the business requirements is received, the data collection success prompt page is displayed, and then it is automatically switched to another page showing the successfully collected data. The page shows the collected license plate number and is set with two controls: "Cancel" and "Confirm, Next". Click "Cancel" to reshoot the license plate number, and click "Confirm, Next" to proceed to the next shooting item. Of course, in the vehicle claims scenario, after clicking "Confirm, Next", if the server detects that the license plate number currently photographed is different from the license plate number on the insurance policy, a reminder message will be sent to the terminal device, which will display the reminder message and proceed to the next shooting item after the user confirms. When the shooting item is "Vehicle Frame Number", it is similar to the case where the shooting item is "License Plate Number", so I will not repeat it here. Then when the shooting item is "Around the Vehicle", the process is similar to the case where the shooting item is "License Plate Number", the difference is that 4 pictures are taken around the vehicle, and after each picture is taken, if the 4 pictures are not completed, the shooting will continue; when the shooting item is "Accident Scene", it is similar to the case where the shooting item is "License Plate Number", so I will not repeat it here; when the shooting item is "Damaged Parts of the Vehicle", it is similar to the case where the shooting item is "Around the Vehicle", so I will not repeat it here; when the shooting item is "Driver's License", it is similar to the case where the shooting item is " The situation is similar to the situation where the shooting item is "License Plate Number", but there is also an "Album Upload" control displayed on the prompt page. By clicking the "Album Upload" control, you can select a picture from the local album, so there is no need to take a picture; when the shooting item is "Vehicle License", it is similar to the situation where the shooting item is "Driver's License", which will not be repeated here; when the shooting item is "ID Card", it is similar to the situation where the shooting item is "Driver's License", which will not be repeated here; when the shooting item is "Bank Card", it is similar to the situation where the shooting item is "Driver's License", which will not be repeated here.
[0089] See also Figure 4 , an embodiment of the present invention provides a method for guiding a user to shoot, which is applied to a terminal device, and the method includes:
[0090] Step 401: Receive a target input from a user.
[0091] In this step, the target input may be input such as clicking, sliding, long pressing, etc., and the object of the target input may be a physical button or a virtual control on the terminal device, but is not limited thereto. The target input may also be a moving operation performed by the user to change the position of the electronic device.
[0092] Step 402: In response to a target input, when the terminal device is in a stationary state, a scene within a current shooting range is captured to obtain a preview image at at least one moment.
[0093] In this step, the terminal device is in a stationary state, and it may move slightly. Here, completely motionless and almost completely motionless are both considered to be stationary. A camera unit for collecting image data is provided on the terminal device. The terminal device may be a mobile phone, and the camera unit may be a camera on the mobile phone, but is not limited thereto. The preview image is not an image generated by triggering the shooting button. Taking a mobile phone as an example, after opening the camera application of the mobile phone, if the user does not trigger the shooting button, the image displayed on the screen is an image generated by the scene within the current shooting range captured in real time, and this image is the preview image. The number of preview images obtained here may be one or more, and may be determined based on a pre-set strategy. In the case where the number of preview images is multiple, each preview image is an image captured at different times, and the different times may be continuous or discontinuous different times, which are not limited here.
[0094] Step 403: Send the preview image at least at one moment to the server.
[0095] It should be noted that data processing is performed by the server here so that when the number of elements in the image set is greater than the target threshold, the server sends first guidance information including a first moving direction based on the exposure information of the target preview image; wherein the image set is a set of preview images having an overexposed area greater than a preset area, the target preview image is any element in the image set, and the first moving direction is opposite to the direction of the overexposed area in the target preview image relative to the center point.
[0096] Step 404: upon receiving the first guidance information including the first moving direction sent by the server, output the first guidance information.
[0097] It should be noted that, since the server can send the first guidance information including the first moving direction based on the exposure information of the target preview image when the number of elements in the image set is greater than the target threshold, the terminal device can receive the first guidance information. The image set is a set of preview images with overexposed areas larger than the preset area. Here, the preview image with overexposed areas larger than the preset area is regarded as an image with obvious overexposure, and the preview image without overexposed areas and the preview image with overexposed areas but the overexposed areas are less than or equal to the preset area are regarded as images without obvious overexposure. Of course, the preview image in the overexposed area but the overexposed area is less than or equal to the preset area can also be regarded as a preview image without overexposed areas. The preset area is a smaller area preset by the user, and the user can set the size of the preset area according to business needs. It can be understood that when the proportion of preview images with obvious overexposure in the preview images received by the server is large, it means that the image currently directly captured by the terminal device is likely to be an image with obvious overexposure, and the user needs to be guided by the first guidance information. Therefore, the target threshold is related to the number of preview images. For example, if the number of preview images is 5, the target threshold may be 1, but is not limited thereto.
[0098] The target preview image is any element in the image set, that is, any preview image whose overexposed area is larger than the preset area. Since each element in the image set is an image captured when the terminal device is in a static state, the image content between each element is basically the same. Therefore, any element in the image set can be selected as the target preview image.
[0099] The first moving direction is opposite to the direction of the overexposed area in the target preview image relative to the center point. That is, moving the terminal device in the first moving direction can remove the overexposed area from the image as quickly as possible, that is, performing the least displacement so that the image captured after the movement is an image without obvious overexposure.
[0100] The data format of the first guide information can be text, picture, voice, video, etc. Here, the output method is related to the data format of the first guide information. For example, when the first guide information is text information, the first guide information can be directly displayed; when the first guide information is voice, the voice can be played. Of course, the first guide information can also be information with multiple different data formats with the same data content. For example, the first guide information includes text information and voice, and the voice is played while the text information is displayed. It can be understood that the user can learn through the output first guide information that the image captured under the current situation is an image with obvious overexposure, and can learn how the mobile terminal device can capture an image without obvious overexposure.
[0101] In an embodiment of the present invention, based on the user's target input, when the terminal device is in a stationary state, the scene within the current shooting range is collected to obtain a preview image at at least one moment. Since it is not certain that an image without obvious overexposure can be captured under the current situation, a preview image is used here, and the preview image is the image to be captured at this time. The obtained preview image is sent to the server, and the server determines the exposure information of the preview image, and then determines the first moving direction to improve the overexposure of the preview image when the preview image is obviously overexposed. When the terminal device receives the first guide information containing the first moving direction sent by the server, the terminal device displays the first guide information, which can guide the user to move the terminal device in the first moving direction, so that an image with improved overexposure can be captured. Finally, an image without obvious overexposure is captured by continuously moving the terminal device. Since each movement is targeted, compared with blindly moving the terminal device, the present invention can undoubtedly reduce the number of times the terminal device is moved, thereby improving the efficiency of obtaining an image without obvious overexposure. At the same time, the user only needs to move the terminal device according to the first guide information, which reduces the requirements for the user.
[0102] Optionally, after sending the preview image at at least one moment to the server, the method further includes:
[0103] When the second guidance information sent by the server for guiding the user to keep the terminal device in a stationary state is received, the second guidance information is output.
[0104] When the terminal device is continuously in a stationary state within a target time period, a target image is captured, wherein the target time period is a time period with a target duration starting from outputting the second guidance information.
[0105] It should be noted that when the preview images received by the server have a relatively small proportion of the preview images with obvious overexposure, it means that the image currently directly captured by the terminal device is very likely to be an image without obvious overexposure. At this time, it is necessary to guide the user to keep the terminal device in the current state and take pictures through the second guidance information. Here, the situation where the number of elements in the image set is less than or equal to the target threshold is regarded as the situation where the preview images received by the server have a relatively small proportion of the preview images with obvious overexposure. It can be understood that the situation of the second guidance information is similar to that of the first guidance information, and the difference between the two is that the data content has different semantics. And the way of outputting the second guidance information is also similar to the way of outputting the first guidance information, which will not be repeated here.
[0106] The target duration can be any duration pre-set by the user, for example, 0 seconds, 3 seconds or 5 seconds, but not limited thereto. When the target duration is greater than 0 seconds, a countdown starting from the target duration can also be displayed, and the target image can be photographed at the end of the countdown. For example, when the target duration is 3 seconds, while outputting the second guide information, a countdown control starting from 3 seconds is displayed, and the value on the countdown control is reduced by 1 every second until the target image is photographed when 0 is displayed. Of course, the second guide information can also be the countdown control itself. It can be understood that within the target time period, if the terminal device enters a non-stationary state, steps 102 and 103 are re-executed.
[0107] In an embodiment of the present invention, when the preview image currently captured by the terminal device is not obviously overexposed, a second guidance message may be displayed to guide the user to keep the terminal device in a stationary state so as to capture a target image without obvious overexposure.
[0108] Optionally, before receiving the user's target input, the method further includes:
[0109] The third guide information for guiding the user to aim at the first photographing object is output.
[0110] After the target image is captured, the method further includes:
[0111] The target image is sent to the server, so that the server determines the target image as a business picture of the first photographed object in the first business.
[0112] It should be noted that the first business is a business related to the first photographed object. In the first business, the business picture of the first photographed object needs to be used, and the business picture should not be a picture with obvious overexposure. For example, in the vehicle claims system, there is a business that handles vehicle damage. This business needs to use the business picture of the damaged part of the accident vehicle, and the business picture should not be a picture with obvious overexposure, otherwise it will be impossible to accurately judge the vehicle damage due to image distortion. Among them, the business of handling vehicle damage is the first business, and the damaged part of the accident vehicle is the first photographed object.
[0113] It is understandable that the third guidance information is similar to the first guidance information, and the difference between the two is that the data content has different semantics. And the way of outputting the third guidance information is also similar to the way of outputting the first guidance information, which will not be repeated here.
[0114] In the embodiment of the present invention, in a specific business scenario, the third guidance information can guide the user to aim the terminal device at a shooting object that matches the business, without the user having to understand the business, further reducing the requirements on the user.
[0115] Optionally, before outputting third guidance information for guiding the user to align with the first photographing object, the method further includes:
[0116] A service list including service information of multiple services is displayed.
[0117] It should be noted that, in the service list, an identifier is provided for each service to indicate whether the shooting object corresponding to the service has been completed. Each service corresponds to a shooting object, and the first shooting object is the shooting object corresponding to the first service among the multiple services. Here, the identifier can be text, numbers, letters, special symbols or regular graphics, but is not limited to this. Preferably, for each service, there are at least two different forms of identifiers set for the service, and each form represents a service status. For example, if the service status is that the shooting has been completed, the identifier can be a circle, and a pair of numbers are displayed in the circle; if the service status is that the shooting is in progress, the identifier can be a solid dot. Preferably, if the service status is that the shooting is not completed, the identifier may not be displayed.
[0118] After the target image is captured, the method further includes:
[0119] The identifier corresponding to the first service setting in the service list is set as the second identifier indicating that the photographing of the photographing object is completed.
[0120] It should be noted that after the target image is captured, it indicates that the service state of the first service has changed. To ensure the accuracy of the service list, the identifier set for the first service will be changed accordingly.
[0121] In the embodiment of the present invention, in a multi-service scenario, the service status of each service can be accurately displayed to the user through the identifiers corresponding to different service settings in the service list, thereby facilitating the user to view.
[0122] Optionally, in response to the target input, when the terminal device is in a stationary state, a scene within a current shooting range is captured to obtain a preview image at at least one moment, including:
[0123] In response to the target input, the acceleration information of the terminal device in three different directions is monitored.
[0124] It should be noted that the three different directions can be understood as the X-axis direction, the Y-axis direction and the Z-axis direction, that is, the directions of the three coordinate axes in the three-dimensional space coordinate system. Through the acceleration information in the three different directions, the movement of the current terminal device in the three-dimensional space can be accurately determined.
[0125] Based on the acceleration information, when the acceleration in each of three different directions within a preset time period is less than the acceleration threshold, it is determined that the terminal device is in a stationary state.
[0126] It should be noted that the value of the acceleration threshold is relatively small, and can be a relatively small value pre-set by the user, for example, the acceleration threshold can be 0.05 m / s2, but is not limited thereto. It is understandable that when the acceleration of the terminal device in each of the three different directions is less than the acceleration threshold, the terminal device can be regarded as stationary. In this step, in order to facilitate the user to move the terminal device to aim at the shooting object, a preset time length will be set. When the terminal device remains stationary within the preset time length, the terminal device is regarded as being in a stationary state. Specifically, taking a mobile phone as an example, the gyroscope in the mobile phone is monitored, and the acceleration values of the X, Y, and Z coordinate axes of the gyroscope are obtained every 200 milliseconds. If the accelerations in the three directions are less than 0.05 at the same time, the mobile phone is considered to be stationary. When the mobile phone is stationary for three consecutive times, it is considered that the user has aimed at the shooting object and is stationary.
[0127] The scene within the current shooting range is captured to obtain a preview image at at least one moment.
[0128] In the embodiment of the present invention, based on the acceleration information of the terminal device in three different directions, the current state of the terminal device can be quickly identified, and a preview image can be acquired when the terminal device is in a stationary state.
[0129] Optionally, capturing a scene within a current shooting range to obtain a preview image at at least one moment includes:
[0130] The camera unit of the terminal device continuously collects scene information within the current shooting range to obtain video data;
[0131] Video frames at different moments in the video data are acquired to obtain a preview image at least one moment.
[0132] It should be noted that after obtaining the video data, the video data can be analyzed in real time, and the last frame of each target number of video frames in the video data is taken as a preview image, thereby obtaining preview images at multiple moments. The target number can be set according to needs, for example, the target number can be 10 or 15, but is not limited thereto.
[0133] In the embodiment of the present invention, a preview image can be quickly obtained by extracting video frames from video data.
[0134] The method for guiding a user to take photos provided by an embodiment of the present invention is introduced above from the terminal device side. The method for guiding a user to take photos on the server side will be introduced below with reference to the accompanying drawings.
[0135] An embodiment of the present invention provides a method for guiding a user to take photos, which is applied to a server. The method includes:
[0136] Step 51: Receive a preview image at at least one time point sent by a terminal device.
[0137] It should be noted that the preview image at at least one moment is an image obtained by capturing a scene within a current shooting range when the terminal device is in a stationary state in response to a target input by the user.
[0138] Step 52: Determine exposure information of each preview image.
[0139] It should be noted that the exposure information is acquired for each preview image respectively to obtain the exposure information of each preview image. The exposure information can characterize the overexposure of the preview image, that is, whether the preview image is obviously overexposed. Here, the standard for the presence of obvious overexposure and the absence of obvious overexposure is the same as the standard for the presence of obvious overexposure and the absence of obvious overexposure in the embodiment of the method for guiding a user to shoot applied to the terminal device, and will not be repeated here.
[0140] Step 53: when the number of elements in the image set is greater than the target threshold, determine first guidance information including a first moving direction based on exposure information of the target preview image.
[0141] It should be noted that the image set is a set of preview images with overexposed areas larger than a preset area. Here, preview images with overexposed areas larger than a preset area are regarded as images with obvious overexposure, and preview images without overexposed areas and preview images with overexposed areas but with overexposed areas less than or equal to the preset area are regarded as images without obvious overexposure. Of course, preview images with overexposed areas but with overexposed areas less than or equal to the preset area can also be regarded as preview images without overexposed areas. Among them, the preset area is a relatively small area preset by the user, and the user can set the size of the preset area according to business needs. It can be understood that when the preview images received by the server have a large proportion of preview images with obvious overexposure, it means that the image currently directly captured by the terminal device is likely to be an image with obvious overexposure, and the user needs to be guided by the first guidance information. Therefore, the target threshold is related to the number of preview images. For example, if the number of preview images is 5, the target threshold can be 1, but it is not limited to this.
[0142] The target preview image is any element in the image set. Since each element in the image set is an image captured when the terminal device is in a static state, the image contents of each element are basically the same. Therefore, any element in the image set can be selected as the target preview image.
[0143] The first moving direction is opposite to the direction of the overexposed area in the target preview image relative to the center point. That is, moving the terminal device in the first moving direction can remove the overexposed area from the image as quickly as possible, that is, performing the least displacement so that the image captured after the movement is an image without obvious overexposure.
[0144] Step 54: Send the first guide information to the terminal device, so that the terminal device outputs the first guide information.
[0145] It should be noted that the data format of the first guide information can be text, picture, voice, video, etc. Here, the output mode of the terminal device is related to the data format of the first guide information. For example, when the first guide information is text information, the first guide information can be directly displayed; when the first guide information is voice, the voice can be played. Of course, the first guide information can also be information with the same data content in multiple different data formats. For example, the first guide information includes text information and voice, and the voice is played while the text information is displayed. It can be understood that the user can learn through the first guide information output by the terminal device that the image captured under the current situation is an image with obvious overexposure, and can learn how to move the terminal device to capture an image without obvious overexposure.
[0146] In an embodiment of the present invention, a preview image at at least one moment sent by a terminal device can be received, wherein the preview image at at least one moment is an image obtained by the terminal device in response to a user's target input, when the terminal device is in a stationary state, by capturing a scene within a current shooting range. Since it is not certain that the terminal device can capture an image without obvious overexposure under the current situation, a preview image is used here, and the preview image is the image that the terminal device will obtain when shooting at this time. The server determines the exposure information of the preview image, and then when the preview image is obviously overexposed, determines a first moving direction to improve the overexposure of the preview image, and sends first guidance information including the first moving direction to the terminal device, which is displayed to the user by the terminal device, guiding the user to move the terminal device in the first moving direction, so that an image with improved overexposure can be captured. Finally, an image without obvious overexposure is captured by continuously moving the terminal device. Since each movement is targeted, compared with blindly moving the terminal device, the present invention can undoubtedly reduce the number of times the terminal device is moved, thereby improving the efficiency of obtaining an image without obvious overexposure. At the same time, the user only needs to move the terminal device according to the first guidance information, which reduces the requirements for the user.
[0147] Optionally, determining exposure information of each preview image includes:
[0148] For each preview image, input the preview image into a pre-trained neural network model to obtain a detection result output by the neural network model, wherein the detection result includes a first result that an overexposed area larger than a preset area exists and a position of the overexposed area is recorded, or a second result that an overexposed area larger than the preset area does not exist;
[0149] For each preview image, the detection result of the corresponding preview image output by the neural network model is determined as the exposure information of the preview image.
[0150] It should be noted that the pre-trained neural network model has the function of detecting the exposure information of the image, and after the image is input into the pre-trained neural network model, the detection result output by the pre-trained neural network model is the exposure information. Here, the pre-trained neural network model can determine whether the image is obviously overexposed, and determine the relative position of the overexposed area in the image when there is obvious overexposure. In other words, the pre-trained neural network model can determine whether there is an overexposed area larger than a preset area in the image, and when it is determined that there is an overexposed area larger than the preset area, continue to determine the relative position of the overexposed area in the image.
[0151] In the embodiment of the present invention, the exposure information of each preview image can be obtained quickly and accurately using a pre-trained neural network model.
[0152] Optionally, the step of pre-training the neural network model includes:
[0153] Build an initial neural network model;
[0154] Adjust the anchor box parameters of the initial neural network model to the target parameters;
[0155] Acquire sample data containing an overexposed area larger than a preset area, wherein the label of each sample in the sample data is the coordinates of the minimum circumscribed rectangular frame of the overexposed area in the sample;
[0156] The sample data is used to train the initial neural network model.
[0157] It should be noted that the initial neural network model can be a target detection model, for example, a deep neural network model of an EfficientDet-D2 structure, but is not limited thereto. By modifying its anchor box parameters, it can be adapted to detect small targets such as overexposed areas. Here, a batch of images containing overexposed areas larger than a preset area are annotated with an annotation tool as samples, and the coordinates of the minimum circumscribed rectangular box of the overexposed area on each image are used as labels. It can be understood that the process of using sample data for model training is a process of continuously optimizing model parameters, and the training is stopped after the model converges to obtain a trained neural network model.
[0158] In an embodiment of the present invention, during the process of training a neural network model, the anchor box parameters of the constructed initial neural network model are adjusted to adapt it to detecting small targets such as overexposed areas, thereby obtaining a robust neural network model.
[0159] Optionally, the initial neural network model in the above-mentioned invention embodiment can be replaced with a deep learning target detection model of the same type as the EfficientDet-D2 structure, for example, the same type of deep learning target detection model can be RCNN (Region Detection Convolutional Neural Network, Region-CNN), Faster-RCNN (Fast Region Detection Convolutional Neural Network), YOLO (You Only Look Once) v1 / v2 / v3, SSD (Single Shot MultiBox Detector), Mask RCNN (Mask Region Detection Convolutional Neural Network, Mask Region-CNN) and RefineDet (Refinement Detector) and other models. Of course, the deep learning image segmentation model can also be used as the initial neural network model in the above-mentioned invention embodiment. Specifically, the deep learning image segmentation model can be FCN (Fully Convolution Network), SegNet (Segmentation Region Detection Network), DeepLab v1 / v2 / v3 / v3+, PSPNet (Pyramid Pooling Scene Parsing Network, Pyramid Scene Parseing Network,) and other models, but are not limited to this.
[0160] Optionally, determining first guidance information including a first moving direction based on exposure information of the target preview image includes:
[0161] Based on the exposure information, determining the exposure area type of the target preview image; wherein the exposure area type includes: a first type characterizing that the overexposed area is located within the central area and a second type characterizing that the overexposed area is located outside the central area, and the central area is a preset range centered on the central point in the target preview image;
[0162] In a case where the exposure area type of the target preview image is a first type, determining first guide information including a translation direction opposite to a direction of the overexposed area in the target preview image relative to the center point;
[0163] Alternatively, when the exposure area type of the target preview image is the second type, first guide information including a rotation direction opposite to a direction of the overexposed area in the target preview image relative to the center point is determined.
[0164] It should be noted that the range of the central area is a predetermined range, which is related to the height and width of the preview image. Here, any area within the central area can be regarded as being closer to the center point of the preview image, and any area outside the central area can be regarded as being farther from the center point of the preview image. For example, the central area can be within (0.4W, 0.4H), but not limited to this, where W is the width of the preview image and H is the height of the preview image. Taking the position of the lower left corner of the preview image as the coordinate origin, the width direction is the X-axis direction, and the height direction is the Y-axis direction, the coordinates of the overexposed area in the preview image are bbox=[x min ,y min ,x max ,y max ], where x min Indicates the horizontal coordinate of the leftmost edge of the overexposed area, x max Indicates the horizontal coordinate of the rightmost edge of the overexposed area, y min Indicates the ordinate at the bottom edge of the overexposed area, y max Indicates the ordinate at the top edge of the overexposed area. Based on formula 1, the exposure area type of the target preview image can be obtained.
[0165] Formula 1:
[0166] Among them, x min Indicates the horizontal coordinate of the leftmost edge of the overexposed area, x max Indicates the horizontal coordinate of the rightmost edge of the overexposed area, y min Indicates the ordinate at the bottom edge of the overexposed area, y max Indicates the ordinate at the top edge of the overexposed area. type indicates the exposure area type. If type is equal to 0, the exposure area type is the first type; if type is equal to 1, the exposure area type is the second type.
[0167] When determining the first moving direction, it is necessary to calculate the horizontal distance and the vertical distance from the center position of the overexposed area to the center point of the preview image, and the specific calculation is as shown in Formula 2 and Formula 3.
[0168] Formula 2: d1 = (x min +x max ) / 2-W / 2; d1 represents the horizontal distance from the center of the overexposed area to the center of the preview image, x min Indicates the horizontal coordinate of the leftmost edge of the overexposed area, x max It represents the horizontal coordinate of the rightmost edge of the overexposed area, and W is the width of the preview image.
[0169] Formula 3: d2=(y min+y max ) / 2-H / 2; d2 represents the vertical distance from the center of the overexposed area to the center of the preview image, y min Indicates the ordinate at the bottom edge of the overexposed area, y max Indicates the ordinate of the top edge of the overexposed area, where H is the height of the preview image.
[0170] If type is 0, when d1≥0, it means that the overexposed area is located to the right of the center. At this time, the amplitude of the change of the terminal device should be small, and the translation direction in the first guidance information is the left side. The first guidance information can be "Please move to the left", otherwise the first guidance information is "Please move to the right".
[0171] If type is 1, it is considered that the overexposed area is outside the preview image. In this case, the amplitude of the change of the terminal device should be large. Assuming that the overexposed area is within the left border of the preview image, the rotation direction in the first guidance information is the right side. The first guidance information can be "Please rotate the camera to the right."
[0172] In the embodiment of the present invention, based on the relative position of the overexposed area and the center position of the preview image, a translation direction or a rotation direction is added to the first guidance information so as to improve the overexposure in the preview image with a small position change of the terminal device.
[0173] Optionally, after determining the exposure information of each preview image, the method further includes:
[0174] When the number of elements in the image set is less than or equal to the target threshold, determining second guidance information for guiding the user to keep the terminal device in a stationary state;
[0175] The second guide information is sent to the terminal device so that the terminal device outputs the second guide information.
[0176] It should be noted that when the preview images received by the server have a small proportion of preview images with obvious overexposure, it means that the image currently directly captured by the terminal device is very likely to be an image without obvious overexposure. At this time, it is necessary to guide the user to keep the terminal device in the current state and take pictures through the second guidance information. It can be understood that the situation of the second guidance information is similar to that of the first guidance information. The difference between the two is that the data content has different semantics. And the way of outputting the second guidance information is also similar to the way of outputting the first guidance information, which will not be repeated here.
[0177] In the embodiment of the present invention, when it is determined that the preview image currently captured by the terminal device is not obviously overexposed, second guidance information may be displayed to guide the user to directly capture the target image without obvious overexposure.
[0178] like Figure 5 As shown, an embodiment of the present invention provides a method for guiding users to take photos, which is applied to a system including a terminal device and a server. Here, a specific business scenario is used as an example for explanation, but the present invention is not limited to this business scenario. In a small vehicle damage report scenario, users are required to take corresponding images according to the damage report requirements. The method includes:
[0179] Step 501: prompting the user to aim at the target and shoot in the terminal device, that is, displaying the third guidance information for guiding the user to aim at the first shooting object. Specifically, after the user enters the guided shooting interface, the upper left corner of the interface displays "Please aim the camera at the damaged part of the vehicle", and a voice prompts the user with relevant content.
[0180] Step 502: Determine whether the camera is in a stationary state. If so, collect a preview image and send the preview image to the server, and then execute step 503 in the server. If not, continue to execute step 501. Here, the process of collecting the preview image and sending the preview image to the server is similar to the above steps 402 and 403, and the steps are repeated here. The method of determining whether the camera is in a stationary state is similar to the method of determining whether the terminal device is in a stationary state in the above embodiment of the invention, and it is also not repeated here.
[0181] Step 503: Using a pre-trained exposure point detection model in the server, the exposure point in the preview image is detected. The exposure point here is an over-exposed area larger than a preset area in the above-mentioned embodiment of the invention. The exposure point detection model is the neural network model in the above-mentioned embodiment of the invention.
[0182] Step 504: The server determines whether there is an exposure point in the preview image based on the output of the exposure point detection model. If yes, step 506 is executed; otherwise, step 505 is executed. In the case where there are multiple preview images, the case where the number of preview images with exposure points is greater than the target threshold is regarded as the case where there are exposure points in the preview image; the case where the number of preview images with exposure points is less than or equal to the target threshold is regarded as the case where there are no exposure points in the preview image. The server sends the judgment result back to the terminal device.
[0183] Step 505: When the terminal device determines that there is no exposure point, it displays a countdown to remind the user to keep the terminal device still before the countdown ends, and to capture the target image at the end of the countdown.
[0184] Step 506: When the terminal device determines that the judgment result is that there is an exposure point, it will display guidance information according to the rules to guide the user, so that the user can adjust the position of the camera or the terminal device to improve the overexposure. The display of guidance information according to the rules can be referred to in the above-mentioned invention embodiment, and the steps are repeated here.
[0185] In the embodiment of the present invention, the user's shooting situation can be analyzed and prompted in real time, which solves the problem that the shot image has exposure points, resulting in poor image quality and the need for reshooting. The shooting guidance process is simple and there are both voice and text prompts, which can standardize the user's shooting habits and improve the efficiency of using the image in subsequent claims or insurance.
[0186] See also Figure 6 The embodiment of the present invention further provides a device for guiding a user to shoot, which is applied to a terminal device, and the device includes:
[0187] An input module 61, used to receive a target input from a user;
[0188] A response module 62, configured to respond to a target input and, when the terminal device is in a stationary state, collect a scene within a current shooting range to obtain a preview image at at least one moment;
[0189] A first sending module 63, configured to send a preview image at at least one moment to a server;
[0190] The first guidance module 64 is used to output the first guidance information when receiving the first guidance information including the first moving direction sent by the server; wherein the first moving direction is opposite to the direction of the overexposed area in the target preview image relative to the center point, and the target preview image is any preview image in which the area of the overexposed area is larger than the preset area.
[0191] Optionally, the device further comprises:
[0192] A first guidance output module, configured to output second guidance information when receiving second guidance information sent by the server and used to guide the user to keep the terminal device in a stationary state;
[0193] The shooting module is used to shoot and obtain a target image when the terminal device is continuously in a stationary state within a target time period, wherein the target time period is a time period with a target duration starting from the output of the second guidance information.
[0194] Optionally, the device further comprises:
[0195] A second guidance output module, used to output third guidance information for guiding the user to aim at the first photographing object;
[0196] The business image module is used to send the target image to the server, so that the server determines the target image as a business image of the first photographed object in the first business.
[0197] Optionally, the device further comprises:
[0198] A list module, used to display a service list including service information of multiple services, wherein in the service list, an identifier is set corresponding to each service to indicate whether the shooting object corresponding to the service has been completed, each service corresponds to a shooting object, and the first shooting object is the shooting object corresponding to the first service among the multiple services;
[0199] The identification setting module is used to set the identification corresponding to the first service setting in the service list to a second identification indicating that the photographing of the photographing object has been completed.
[0200] Optionally, the response module 62 includes:
[0201] A first response unit, configured to monitor acceleration information of the terminal device in three different directions in response to a target input;
[0202] A second response unit is used to determine that the terminal device is in a stationary state based on the acceleration information when the acceleration in each of three different directions within a preset time period is less than an acceleration threshold;
[0203] The third response unit is used to capture the scene within the current shooting range to obtain a preview image at at least one moment.
[0204] Optionally, capturing a scene within a current shooting range to obtain a preview image at at least one moment includes:
[0205] The camera unit of the terminal device continuously collects scene information within the current shooting range to obtain video data;
[0206] Video frames at different moments in the video data are acquired to obtain a preview image at least one moment.
[0207] The device for guiding a user to take a photo provided by the embodiment of the present invention can achieve Figure 4 The various processes of the method for guiding the user to take photos in the method embodiment are implemented, and to avoid repetition, they are not described here.
[0208] In an embodiment of the present invention, based on the user's target input, when the terminal device is in a stationary state, the scene within the current shooting range is collected to obtain a preview image at at least one moment. Since it is not certain that an image without obvious overexposure can be captured under the current situation, a preview image is used here, and the preview image is the image to be captured at this time. The obtained preview image is sent to the server, and the server determines the exposure information of the preview image, and then determines the first moving direction to improve the overexposure of the preview image when the preview image is obviously overexposed. When the terminal device receives the first guide information containing the first moving direction sent by the server, the terminal device displays the first guide information, which can guide the user to move the terminal device in the first moving direction, so that an image with improved overexposure can be captured. Finally, an image without obvious overexposure is captured by continuously moving the terminal device. Since each movement is targeted, compared with blindly moving the terminal device, the present invention can undoubtedly reduce the number of times the terminal device is moved, thereby improving the efficiency of obtaining an image without obvious overexposure. At the same time, the user only needs to move the terminal device according to the first guide information, which reduces the requirements for the user.
[0209] See also Figure 7 The embodiment of the present invention further provides a device for guiding a user to shoot, which is applied to a terminal device, and the device includes:
[0210] The first receiving module 71 is used to receive a preview image at at least one moment sent by a terminal device, wherein the preview image at at least one moment is an image obtained by the terminal device in response to a user's target input and when the terminal device is in a stationary state and acquires a scene within a current shooting range;
[0211] An exposure module 72, for determining exposure information of each preview image;
[0212] A second guiding module 73 is configured to determine, when the number of elements in the image set is greater than a target threshold, first guiding information including a first moving direction based on exposure information of a target preview image, wherein the image set is a set of preview images having an overexposed area greater than a preset area, the target preview image is any element in the image set, and the first moving direction is opposite to a direction of the overexposed area in the target preview image relative to a center point;
[0213] The second sending module 74 is used to send the first guidance information to the terminal device, so that the terminal device outputs the first guidance information.
[0214] Optionally, the exposure module comprises:
[0215] A first exposure unit is used for inputting the preview image into a pre-trained neural network model for each preview image to obtain a detection result output by the neural network model, wherein the detection result includes a first result that an over-exposed area larger than a preset area exists and a position of the over-exposed area is recorded, or a second result that an over-exposed area larger than the preset area does not exist;
[0216] The second exposure unit is used to determine, for each preview image, the detection result of the corresponding preview image output by the neural network model as the exposure information of the preview image.
[0217] Optionally, the step of pre-training the neural network model includes:
[0218] Build an initial neural network model;
[0219] Adjust the anchor box parameters of the initial neural network model to the target parameters;
[0220] Acquire sample data containing an overexposed area larger than a preset area, wherein the label of each sample in the sample data is the coordinates of the minimum circumscribed rectangular frame of the overexposed area in the sample;
[0221] The sample data is used to train the initial neural network model.
[0222] Optionally, the second guiding module 73 includes:
[0223] a type unit, configured to determine an exposure area type of the target preview image based on the exposure information; wherein the exposure area type includes: a first type characterizing that the overexposed area is located within the central area and a second type characterizing that the overexposed area is located outside the central area, wherein the central area is a preset range centered on the central point in the target preview image;
[0224] a moving unit, configured to determine, when the exposure area type of the target preview image is a first type, first guide information including a translation direction opposite to a direction of the overexposed area in the target preview image relative to the center point;
[0225] Alternatively, when the exposure area type of the target preview image is the second type, first guide information including a rotation direction opposite to a direction of the overexposed area in the target preview image relative to the center point is determined.
[0226] Optionally, the device further comprises:
[0227] A third guiding module, configured to determine, when the number of elements in the image set is less than or equal to a target threshold, second guiding information for guiding the user to keep the terminal device in a stationary state;
[0228] The fourth guiding module is used to send the second guiding information to the terminal device so that the terminal device outputs the second guiding information.
[0229] In an embodiment of the present invention, a preview image at at least one moment sent by a terminal device can be received, wherein the preview image at at least one moment is an image obtained by the terminal device in response to a user's target input, when the terminal device is in a stationary state, by capturing a scene within a current shooting range. Since it is not certain that the terminal device can capture an image without obvious overexposure under the current situation, a preview image is used here, and the preview image is the image that the terminal device will obtain when shooting at this time. The server determines the exposure information of the preview image, and then when the preview image is obviously overexposed, determines a first moving direction to improve the overexposure of the preview image, and sends first guidance information including the first moving direction to the terminal device, which is displayed to the user by the terminal device, guiding the user to move the terminal device in the first moving direction, so that an image with improved overexposure can be captured. Finally, an image without obvious overexposure is captured by continuously moving the terminal device. Since each movement is targeted, compared with blindly moving the terminal device, the present invention can undoubtedly reduce the number of times the terminal device is moved, thereby improving the efficiency of obtaining an image without obvious overexposure. At the same time, the user only needs to move the terminal device according to the first guidance information, which reduces the requirements for the user.
[0230] Alternatively, if Figure 8 As shown, an embodiment of the present application also provides an electronic device 800, including a processor 801 and a memory 802, wherein the memory 802 stores programs or instructions that can be executed on the processor 801, and when the program or instructions are executed by the processor 801, the various steps of the above-mentioned method embodiment for guiding the user to take pictures are implemented, and the same technical effect can be achieved. To avoid repetition, they are not repeated here.
[0231] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.
[0232] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned method embodiment for guiding a user to take photos is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.
[0233] The processor is a processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0234] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the method and device in the embodiment of the present invention is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.
[0235] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.
[0236] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation modes, which are merely illustrative rather than restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which are within the protection of the present invention.
Claims
1. A method for guiding a user to take photos, applied to a terminal device, characterized in that: The method comprises: Receive target input from the user; In response to the target input, when the terminal device is in a stationary state, a scene within a current shooting range is captured to obtain a preview image at at least one moment; Sending the preview image at the at least one moment to the server; outputting the first guiding information upon receiving the first guiding information including the first moving direction sent by the server; wherein the first moving direction is opposite to the direction of the overexposed area in the target preview image relative to the center point, and the target preview image is any preview image in which the area of the overexposed area is larger than the preset area; When the information returned by the server is confirmation information indicating that there is no overexposed area, capturing a target image; Before receiving the target input from the user, the method further includes: Displaying a service list including service information of multiple services, wherein in the service list, an identifier for indicating whether a photographing object corresponding to the service has been photographed is set corresponding to each service, each service corresponds to a photographing object, and a first photographing object is a photographing object corresponding to a first service among the multiple services; After the target image is captured, the method further includes: The identifier in the service list corresponding to the first service setting is set as a second identifier indicating that the shooting of the shooting object is completed.
2. The method according to claim 1, characterized in that After sending the preview image at the at least one moment to the server, the method further includes: outputting the second guidance information upon receiving the second guidance information sent by the server and used to guide the user to keep the terminal device in a stationary state; When the terminal device is continuously in a stationary state within a target time period, a target image is captured, wherein the target time period is a time period with a target duration starting from outputting the second guidance information.
3. The method according to claim 2, characterized in that Before receiving the target input from the user, the method further includes: outputting third guide information for guiding the user to aim at the first photographing object; After the target image is obtained by photographing, the method further includes: The target image is sent to the server, so that the server determines the target image as a business picture of the first photographed object in the first business.
4. The method according to claim 1, characterized in that: In response to the target input, when the terminal device is in a stationary state, capturing a scene within a current shooting range to obtain a preview image at at least one moment includes: In response to the target input, monitoring acceleration information of the terminal device in three different directions; Based on the acceleration information, when the acceleration in each of the three different directions within a preset time period is less than an acceleration threshold, determining that the terminal device is in a stationary state; The scene within the current shooting range is captured to obtain a preview image at at least one moment.
5. The method according to claim 1 or 4, characterized in that: The step of collecting the scene within the current shooting range to obtain a preview image at at least one moment includes: Continuously collecting scene information within a current shooting range through a camera unit of the terminal device to obtain video data; Video frames at different moments in the video data are acquired to obtain a preview image at at least one moment.
6. A method for guiding a user to take photos, applied to a server, characterized in that: The method comprises: Receiving a preview image at at least one moment sent by a terminal device, wherein the preview image at at least one moment is an image acquired by the terminal device in response to a user's target input and when the terminal device is in a stationary state and acquires a scene within a current shooting range; determining exposure information for each of the preview images; In a case where the number of elements in the image set is greater than a target threshold, determining first guidance information including a first moving direction based on exposure information of a target preview image, wherein the image set is a set of preview images having an overexposed area greater than a preset area, the target preview image is any element in the image set, and the first moving direction is opposite to a direction of the overexposed area in the target preview image relative to a center point; Sending the first guide information to the terminal device so that the terminal device outputs the first guide information; When the number of elements in the image set is less than or equal to the target threshold, confirmation information representing a non-overexposed area is sent to the terminal device, so that the terminal device shoots the first shooting object corresponding to the first business to obtain the target image; wherein the first business is any business in a business list containing business information of multiple businesses displayed by the terminal device; in the business list, an identifier for representing whether the shooting object corresponding to the business has been completed is set for each of the businesses, and each of the businesses corresponds to a shooting object; when the shooting of the shooting object corresponding to the first business in the business list is completed, the identifier corresponding to the first business is a second identifier representing that the shooting of the shooting object has been completed.
7. The method according to claim 6, characterized in that The determining of exposure information of each preview image includes: For each of the preview images, input the preview image into a pre-trained neural network model to obtain a detection result output by the neural network model, wherein the detection result includes a first result indicating that there is an overexposed area larger than a preset area and recording the position of the overexposed area, or a second result indicating that there is no overexposed area larger than the preset area; For each of the preview images, the detection result output by the neural network model corresponding to the preview image is determined as the exposure information of the preview image.
8. The method according to claim 7, characterized in that The steps of pre-training the neural network model include: Build an initial neural network model; Adjusting the anchor box parameters of the initial neural network model to target parameters; Acquire sample data containing an overexposed area larger than a preset area, wherein the label of each sample in the sample data is the coordinates of the minimum circumscribed rectangular frame of the overexposed area in the sample; The sample data is used to train the initial neural network model.
9. The method according to claim 6, characterized in that The step of determining first guidance information including a first moving direction based on exposure information of the target preview image includes: Based on the exposure information, determining the exposure area type of the target preview image; wherein the exposure area type includes: a first type characterizing that the overexposed area is located within the central area and a second type characterizing that the overexposed area is located outside the central area, and the central area is a preset range centered on the central point in the target preview image; In a case where the exposure area type of the target preview image is a first type, determining first guide information including a translation direction opposite to a direction of the overexposed area in the target preview image relative to a center point; Alternatively, when the exposure area type of the target preview image is the second type, first guide information including a rotation direction opposite to a direction of the overexposed area relative to the center point in the target preview image is determined.
10. The method according to claim 6, characterized in that After determining the exposure information of each of the preview images, the method further includes: In a case where the number of elements in the image set is less than or equal to the target threshold, determining second guidance information for guiding the user to keep the terminal device in a stationary state; The second guide information is sent to the terminal device, so that the terminal device outputs the second guide information.
11. A device for guiding a user to take photos, applied to a terminal device, characterized in that: The device comprises: An input module, used for receiving a user's target input; A response module, configured to, in response to the target input, capture a scene within a current shooting range when the terminal device is in a stationary state, and obtain a preview image at at least one moment; A first sending module, used for sending the preview image at the at least one moment to a server; a first guiding module, configured to output the first guiding information upon receiving first guiding information including a first moving direction sent by the server; wherein the first moving direction is opposite to a direction of an overexposed area in a target preview image relative to a center point, and the target preview image is any preview image in which an area of the overexposed area is larger than a preset area; The device is also used for: When the information returned by the server is confirmation information indicating that there is no overexposed area, capturing a target image; Before receiving the user's target input, displaying a service list including service information of multiple services, wherein in the service list, an indicator for indicating whether the shooting object corresponding to the service has been shot is set corresponding to each service, each service corresponds to a shooting object, and the first shooting object is the shooting object corresponding to the first service among the multiple services; After the target image is captured, the identifier in the service list corresponding to the first service setting is set as a second identifier indicating that the capture of the captured object is completed.
12. A device for guiding a user to take photos, applied to a server, characterized in that: The device comprises: A first receiving module is configured to receive a preview image at at least one moment sent by a terminal device, wherein the preview image at at least one moment is an image acquired by the terminal device in response to a user's target input and when the terminal device is in a stationary state and acquires a scene within a current shooting range; An exposure module, used for determining exposure information of each of the preview images; a second guiding module, configured to determine, when the number of elements in the image set is greater than a target threshold, first guiding information including a first moving direction based on exposure information of a target preview image, wherein the image set is a set of preview images having an overexposed area greater than a preset area, the target preview image is any element in the image set, and the first moving direction is opposite to a direction of the overexposed area in the target preview image relative to a center point; A second sending module, configured to send the first guiding information to the terminal device, so that the terminal device outputs the first guiding information; The second sending module is also used to, when the number of elements in the image set is less than or equal to the target threshold, send confirmation information representing a non-overexposed area to the terminal device, so that the terminal device shoots the first shooting object corresponding to the first business to obtain the target image; wherein the first business is any business in a business list containing business information of multiple businesses displayed by the terminal device; in the business list, an identifier is set for each business to represent whether the shooting object corresponding to the business has been completed, and each business corresponds to a shooting object; when the shooting object corresponding to the first business in the business list is completed, the identifier corresponding to the first business is a second identifier representing that the shooting of the shooting object has been completed.
13. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the method for guiding a user to take photos as described in any one of claims 1 to 5 or the steps of the method for guiding a user to take photos as described in any one of claims 6 to 10 are implemented.
14. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by the processor, the method for guiding a user to take photos as described in any one of claims 1 to 5 or the steps of the method for guiding a user to take photos as described in any one of claims 6 to 10 are implemented.
