Photographing posture prompting method, device, equipment and computer-readable storage medium

Abstract

The present application provides a method, apparatus, device, and computer-readable storage medium for prompting a photographic posture. The method is applied to an electronic device comprising a camera and an accelerometer. The method comprises: obtaining a photographic scene identifier upon detecting that the camera is activated; receiving a user's photographic operation instruction and collecting three-axis acceleration data of the electronic device via the accelerometer; determining photographic posture information of the electronic device based on the three-axis acceleration data and pre-calibrated external parameters; and outputting a photographic posture adjustment prompt if the photographic posture information does not match standard posture information pre-associated with the photographic scene identifier. In the embodiments of the present application, the photographic posture information is determined based on pre-calibrated external parameters, thereby reducing the amount of data processing required for the photographic posture prompt and making the photographic posture prompt more efficient and accurate.

Description

Technical Field

[0001] The present application relates to the field of photography, and in particular to a method, apparatus, device and computer-readable storage medium for prompting photographic postures. Background Art

[0002] The merchandise cabinets in stores are filled with a wide variety of goods. Inspectors will inspect and photograph the goods in the merchandise cabinets. When inspectors photograph merchandise shelves, liquor counters or freezers, it is easy for them to take skewed or non-compliant pictures, which brings great inconvenience to the subsequent product image detection and recognition.

[0003] Therefore, it is necessary to provide photo posture prompts when taking photos during inspections (or taking photos in other prescribed formats). Currently, photo posture prompts are mainly implemented in the following ways: Implementation method 1: Photo posture prompts based on plane detection and posture estimation of visual simultaneous localization and mapping (SLAM). This photo posture prompt method has fewer usage scenarios, requires a lot of computing power when providing photo posture prompts, and has a slow photo posture prompt speed; Implementation method 2: Photo posture prompts based on deep learning. This photo posture prompt method requires pre-collection and annotation of a large amount of image data to build an image analysis model. The effect of the image analysis model for photo posture prompts mainly depends on the amount of training data. The data preparation workload is large, and the accuracy of the photo posture prompts cannot be guaranteed. Summary of the Invention

[0004] The present application provides a method, apparatus, device, and computer-readable storage medium for providing a photographic posture prompt, which accurately determines photographic posture information by fusing pre-calibrated external parameters and three-axis acceleration data, thereby enabling efficient and accurate photographic posture prompts without requiring excessive user operation, lowering the hardware requirements for the device, and reducing device costs.

[0005] In one aspect, the present application provides a method for prompting a photographing posture, which is applied to an electronic device including a camera and an acceleration sensor. The method comprises the following steps:

[0006] When detecting that the photographing device is started, obtaining a photographing scene identifier;

[0007] receiving a user's photographing operation instruction, and collecting three-axis acceleration data of the electronic device through the acceleration sensor;

[0008] Determining photographing posture information of the electronic device based on the three-axis acceleration data and pre-calibrated extrinsic parameters, where the extrinsic parameters refer to angular parameters of a photographing device and an acceleration sensor in the electronic device on a mounting structure of the electronic device;

[0009] If the photographing posture information does not match the standard posture information pre-associated with the photographing scene identifier, a photographing posture adjustment prompt is output.

[0010] In some embodiments of the present application, after receiving the user's photo-taking instruction and collecting the three-axis acceleration data of the electronic device through the acceleration sensor, the method further includes:

[0011] Determine whether there are pre-calibrated external parameters;

[0012] If the pre-calibrated external parameters do not exist, collecting calibration acceleration data through the acceleration sensor;

[0013] Constructing an extrinsic rotation matrix according to the calibration acceleration data;

[0014] The extrinsic rotation matrix is ​​used as a pre-calibrated extrinsic parameter, and the photographing posture information of the electronic device is determined based on the three-axis acceleration data and the pre-calibrated extrinsic parameter.

[0015] In some embodiments of the present application, if the pre-calibrated external parameter does not exist, collecting calibrated acceleration data through the acceleration sensor includes:

[0016] If the pre-calibrated external parameters do not exist, outputting a calibration prompt to enable the user to adjust the electronic device to a target posture;

[0017] When the posture of the electronic device is the same as the target posture, calibration acceleration data is collected by the acceleration sensor.

[0018] In some embodiments of the present application, constructing an extrinsic rotation matrix based on the calibrated acceleration data includes:

[0019] Calculating a standard deviation of the calibrated acceleration data, and determining whether the electronic device is in a stationary state according to the standard deviation of the calibrated acceleration data;

[0020] If the electronic device is in a stationary state, an average value of the calibration acceleration data is obtained, and the average value is normalized to calculate the deviation angle corresponding to the target posture;

[0021] The extrinsic rotation matrix is ​​constructed by combining the offset angle corresponding to the target posture.

[0022] In some embodiments of the present application, determining the photographing posture information of the electronic device based on the three-axis acceleration data and pre-calibrated external parameters includes:

[0023] Performing modulus calculation on the three-axis acceleration data to obtain modulus values ​​corresponding to the three-axis acceleration data;

[0024] Comparing the modulus corresponding to the three-axis acceleration data with a preset static threshold to determine whether the electronic device is in a static state;

[0025] When the electronic device is in a stationary state, constructing a matrix according to the module values ​​corresponding to the three-axis acceleration data to obtain a three-axis acceleration matrix;

[0026] The three-axis acceleration matrix is ​​multiplied by the pre-calibrated external parameters to obtain a device deviation angle, and the device deviation angle is used as the photographing posture information of the electronic device.

[0027] In some embodiments of the present application, if the photographing posture information does not match the standard posture information pre-associated with the photographing scene identifier, before outputting a photographing posture adjustment prompt, the method includes:

[0028] Obtaining standard posture information corresponding to the photographing scene identifier;

[0029] Comparing the device deviation angle in the photographing posture information with the standard angle in the standard posture information;

[0030] If the difference between the device deviation angle and the standard angle does not exceed a preset angle deviation threshold, determining that the photographing posture information matches the standard posture information associated with the photographing scene identifier;

[0031] If the difference between the device deviation angle and the standard angle exceeds a preset angle deviation threshold, it is determined that the photographing posture information does not match the standard posture information associated with the photographing scene identifier.

[0032] In some embodiments of the present application, when detecting that the shooting device is started, obtaining a photographing scene identifier includes:

[0033] When detecting that the shooting device is started, determining whether there is a photo taking task;

[0034] If there is a photo-taking task, obtain a photo-taking scene identifier associated with the photo-taking task;

[0035] If there is no photo-taking task, output a prompt for selecting a photo-taking scene;

[0036] After the user selects a shooting scene, the shooting scene identifier selected by the user is obtained.

[0037] In some embodiments of the present application, after outputting a prompt for selecting a photo scene if no photo task exists, the method further includes:

[0038] If the user-selected photographing scene identifier is not obtained within the preset time interval, capturing an initial scene image by the photographing device;

[0039] Comparing the initial scene image with preset scene images in a preset image set;

[0040] A target scene image having the highest similarity to the initial scene image is obtained, and a scene identifier corresponding to the target scene image is used as a photographing scene identifier.

[0041] In some embodiments of the present application, if the photographing posture information does not match the standard posture information pre-associated with the photographing scene identifier, after outputting a photographing posture adjustment prompt, the method further includes:

[0042] Acquiring an image of a product captured by the camera;

[0043] parsing the product image to obtain product shooting information of the product image;

[0044] Comparing the product shooting information with the product standard information corresponding to the photographing scene identifier;

[0045] If the product shooting information does not match the product standard information corresponding to the photographing scene identifier, outputting a second shooting prompt;

[0046] If the product shooting information matches the product standard information corresponding to the photographing scene identifier, the product image is uploaded to a preset inspection platform.

[0047] On the other hand, the present application provides a device for prompting a photographing posture, wherein the device is provided in an electronic device, wherein the electronic device includes a photographing device and an acceleration sensor, and the device includes:

[0048] A scene acquisition module, configured to acquire a photographing scene identifier when detecting that the photographing device is started;

[0049] An acceleration acquisition module, configured to receive a user's camera operation instruction and collect three-axis acceleration data of the electronic device through the acceleration sensor;

[0050] a posture determination module, configured to determine photographing posture information of the electronic device based on the three-axis acceleration data and pre-calibrated extrinsic parameters, wherein the extrinsic parameters refer to angular parameters of a camera and an acceleration sensor in the electronic device on a mounting structure of the electronic device;

[0051] The posture prompt module is used to output a prompt for adjusting the photographing posture if the photographing posture information does not match the standard posture information pre-associated with the photographing scene identifier.

[0052] In some embodiments of the present application, the photographing posture prompting device further includes:

[0053] An external parameter judgment module is used to judge whether there are pre-calibrated external parameters;

[0054] an acceleration acquisition module, configured to acquire calibrated acceleration data through the acceleration sensor if the pre-calibrated external parameters do not exist;

[0055] The matrix construction module is used to construct an extrinsic parameter rotation matrix according to the calibrated acceleration data; and use the extrinsic parameter rotation matrix as a pre-calibrated extrinsic parameter.

[0056] In some embodiments of the present application, the acceleration acquisition module is specifically used to:

[0057] If the pre-calibrated external parameters do not exist, outputting a calibration prompt to enable the user to adjust the electronic device to a target posture;

[0058] When the posture of the electronic device is the same as the target posture, collecting calibration acceleration data through the acceleration sensor;

[0059] The matrix building module is specifically used for:

[0060] Calculating a standard deviation of the calibrated acceleration data, and determining whether the electronic device is in a stationary state according to the standard deviation of the calibrated acceleration data;

[0061] If the electronic device is in a stationary state, an average value of the calibration acceleration data is obtained, and the average value is normalized to calculate the deviation angle corresponding to the target posture;

[0062] The extrinsic rotation matrix is ​​constructed by combining the offset angle corresponding to the target posture.

[0063] In some embodiments of the present application, the posture determination module is specifically used to:

[0064] Performing modulus calculation on the three-axis acceleration data to obtain modulus values ​​corresponding to the three-axis acceleration data;

[0065] Comparing the modulus corresponding to the three-axis acceleration data with a preset static threshold to determine whether the electronic device is in a static state;

[0066] When the electronic device is in a stationary state, constructing a matrix according to the module values ​​corresponding to the three-axis acceleration data to obtain a three-axis acceleration matrix;

[0067] The three-axis acceleration matrix is ​​multiplied by the pre-calibrated external parameters to obtain a device deviation angle, and the device deviation angle is used as the photographing posture information of the electronic device.

[0068] In some embodiments of the present application, the posture prompt module is specifically used to:

[0069] Obtaining standard posture information corresponding to the photographing scene identifier;

[0070] Comparing the device deviation angle in the photographing posture information with the standard angle in the standard posture information;

[0071] If the difference between the device deviation angle and the standard angle does not exceed a preset angle deviation threshold, determining that the photographing posture information matches the standard posture information associated with the photographing scene identifier;

[0072] If the difference between the device deviation angle and the standard angle exceeds a preset angle deviation threshold, it is determined that the photographing posture information does not match the standard posture information associated with the photographing scene identifier.

[0073] In some embodiments of the present application, the scene acquisition module is specifically used to:

[0074] When detecting that the shooting device is started, determining whether there is a photo taking task;

[0075] If there is a photo-taking task, obtain a photo-taking scene identifier associated with the photo-taking task;

[0076] If there is no photo-taking task, output a prompt for selecting a photo-taking scene;

[0077] After the user selects a shooting scene, the shooting scene identifier selected by the user is obtained.

[0078] In some embodiments of the present application, the scene acquisition module is specifically used to:

[0079] If the user-selected photographing scene identifier is not obtained within the preset time interval, capturing an initial scene image by the photographing device;

[0080] Comparing the initial scene image with preset scene images in a preset image set;

[0081] A target scene image having the highest similarity to the initial scene image is obtained, and a scene identifier corresponding to the target scene image is used as a photographing scene identifier.

[0082] In some embodiments of the present application, the photographing posture prompting device further includes:

[0083] an acquisition and analysis module, configured to acquire the commodity image captured by the camera;

[0084] parsing the product image to obtain product shooting information of the product image;

[0085] An information comparison module, configured to compare the product shooting information with the product standard information corresponding to the photographing scene identifier;

[0086] a shooting prompt module, configured to output a second shooting prompt if the product shooting information does not match the product standard information corresponding to the photographing scene identifier;

[0087] The image uploading module is used to upload the product image to a preset inspection platform when the product shooting information matches the product standard information corresponding to the photographing scene identifier.

[0088] On the other hand, the present application further provides an electronic device, comprising:

[0089] Filming device;

[0090] Accelerometer;

[0091] one or more processors;

[0092] Memory; and

[0093] One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the photographing gesture prompting method.

[0094] On the other hand, the present application also provides a computer-readable storage medium on which a computer program is stored, and the computer program is loaded by a processor to execute the photo gesture prompt method.

[0095] In the embodiment of the present application, the external parameters corresponding to the electronic device are pre-calibrated, the photo scene identification is determined when the photo is taken, the three-axis acceleration data is collected by the acceleration sensor in the electronic device, and the acceleration data is combined with the external parameters to obtain the photo posture information. Finally, the photo posture information is compared with the standard posture information corresponding to the photo scene identification to achieve the photo posture prompt. In the embodiment of the present application, when providing the photo posture prompt, a large amount of data analysis and processing is not required. The photo posture information is accurately determined by fusing the pre-calibrated external parameters and the three-axis acceleration data, making the photo posture prompt efficient and accurate. In addition, the present application can pre-set the standard posture information corresponding to a variety of photo scene identifications to achieve multi-scene photo posture prompts, making the photo posture prompts have a wide range of applications. In addition, the embodiment of the present application does not require excessive user operations, has low hardware requirements for the device, and reduces device costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0096] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0097] Figure 1 This is a schematic diagram of a scene for providing a photo-taking gesture prompt according to an embodiment of the present application;

[0098] Figure 2 This is a flow chart of an embodiment of the method for prompting a photographic posture provided in the embodiments of the present application;

[0099] Figure 3 This is a flow chart of an embodiment of the method for prompting a photographic posture provided in the embodiments of the present application;

[0100] Figure 4 This is a schematic diagram of a specific scenario of an embodiment of pre-calibrating extrinsic parameters in the method for prompting a photographic posture provided in an embodiment of the present application;

[0101] Figure 5 This is a schematic structural diagram of an embodiment of a device for prompting a photographic posture provided in an embodiment of the present application;

[0102] Figure 6 It is a schematic diagram of the structure of an embodiment of an electronic device provided in the embodiments of the present application. DETAILED DESCRIPTION

[0103] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of this application.

[0104] In the description of the present application, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present application. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the said features. In the description of the present application, "multiple" means two or more, unless otherwise clearly and specifically defined.

[0105] In this application, the word "exemplary" is used to mean "serving as an example, illustration, or illustration." Any embodiment described in this application as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments. The following description is given to enable any person skilled in the art to implement and use the present application. In the following description, details are listed for the purpose of explanation. It should be understood that one of ordinary skill in the art can recognize that the present application can be implemented without using these specific details. In other instances, well-known structures and processes are not elaborated in detail to avoid obscuring the description of the present application with unnecessary details. Therefore, the present application is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in this application.

[0106] The embodiments of the present application provide a method, apparatus, device, and computer-readable storage medium for prompting a photographing posture, which are described in detail below.

[0107] The photo-taking gesture prompting method in the embodiment of the present application is applied to a photo-taking gesture prompting device, which is arranged in an electronic device. The electronic device is provided with one or more processors, memories, and one or more applications, wherein the one or more applications are stored in the memories and configured to be executed by the processor to implement the photo-taking gesture prompting method; the electronic device can be a mobile terminal, such as a mobile phone, a tablet computer or a camera.

[0108] like Figure 1 As shown, Figure 1This is a schematic diagram of a scene for photo gesture prompting in an embodiment of the present application. The photo gesture prompting scene in the embodiment of the present application includes an electronic device 100 (a photo gesture prompting device is integrated in the electronic device 100), and a computer-readable storage medium corresponding to the photo gesture prompting is run in the electronic device 100 to execute the photo gesture prompting.

[0109] It is understandable that Figure 1 The electronic devices in the scene of the photo gesture prompt shown, or the devices included in the electronic devices, do not constitute a limitation on the embodiments of the present application. That is, the number of devices and types of devices included in the scene of the photo gesture prompt, or the number and types of devices included in each device do not affect the overall implementation of the technical solution in the embodiments of the present application, and can all be regarded as equivalent replacements or derivatives of the technical solution claimed to be protected in the embodiments of the present application.

[0110] In the embodiment of the present application, the electronic device 100 is mainly used to obtain a photo scene identification when it is detected that the shooting device is started; receive a user's photo operation instruction, and collect three-axis acceleration data of the electronic device through the acceleration sensor; determine the photo posture information of the electronic device based on the three-axis acceleration data and pre-calibrated external parameters, and the external parameters refer to the angle parameters of the shooting device and the acceleration sensor in the electronic device on the mounting structure of the electronic device; if the photo posture information does not match the standard posture information pre-associated with the photo scene identification, output a photo posture adjustment prompt.

[0111] In the embodiment of the present application, the electronic device 100 can be an independent electronic device, or it can be an electronic device network or electronic device cluster composed of multiple electronic devices. For example, the electronic device 100 described in the embodiment of the present application includes but is not limited to a terminal (such as a mobile phone or a tablet computer) and a computer. The electronic device in the embodiment of the present application includes a shooting device and an acceleration sensor. The position of the electronic device itself and the shooting device and the acceleration sensor in the electronic device is relatively fixed, so that the movement conditions of the electronic device itself and the shooting device and the acceleration sensor in the electronic device are the same. Therefore, the three-axis acceleration data of the acceleration sensor, the three-axis acceleration data of the electronic device itself and the three-axis acceleration data of the shooting device in the electronic device are the same, and the three-axis acceleration data of the acceleration sensor can be used to represent the three-axis acceleration data of the electronic device and the shooting device; similarly, the posture information of the acceleration sensor, the posture information of the electronic device itself and the posture information of the shooting device in the electronic device are the same, and the posture information of the acceleration sensor can be used to represent the electronic device and the shooting posture information of the shooting device in the electronic device.

[0112] Those skilled in the art will understand that the electronic device includes a camera and an acceleration sensor. In addition, the electronic device may also include other devices or components, for example, a memory, a processor, a display device, a communication device, etc. The type of electronic device and the specific devices and components in the electronic device are not limited in this embodiment. The electronic device in this embodiment can be a mobile phone, a tablet, an e-reader, etc.

[0113] Those skilled in the art will understand that Figure 1 The application environment shown in the figure is only one application scenario of the present application solution and does not constitute a limitation on the application scenario of the present application solution. Other application environments may also include Figure 1 More or fewer electronic devices shown in, or electronic device network connection relationships, such as Figure 1 Only one electronic device is shown. It can be understood that the scene of the photo gesture prompt may also include one or more other electronic devices, which are not limited here. The electronic device 100 may also include a memory for storing data, for example, storing photos taken.

[0114] In addition, in the scenario of the photo gesture prompt of the present application, the electronic device 100 can be provided with a display device, and the display device can be directly provided in the electronic device, such as the display screen of the electronic device, the bar gun in the logistics scenario, etc. The electronic device 100 can access the backend database 200 (the backend database can be in the local memory of the electronic device, or the backend database can be set in the cloud), and the backend database 200 stores information related to the photo gesture prompt, for example, the backend database 200 stores photos that match the photo gesture information.

[0115] It should be noted that Figure 1 The schematic diagram of the scene of the photo posture prompt shown is only an example. The scene of the photo posture prompt described in the embodiment of the present application is to more clearly illustrate the technical solution of the embodiment of the present application, and does not constitute a limitation on the technical solution provided by the embodiment of the present application.

[0116] like Figure 2 As shown, Figure 2 201 to 203:

[0117] 201. When it is detected that the photographing device is started, obtain a photographing scene identifier.

[0118] The electronic device includes a camera and an acceleration sensor; in addition, the electronic device may also include other devices or components, for example, a memory, a processor, a display device, and a communication device, etc. The type of electronic device in this embodiment and the specific devices and components in the electronic device are not limited. The electronic device in this embodiment can be a mobile phone, a tablet, an e-reader, etc. The camera refers to a component used for taking pictures, for example, a camera device; the acceleration sensor is a device that can measure the three-axis attitude angle and acceleration of an object. The acceleration sensor is usually composed of a mass block, a damper, an elastic element, a sensitive element, and an adaptive circuit. The principle of the acceleration sensor measuring the three-axis attitude angle and acceleration of an object is: during the acceleration process, by measuring the inertial force exerted on the mass block, the acceleration value is obtained using Newton's second law, thereby realizing acceleration data measurement. The type of acceleration sensor in the embodiment of the present application is not specifically limited. That is, depending on the different sensitive elements of the sensor, common acceleration sensors include capacitive, inductive, strain, piezoresistive, piezoelectric, etc. The acceleration sensor in the embodiment of the present application can be an inertial measurement unit (IMU).

[0119] In an embodiment of the present application, the electronic device can monitor the status of the camera in real time and determine whether the camera is started. In the embodiment of the present application, the method of starting the camera is not specifically limited, that is, the user manually starts the camera, for example, the user clicks the camera icon in the display interface of the electronic device to start the camera; in addition, the start of the camera can also be automatically triggered by the electronic device, for example, the electronic device is pre-set to automatically start the camera to scan the object when performing the object recognition function.

[0120] When the electronic device detects that the shooting device is started, the electronic device obtains the photo scene identifier. The photo scene identifier refers to identification information that uniquely determines the photo scene. The specific content and format of the photo scene identifier are not limited, for example, the photo scene number.

[0121] There is no limitation on the implementation scenario in which the electronic device obtains the photo scene identifier. For example, in one specific implementation scenario, the electronic device outputs a photo scene selection list when the shooting device is started, and prompts the user to select a photo scene identifier from the photo scene list; for example, in another specific implementation scenario, the user inputs a photo task in the electronic device, and the electronic device selects a photo scene identifier according to the photo task input by the user.

[0122] For ease of understanding, this embodiment is described by way of example. When the electronic device detects that the camera is activated, it outputs an interactive interface including a photographing scene number definition table, allowing the user to select a photographing scene identifier on the interactive interface. The following Table 1 shows the photographing scene number definition table output on the display interface of the electronic device:

[0123] Table 1 Photographing scene number definition table

[0124]

[0125] Among them, vertical scenes refer to scenes that require electronic devices (such as mobile phones) to shoot vertically or horizontally, including merchandise shelves, vertical counters, vertical freezers and other vertical scenes; for example, 1116 corresponds to a certain brand of instant noodles on merchandise shelves, 1213 corresponds to a certain brand of liquor on vertical counters, etc.; horizontal scenes refer to scenes that require electronic devices (such as mobile phones) to shoot from above, including horizontal exhibition stands, horizontal freezers and other horizontal scenes; for example, 2115 corresponds to a certain brand of jewelry on a horizontal exhibition stand; scenes with no requirements refer to scenes that do not require any constraints on the photo posture, such as 3017 corresponding to an offline promotion meeting audit for a certain brand that does not require any constraints on the photo posture.

[0126] In the embodiment of the present application, after obtaining the photographing scene identifier, the electronic device provides a photographing posture prompt based on the photographing scene identifier, specifically:

[0127] 202. Receive a photo-taking operation instruction from a user, and collect three-axis acceleration data of the electronic device through the acceleration sensor.

[0128] Specifically, the electronic device receives a photo-taking operation instruction from the user (the triggering method of the photo-taking operation instruction is not specifically limited), and the electronic device collects three-axis acceleration data through the acceleration sensor, wherein the three-axis acceleration data refers to the acceleration information of the electronic device in the spatial rectangular coordinate system, and the size of the three-axis acceleration data is not limited.

[0129] It can be understood that the three-axis acceleration data collected by the acceleration sensor is actually the three-axis acceleration data of the acceleration sensor, but the electronic device includes a shooting device and an acceleration sensor, and the relative positions of the electronic device itself and the shooting device and the acceleration sensor in the electronic device are fixed. The movement conditions of the electronic device itself and the shooting device and the acceleration sensor in the electronic device are the same. Therefore, the three-axis acceleration data of the acceleration sensor, the three-axis acceleration data of the electronic device itself and the three-axis acceleration data of the shooting device in the electronic device are the same, and the three-axis acceleration data of the acceleration sensor can be used to represent the three-axis acceleration data of the electronic device and the shooting device; in the embodiment of the present application, the electronic device collects three-axis acceleration data through the acceleration sensor in order to determine the posture information of the electronic device at the time of shooting, so that the taken photos meet the requirements.

[0130] 203. Determine photographing posture information of the electronic device based on the three-axis acceleration data and pre-calibrated external parameters.

[0131] The electronic device in the embodiment of the present application includes pre-calibrated extrinsic parameters, wherein the extrinsic parameters refer to the angular parameters of the camera and the acceleration sensor in the electronic device on the mounting structure of the electronic device. The pre-calibrated extrinsic parameters refer to parameter information pre-calibrated to determine the photographing posture information. For example, several fixed positions are first determined in the electronic device, and the angular parameters of the camera and the acceleration sensor corresponding to the fixed positions are normalized to determine the extrinsic parameters; specifically, the extrinsic parameters can be obtained based on the angular parameters of the camera and the acceleration sensor corresponding to the preset multiple fixed positions.

[0132] It can be understood that the posture information of the acceleration sensor is determined by the three-axis acceleration data and the pre-calibrated external parameters, but the electronic device includes a shooting device and an acceleration sensor, and the relative positions of the electronic device itself and the shooting device and the acceleration sensor in the electronic device are fixed. The posture information of the electronic device itself and the shooting device and the acceleration sensor in the electronic device are the same. Therefore, the posture information of the acceleration sensor, the posture information of the electronic device itself and the posture information of the shooting device in the electronic device are the same. The posture information of the acceleration sensor can be used to represent the electronic device and the shooting posture information of the shooting device in the electronic device.

[0133] When determining the photographing posture information in the embodiment of the present application, the electronic device collects three-axis acceleration data through the acceleration sensor, and then processes the three-axis acceleration data and the pre-calibrated external parameters to determine the photographing posture information. In a specific embodiment, the external parameters can refer to the m*n rotation matrix from the acceleration sensor coordinate system to the shooting device coordinate system (such as the mobile phone camera coordinate system), where m and n are positive integers, such as a 3*3 rotation matrix.

[0134] The electronic device performs a product operation based on the three-axis acceleration data and the pre-calibrated external parameters to obtain the electronic device's photographing posture information, specifically including:

[0135] (1) performing modulus calculation on the three-axis acceleration data to obtain the modulus value corresponding to the three-axis acceleration data;

[0136] (2) comparing the modulus value corresponding to the three-axis acceleration data with a preset static threshold value to determine whether the electronic device is in a stationary state;

[0137] (3) When the electronic device is in a stationary state, construct a matrix according to the module values ​​corresponding to the three-axis acceleration data to obtain a three-axis acceleration matrix;

[0138] (4) Multiplying the three-axis acceleration matrix and the pre-calibrated external parameters to obtain a device deviation angle, and using the device deviation angle as the photographing posture information of the electronic device.

[0139] Specifically, the electronic device calculates the modulus (the modulus of acceleration refers to the value of acceleration, that is, the magnitude of acceleration) and normalizes the three-axis acceleration data to obtain the modulus value corresponding to the three-axis acceleration data; then, the electronic device compares the modulus value corresponding to the three-axis acceleration data with a preset static threshold (the preset static threshold refers to the modulus value of the three-axis acceleration when the electronic device is in a state similar to stillness, and the preset static threshold can be flexibly set according to specific requirements) to determine whether the electronic device is in a still state; that is, if the modulus value corresponding to the three-axis acceleration data is less than the preset static threshold, it is determined that the electronic device is in a still state; if the modulus value corresponding to the three-axis acceleration data is greater than or equal to the preset static threshold, it is determined that the electronic device is in a non-stationary state.

[0140] When the electronic device is in a non-stationary state, a prompt message is output to prompt the user to maintain a photographic posture and keep the device still, and then perform a photographic posture information confirmation operation.

[0141] When the electronic device is in a stationary state, the electronic device constructs a matrix based on the modulus values ​​corresponding to the three-axis acceleration data to obtain a three-axis acceleration matrix; that is, the electronic device performs an arc cosine calculation on the modulus values ​​corresponding to the normalized three-axis acceleration data to construct a matrix with three rows and one column. The electronic device multiplies the three-axis acceleration matrix by pre-calibrated external parameters to obtain a device deviation angle (the device deviation angle includes pitch deviation and roll deviation) for a photo posture prompt. The electronic device characterizes the photo posture information through the device deviation angle.

[0142] Specifically, in the embodiment of the present application, the photographing posture information of the electronic device is determined based on the three-axis acceleration data and the pre-calibrated external parameters. The following example illustrates:

[0143] First, the electronic device modulo-calculates the three-axis acceleration data collected by the acceleration sensor to obtain the modulus value corresponding to the three-axis acceleration data, and normalizes the modulus value corresponding to the three-axis acceleration data to facilitate subsequent calculations. Specifically:

[0144]

[0145]

[0146] Among them, acc norm is the modulus of the three-axis acceleration data, is the square of the X-axis acceleration data, is the square of the Y-axis acceleration data, is the square of the Z-axis acceleration data, acc normj The modulus of the three-axis acceleration data obtained by normalization;

[0147] Then, the electronic device converts the modulus value acc_norm corresponding to the three-axis acceleration data j and the preset static threshold Thr norm Compare to determine whether the electronic device is in a static state; when the modulus value corresponding to the three-axis acceleration data is less than the preset static threshold, that is, bool Flag allow The result of judging the status of the electronic device, bool Flag allow =acc norm <Thr norm , Thr norm Flag is the preset static threshold for the accelerometer's approximate static judgment. allow When true, perform posture solving; Flag static When it is false, the attitude solution and the determination result are not updated to the modulus value of the three-axis acceleration data, that is, bool Flag allow =acc norm <Thr norm The electronic device is approximately stationary, allowing posture solution to obtain the electronic device's photo posture information; conversely, when the modulus value corresponding to the three-axis acceleration data is greater than or equal to the preset static threshold, that is, boolFlag allow =acc norm ≥Thr norm The device is shaking and the attitude solution is not performed.

[0148] Finally, when the electronic device determines that the device is approximately stationary, the device deviation angle is solved and the device deviation angle is used to represent the photographing posture information of the electronic device; for example,

[0149]

[0150] in, It is used to characterize the device deviation angle of electronic equipment. The three parameters represent the device deviation angles in the x-axis, y-axis, and z-axis directions respectively. R is an external parameter. The matrix obtained by finding the inverse trigonometric function of the modulus value corresponding to the three-axis acceleration data, that is, acos is the inverse trigonometric function, is the modulus value corresponding to the X-axis acceleration data, is the modulus value corresponding to the Y-axis acceleration data, The modulus value corresponding to the Z-axis acceleration data.

[0151] In the implementation of this application, after determining the photographing posture information of the electronic device through the three-axis acceleration data and the pre-calibrated external parameters, the electronic device determines whether the photographing posture information conforms to the standard posture information corresponding to the photographing scene identifier, specifically including:

[0152] (1) Obtain standard posture information corresponding to the photographing scene identifier.

[0153] Comparing the device deviation angle in the photographing posture information with the standard angle in the standard posture information;

[0154] (2) If the difference between the device deviation angle and the standard angle does not exceed a preset angle deviation threshold, it is determined that the photographing posture information matches the standard posture information associated with the photographing scene identifier.

[0155] (3) If the difference between the device deviation angle and the standard angle exceeds a preset angle deviation threshold, it is determined that the photographing posture information does not match the standard posture information associated with the photographing scene identifier.

[0156] The electronic device obtains the standard posture information corresponding to the photographing scene identifier. The standard posture information refers to the posture information corresponding to the photos that meet the requirements under different pre-set photographing scenes. For example, the photographing scene is a horizontal cabinet, and the standard shooting angle information corresponding to the standard posture information is (90, 0, 90); the electronic device compares the device deviation angle in the photographing posture information with the standard angle in the standard posture information; if the difference between the device deviation angle and the standard angle does not exceed the preset angle deviation threshold (the preset angle deviation threshold refers to the preset photographing posture information comparison critical value. If the preset angle deviation threshold is exceeded, the photographed photo is not standardized and a prompt is required to be given if the photo is not allowed. Conversely, the preset angle deviation threshold can be flexibly set according to the photographing requirements), the electronic device determines that the photographing posture information matches the standard posture information associated with the photographing scene identifier; if the difference between the device deviation angle and the standard angle exceeds the preset angle deviation threshold, it is determined that the photographing posture information does not match the standard posture information associated with the photographing scene identifier.

[0157] Combined with the data in the previous step, for example, the shooting scene can be known through the device number:

[0158] When the shooting scene is a vertical scene, the electronic device determines that the shooting posture information meets the standard posture information corresponding to the shooting scene identifier, and determines that the allowed shooting posture judgment logic is:

[0159] bool Flag=(fabs(Orie z -90) <Thr ang )&&((fabs(Orie x -90) <Thr ang )||(fabs(Orie y -90) <Thr ang ))

[0160] When the shooting scene is a horizontal scene, the electronic device determines that the shooting posture information meets the standard posture information corresponding to the shooting scene identifier, and determines that the allowed shooting posture judgment logic is:

[0161] bool Flag=(fabs(Orie x -90) <Thr ang )&&(fabs(Orie y -90) <Thr ang )

[0162] Among them, bool Flag is the result of the photo posture information judgment, Orie x Orie, the device deviation angle of the X-axis of the electronic device y Device deviation angle of the Y axis of the electronic device, Thr ang To preset the angle deviation threshold, you can set it in advance according to the specific scenario, for example, 15 degrees. When the Flag is true, the photo posture meets the requirement; when the Flag is false, the photo posture does not meet the requirement.

[0163] 204. If the photographing posture information does not match the standard posture information pre-associated with the photographing scene identifier, output a photographing posture adjustment prompt.

[0164] If the photographing posture information does not match the standard posture information associated with the photographing scene identifier, the electronic device outputs a photographing posture adjustment prompt, wherein the content and output method of the photographing posture adjustment prompt are not specifically limited, for example, a voice prompt is used to prompt the user to adjust the photographing posture.

[0165] In the embodiment of the present application, the external parameters corresponding to the electronic device are pre-calibrated, and the photo scene identification is determined when the photo is taken. Then, the three-axis acceleration data is collected by the acceleration sensor in the electronic device, and the acceleration data is combined with the external parameters to obtain the photo posture information. Finally, the photo posture information is compared with the standard posture information corresponding to the photo scene identification to realize the photo posture prompt; in the embodiment of the present application, a large amount of data analysis and processing is not required. The photo posture information is accurately determined by fusing the pre-calibrated external parameters and the three-axis acceleration data, so that the photo posture prompt is efficient and accurate. In addition, in the present application, the standard posture information corresponding to a variety of photo scene identifications can be set to realize multi-scene photo posture prompts, so that the photo posture prompt has a wide range of applicability. In addition, the simple data processing in the present application does not require too much operation by the user, has low hardware requirements for the device, and reduces equipment costs.

[0166] In some embodiments of the present application, in order to prevent the electronic device from not storing the pre-calibrated external parameters, the electronic device cannot accurately determine the photographing posture information, such as Figure 3 As shown, Figure 3This is a flow chart of an embodiment of the method for prompting a photographing posture provided in an embodiment of the present application. After step 202 in the method for prompting a photographing posture in an embodiment of the present application, the electronic device may perform the following steps 301 to 304:

[0167] 301. Determine whether there are pre-calibrated external parameters;

[0168] In this embodiment, the electronic device determines whether there are pre-calibrated external parameters. If there are pre-calibrated external parameters, the electronic device executes step 203 to determine the photographing posture information of the electronic device based on the three-axis acceleration data and the pre-calibrated external parameters, so as to quickly and accurately provide photographing posture prompts.

[0169] 302. If the pre-calibrated external parameters do not exist, collect calibration acceleration data through the acceleration sensor.

[0170] If the pre-calibrated external parameters do not exist, the electronic device collects calibration acceleration data through the acceleration sensor, wherein the calibration acceleration data is the three-axis acceleration data collected by the acceleration sensor in a pre-specified state. For example, the pre-specified state is a stationary state of the electronic device with the display screen facing upward. In this case, the calibration acceleration data is the three-axis acceleration data collected by the acceleration sensor in the stationary state of the electronic device with the display screen facing upward, specifically including:

[0171] (1) If the pre-calibrated external parameters do not exist, outputting a calibration prompt to enable the user to adjust the electronic device to a target posture;

[0172] (2) When the posture of the electronic device is the same as the target posture, calibration acceleration data is collected through the acceleration sensor.

[0173] That is, if there are no pre-calibrated external parameters, the electronic device will output a calibration prompt to prompt the user to perform the calibration operation according to the regulations. For example, the prompt information outputs to place the electronic device horizontally, and the user adjusts the electronic device to the target posture according to the prompt information; the electronic device performs posture detection in real time, and the posture detection can be performed through an acceleration sensor. When it is detected that the electronic device reaches the target posture, the calibration acceleration data is collected through the acceleration sensor.

[0174] 303. Construct an extrinsic rotation matrix based on the calibrated acceleration data.

[0175] A rotation matrix, when multiplied by a vector, changes the vector's direction without changing its magnitude, preserving its chirality. Rotations can be categorized as active and passive. Active rotations rotate a vector counterclockwise around the axis of rotation. Passive rotations rotate the coordinate axis counterclockwise, the inverse of active rotations.

[0176] 304. Use the extrinsic rotation matrix as a pre-calibrated extrinsic parameter, and determine the photographing posture information of the electronic device based on the three-axis acceleration data and the pre-calibrated extrinsic parameter.

[0177] In this embodiment, the electronic device constructs an extrinsic rotation matrix of the electronic device based on the calibration acceleration data, and uses the extrinsic rotation matrix as a calibration extrinsic parameter to determine the photographing posture information. Specifically, the process includes:

[0178] (1) calculating a standard deviation of the calibrated acceleration data, and determining whether the electronic device is in a stationary state based on the standard deviation of the calibrated acceleration data;

[0179] (2) If the electronic device is in a stationary state, an average value of the calibration acceleration data is obtained, and the average value is normalized to calculate the offset angle corresponding to the target posture;

[0180] (3) Constructing an extrinsic rotation matrix by combining the offset angles corresponding to the target postures.

[0181] The electronic device calculates the standard deviation of the calibrated acceleration data, and determines whether the electronic device is in a stationary state based on the standard deviation of the calibrated acceleration data; if the standard deviation of the acceleration data is less than a set threshold (the set threshold is set according to the specific scenario), the electronic device is determined to be in a stationary state; conversely, if the standard deviation of the acceleration data is greater than or equal to the set threshold, the electronic device is determined to be in a non-stationary state, and no calibration operation is performed when the electronic device is in a non-stationary state.

[0182] If the electronic device is in a stationary state, the average value of the calibrated acceleration data is obtained, and the electronic device calculates the offset angle corresponding to the target posture after normalizing the average value; the electronic device constructs the external parameter rotation matrix of the electronic device based on the offset angles corresponding to each target posture, and the electronic device uses the external parameter rotation matrix as a pre-calibrated external parameter to determine the photo posture.

[0183] Reference Figure 4 , Figure 4 This is a schematic diagram of a specific scenario of an embodiment of pre-calibrating external parameters in the method for prompting a photographic posture provided in an embodiment of the present application. In this embodiment, the electronic device performs the following steps (1) to (4) for calibrating external parameters:

[0184] (1) Electronic device output interactive interface: whether to perform external parameter calibration, select "Yes" (the first time you install an application that requires a photo scene (such as a logistics application), you will be forced to select "Yes". After that, you will be forced to select "Yes" every three months. Normally, it will automatically jump to the "No" interface and prompt "How many days are left until the next parameter calibration? No calibration is required this time");

[0185] (2) Place the electronic device with the output device screen facing upward on a horizontal surface (such as the floor or table) and let it sit for 10 seconds;

[0186] (3) Place the electronic device output device at an angle (at any angle) on two horizontal surfaces at different heights (with the short bottom surface of the device completely touching the lower surface) and leave it for 10 seconds;

[0187] (4) The electronic device output device is tilted (angle is not limited) and placed on two planes of different heights that are as horizontal as possible (the long bottom surface of the device is completely in contact with the lower plane), and left to stand for 10 seconds before completing the calibration.

[0188] During the calibration process, the device must be in a stationary state in steps (2), (3), and (4). If the device is not in a stationary state, the algorithm will automatically detect and alarm, and restart the calibration. The device static detection algorithm is that the standard deviation of the three-axis acceleration in seconds is less than the threshold, and the formula is:

[0189]

[0190] The static decision logic is:

[0191] bool Flag static =(std x <Thr acc )&&(std y <Thr acc )&&(std z <Thr acc )

[0192] Among them, acc i is the three-axis acceleration data, j is XYZ, std x 、std y 、std z is the standard deviation of the three-axis acceleration data per second in static state; Thr acc It is the static judgment threshold of the accelerometer, bool Flag static Flag is the result of judging the status of electronic equipment. static When true, the device is in a stationary state; Flag static When false, the device is not stationary.

[0193] The user places the electronic device according to the prompts. The acceleration sensor in the electronic device collects the 10-second static three-axis acceleration data of steps (2), (3), and (4) as calibration acceleration data. The electronic device averages the collected calibration acceleration data, normalizes them, and then calculates the offset angle Ang of the accelerometer relative to the camera in the Z direction in step (2). z , the offset angle Ang of the accelerometer relative to the camera in the X direction in step (3) x and the accelerometer's offset angle Ang in the Y direction relative to the camera in step (4) y , specifically:

[0194] The electronics average the calibrated acceleration data:

[0195]

[0196] Among them, mean acc is the average value of the calibration acceleration data;

[0197] The electronics normalize the calibrated acceleration data:

[0198]

[0199] in, Normalized value for calibration acceleration data;

[0200] The electronic device calculates the offset angle of the acceleration sensor relative to the camera in the Z direction in step (2):

[0201] Ang z is the offset angle of the acceleration sensor relative to the camera in the Z direction;

[0202] The electronic device calculates the offset angle of the acceleration sensor relative to the camera in the X direction in step (3):

[0203] Ang x is the offset angle of the acceleration sensor relative to the camera in the X direction;

[0204] The electronic device calculates the offset angle of the acceleration sensor relative to the camera in the Y direction in step (4):

[0205] Ang y is the offset angle of the acceleration sensor relative to the camera in the Y direction;

[0206] The electronic device combines the offset angles corresponding to each target posture to construct the external parameter rotation matrix R of the electronic device:

[0207]

[0208] This embodiment provides a calibration operation for an electronic device. The pre-calibrated external parameters can be used to quickly determine the photo posture of the electronic device, making the photo posture prompt more efficient and accurate.

[0209] In some embodiments of the present application, in order to facilitate the determination of the photo scene identifier, the embodiment of the present application further includes step 201 in the photo posture prompt method: when it is detected that the shooting device is started, determining whether there is a photo task; if there is a photo task, obtaining the photo scene identifier associated with the photo task; if there is no photo task, outputting a photo scene selection prompt; after the user selects the shooting scene, obtaining the user-selected photo scene identifier; if the user-selected photo scene identifier is not obtained within a preset time interval, collecting an initial scene image through the shooting device; comparing the initial scene image with the preset scene image in the preset image set; obtaining the target scene image with the highest similarity to the initial scene image, and using the scene identifier corresponding to the target scene image as the photo scene identifier.

[0210] In the embodiment of the present application, when the electronic device detects that the shooting device is started, the electronic device determines whether there is a photo-taking task. The content of the photo-taking task is not specifically limited. The photo-taking task may be taking a photo of the xxx counter in the xxx store. If there is a photo-taking task, the electronic device obtains the photo-taking scene identifier associated with the photo-taking task and executes step 202 to finally implement the photo-taking posture prompt.

[0211] If there is no photo-taking task, the electronic device outputs a photo-taking scene selection prompt. After the user selects the photo-taking scene, the electronic device obtains the photo-taking scene identifier selected by the user and executes step 202 to finally realize the photo-taking posture prompt; if the photo-taking scene identifier selected by the user is not obtained within the preset time interval, that is, the user does not select the photo-taking scene identifier, the electronic device collects the initial scene image through the shooting device; the electronic device compares the initial scene image with the preset scene image in the preset image set (the preset image set contains images corresponding to each pre-set scene); the electronic device obtains the target scene image with the highest similarity to the initial scene image, and the electronic device uses the scene identifier corresponding to the target scene image as the photo-taking scene identifier.

[0212] The preset time interval may be a time interval preset by the user, or a default time interval (for example, when the user does not set a time interval), which is not specifically limited here.

[0213] In this embodiment, the electronic device can capture the initial scene image when the user has not selected the photo scene identifier, and automatically determine the photo scene identifier based on the initial scene image, so that the electronic device can automatically determine the scene, thereby determining the standard posture information to provide photo posture prompts.

[0214] In some embodiments of the present application, the electronic device recognizes the captured product image and may prompt a second shot when a shooting error occurs. Specifically: the product image captured by the shooting device is acquired, and the product image is analyzed to obtain product shooting information of the product image; the product shooting information is compared with the product standard information corresponding to the shooting scene identifier; if the product shooting information does not match the product standard information corresponding to the shooting scene identifier, a second shot prompt is output; if the product shooting information matches the product standard information corresponding to the shooting scene identifier, the product image is uploaded to a preset inspection platform.

[0215] Among them, the preset audit platform can be the audit management platform corresponding to the enterprise, such as the audit management platform of a logistics enterprise, etc., and the specific details are not limited here.

[0216] In this embodiment, the electronic device obtains a product image taken by a photographing device, analyzes the product image, and obtains product photographing information of the product image; wherein, the method of product image analysis is not specifically limited. For example, an image recognition model obtained by deep learning is pre-set in the electronic device, and product image recognition is performed through the image recognition model. The electronic device compares the product photographing information with the product standard information corresponding to the photographing scene identifier; if the product photographing information does not match the product standard information corresponding to the photographing scene identifier, the electronic device outputs a second-photography prompt; if the product photographing information matches the product standard information corresponding to the photographing scene identifier, the electronic device uploads the product image to a preset inspection platform.

[0217] In this embodiment, the electronic device can recognize the product image obtained by shooting, upload the product image when it meets the requirements, and prompt to re-shoot when the product image does not meet the requirements. The technical solution in this embodiment makes the photo posture prompt more comprehensive.

[0218] In order to better implement the photographing posture prompting method in the embodiment of the present application, based on the photographing posture prompting method, the embodiment of the present application also provides a photographing posture prompting device, such as Figure 5 As shown, Figure 5 FIG. 5 is a schematic structural diagram of an embodiment of a device for prompting a photographic posture; the device 500 for prompting a photographic posture comprises:

[0219] The scene acquisition module 501 is used to acquire a photographing scene identifier when detecting that the photographing device is started;

[0220] The acceleration acquisition module 502 is configured to receive a user's photo-taking operation instruction and collect three-axis acceleration data of the electronic device through the acceleration sensor;

[0221] a posture determination module 503 for determining the photographing posture information of the electronic device based on the three-axis acceleration data and pre-calibrated extrinsic parameters, wherein the extrinsic parameters refer to angular parameters of the camera and the acceleration sensor in the electronic device on the mounting structure of the electronic device;

[0222] The posture prompt module 504 is configured to output a prompt for adjusting the photographing posture if the photographing posture information does not match the standard posture information pre-associated with the photographing scene identifier.

[0223] In some embodiments of the present application, the photographing posture prompting device further includes:

[0224] An external parameter judgment module is used to judge whether there are pre-calibrated external parameters;

[0225] an acceleration acquisition module, configured to acquire calibrated acceleration data through the acceleration sensor if the pre-calibrated external parameters do not exist;

[0226] A matrix construction module, configured to construct an extrinsic rotation matrix based on the calibration acceleration data;

[0227] The extrinsic parameter rotation matrix is ​​used as a pre-calibrated extrinsic parameter.

[0228] In some embodiments of the present application, the acceleration acquisition module is specifically used to:

[0229] If the pre-calibrated external parameters do not exist, outputting a calibration prompt to enable the user to adjust the electronic device to a target posture;

[0230] When the posture of the electronic device is the same as the target posture, collecting calibration acceleration data through the acceleration sensor;

[0231] The matrix building module is specifically used for:

[0232] Calculating a standard deviation of the calibrated acceleration data, and determining whether the electronic device is in a stationary state according to the standard deviation of the calibrated acceleration data;

[0233] If the electronic device is in a stationary state, an average value of the calibration acceleration data is obtained, and the average value is normalized to calculate the deviation angle corresponding to the target posture;

[0234] The extrinsic rotation matrix is ​​constructed by combining the offset angle corresponding to the target posture.

[0235] In some embodiments of the present application, the posture determination module 503 is specifically used to:

[0236] Performing modulus calculation on the three-axis acceleration data to obtain modulus values ​​corresponding to the three-axis acceleration data;

[0237] Comparing the modulus corresponding to the three-axis acceleration data with a preset static threshold to determine whether the electronic device is in a static state;

[0238] When the electronic device is in a stationary state, constructing a matrix according to the module values ​​corresponding to the three-axis acceleration data to obtain a three-axis acceleration matrix;

[0239] The three-axis acceleration matrix is ​​multiplied by the pre-calibrated external parameters to obtain a device deviation angle, and the device deviation angle is used as the photographing posture information of the electronic device.

[0240] In some embodiments of the present application, the posture prompt module is specifically used to:

[0241] Obtaining standard posture information corresponding to the photographing scene identifier;

[0242] Comparing the device deviation angle in the photographing posture information with the standard angle in the standard posture information;

[0243] If the difference between the device deviation angle and the standard angle does not exceed a preset angle deviation threshold, determining that the photographing posture information matches the standard posture information associated with the photographing scene identifier;

[0244] If the difference between the device deviation angle and the standard angle exceeds a preset angle deviation threshold, it is determined that the photographing posture information does not match the standard posture information associated with the photographing scene identifier.

[0245] In some embodiments of the present application, the scene acquisition module is specifically used to:

[0246] When detecting that the shooting device is started, determining whether there is a photo taking task;

[0247] If there is a photo-taking task, obtain a photo-taking scene identifier associated with the photo-taking task;

[0248] If there is no photo-taking task, output a prompt for selecting a photo-taking scene;

[0249] After the user selects a shooting scene, the shooting scene identifier selected by the user is obtained.

[0250] In some embodiments of the present application, the scene acquisition module is specifically used to:

[0251] If the user-selected photographing scene identifier is not obtained within the preset time interval, capturing an initial scene image by the photographing device;

[0252] Comparing the initial scene image with preset scene images in a preset image set;

[0253] A target scene image having the highest similarity to the initial scene image is obtained, and a scene identifier corresponding to the target scene image is used as a photographing scene identifier.

[0254] In some embodiments of the present application, the photographing posture prompting device further includes:

[0255] an acquisition and analysis module, configured to acquire the commodity image captured by the camera;

[0256] parsing the product image to obtain product shooting information of the product image;

[0257] An information comparison module, configured to compare the product shooting information with the product standard information corresponding to the photographing scene identifier;

[0258] a shooting prompt module, configured to output a second shooting prompt if the product shooting information does not match the product standard information corresponding to the photographing scene identifier;

[0259] The image uploading module is used to upload the product image to a preset inspection platform when the product shooting information matches the product standard information corresponding to the photographing scene identifier.

[0260] In the photographing posture prompting device of this embodiment, the external parameters corresponding to the electronic device are pre-calibrated, and the photographing scene identification is determined when the photo is taken. Then, the three-axis acceleration data is collected by the acceleration sensor in the electronic device, and the acceleration data is combined with the external parameters to obtain the photographing posture information. Finally, the photographing posture information is compared with the standard posture information corresponding to the photographing scene identification to realize the photographing posture prompt. In the embodiment of this application, a large amount of data analysis and processing is not required. The photographing posture information is accurately determined by fusing the pre-calibrated external parameters and the three-axis acceleration data, so that the photographing posture prompt is efficient and accurate. In addition, in this application, the standard posture information corresponding to a variety of photographing scene identifications can be set to realize multi-scene photographing posture prompts, so that the photographing posture prompt has a wide range of applications. In addition, the simple data processing in this application does not require too much operation by the user, and has low hardware requirements for the device, reducing equipment costs.

[0261] The present application also provides an electronic device that integrates any one of the photo-taking gesture prompting devices provided in the present application. The electronic device includes:

[0262] Filming device;

[0263] Accelerometer;

[0264] one or more processors;

[0265] Memory; and

[0266] One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor in the photographing gesture prompting method described in any of the above-mentioned photographing gesture prompting method embodiments.

[0267] like Figure 6 , which shows a schematic diagram of the structure of a specific embodiment of the electronic device involved in the embodiment of the present application. The electronic device may include components such as a processor 601 with one or more processing cores, a memory 602 with one or more computer-readable storage media, a power supply 603, and an input unit 604. Those skilled in the art will understand that Figure 6 The electronic device structure shown in the figure does not constitute a limitation of the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange components differently.

[0268] Processor 601 is the control center of the electronic device, connecting the various parts of the entire electronic device using various interfaces and lines. By running or executing software programs and / or modules stored in memory 602 and accessing data stored in memory 602, it performs various functions of the electronic device and processes data, thereby monitoring the electronic device as a whole. Optionally, processor 601 may include one or more processing cores; preferably, processor 601 may integrate an application processor and a modem processor, wherein the application processor primarily processes the operating system, user interface, and application programs, while the modem processor primarily handles wireless communications. It is understood that the modem processor may not be integrated into processor 601.

[0269] The memory 602 can be used to store software programs and modules. The processor 601 executes various functional applications and data processing by running the software programs and modules stored in the memory 602. The memory 602 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 602 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. Accordingly, the memory 602 may also include a memory controller to provide the processor 601 with access to the memory 602.

[0270] The electronic device also includes a power supply 603 for supplying power to various components. Preferably, the power supply 603 can be logically connected to the processor 601 via a power management system, thereby enabling the power management system to manage charging, discharging, and power consumption. The power supply 603 can also include one or more DC or AC power supplies, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and other arbitrary components.

[0271] The electronic device may further include an input unit 604, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0272] Although not shown, the electronic device may further include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 601 in the electronic device will load the executable files corresponding to the processes of one or more application programs into the memory 602 according to the following instructions, and the processor 601 will run the application programs stored in the memory 602 to implement various functions as follows:

[0273] When detecting that the photographing device is started, obtaining a photographing scene identifier;

[0274] receiving a user's photographing operation instruction, and collecting three-axis acceleration data of the electronic device through the acceleration sensor;

[0275] Determining photographing posture information of the electronic device based on the three-axis acceleration data and pre-calibrated extrinsic parameters, where the extrinsic parameters refer to angular parameters of a photographing device and an acceleration sensor in the electronic device on a mounting structure of the electronic device;

[0276] If the photographing posture information does not match the standard posture information pre-associated with the photographing scene identifier, a photographing posture adjustment prompt is output.

[0277] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be accomplished by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0278] To this end, embodiments of the present application provide a computer-readable storage medium, which may include a read-only memory (ROM), random access memory (RAM), a disk, or an optical disk. A computer program is stored thereon, and the computer program is loaded by a processor to execute any of the methods for prompting a photographic gesture provided in embodiments of the present application. For example, the computer program loaded by the processor may execute the following steps:

[0279] When detecting that the photographing device is started, obtaining a photographing scene identifier;

[0280] receiving a user's photographing operation instruction, and collecting three-axis acceleration data of the electronic device through the acceleration sensor;

[0281] Determining photographing posture information of the electronic device based on the three-axis acceleration data and pre-calibrated extrinsic parameters, where the extrinsic parameters refer to angular parameters of a photographing device and an acceleration sensor in the electronic device on a mounting structure of the electronic device;

[0282] If the photographing posture information does not match the standard posture information pre-associated with the photographing scene identifier, a photographing posture adjustment prompt is output.

[0283] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, please refer to the detailed description of other embodiments above and will not be repeated here.

[0284] In specific implementation, the above units or structures can be implemented as independent entities, or can be arbitrarily combined to implement as the same or several entities. The specific implementation of the above units or structures can refer to the previous method embodiments and will not be repeated here.

[0285] The specific implementation of the above operations can be found in the previous embodiments and will not be repeated here.

[0286] The above is a detailed introduction to a method, device, equipment and computer-readable storage medium for prompting a photographing posture provided by an embodiment of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present application.

Claims

1. A method for prompting a photographing posture, characterized in that: Applied to an electronic device, the electronic device includes a camera and an acceleration sensor, and the photo gesture prompt method includes: When detecting that the photographing device is started, obtaining a photographing scene identifier, wherein the photographing scene identifier refers to identification information for determining a photographing scene, and the photographing scene is an inspection scene; receiving a user's photographing operation instruction, and collecting three-axis acceleration data of the electronic device through the acceleration sensor; Determine whether there are pre-calibrated external parameters; If the pre-calibrated external parameters do not exist, collecting calibration acceleration data through the acceleration sensor; Constructing an extrinsic rotation matrix according to the calibration acceleration data; Using the extrinsic rotation matrix as the pre-calibrated extrinsic parameter; Determining the photographing posture information of the electronic device based on the three-axis acceleration data and pre-calibrated extrinsic parameters, wherein the extrinsic parameters refer to angular parameters of the camera device and the acceleration sensor in the electronic device on the mounting structure of the electronic device, including: performing modulus calculation on the three-axis acceleration data to obtain a modulus value corresponding to the three-axis acceleration data; comparing the modulus value corresponding to the three-axis acceleration data with a preset static threshold to determine whether the electronic device is in a stationary state; when the electronic device is in a stationary state, constructing a matrix based on the modulus value corresponding to the three-axis acceleration data to obtain a three-axis acceleration matrix; performing a product operation on the three-axis acceleration matrix and the pre-calibrated extrinsic parameters to obtain a device deviation angle, and using the device deviation angle as the photographing posture information of the electronic device; If the photographing posture information does not match the standard posture information pre-associated with the photographing scene identifier, a photographing posture adjustment prompt is output.

2. The method for prompting a photographic posture according to claim 1, wherein: If the pre-calibrated external parameters do not exist, collecting calibration acceleration data through the acceleration sensor includes: If the pre-calibrated external parameters do not exist, outputting a calibration prompt to enable the user to adjust the electronic device to a target posture; When the posture of the electronic device is the same as the target posture, calibration acceleration data is collected by the acceleration sensor.

3. The method for prompting a photographic posture according to claim 1, wherein: The step of constructing an external parameter rotation matrix according to the calibration acceleration data includes: Calculating the standard deviation of the calibration acceleration data; determining whether the electronic device is in a stationary state according to a standard deviation of the calibrated acceleration data; If the electronic device is in a stationary state, obtaining an average value of the calibration acceleration data; Normalizing the average value to calculate the deviation angle corresponding to the target posture; The extrinsic rotation matrix is ​​constructed by combining the offset angle corresponding to the target posture.

4. The method for prompting a photographic posture according to claim 1, wherein: If the photographing posture information does not match the standard posture information pre-associated with the photographing scene identifier, before outputting a photographing posture adjustment prompt, the method further includes: Obtaining standard posture information corresponding to the photographing scene identifier; Comparing the device deviation angle in the photographing posture information with the standard angle in the standard posture information; If the difference between the device deviation angle and the standard angle does not exceed a preset angle deviation threshold, determining that the photographing posture information matches the standard posture information associated with the photographing scene identifier; If the difference between the device deviation angle and the standard angle exceeds a preset angle deviation threshold, it is determined that the photographing posture information does not match the standard posture information associated with the photographing scene identifier.

5. The method for prompting a photographic posture according to claim 1, wherein: The step of obtaining a photographing scene identifier when detecting that the photographing device is started includes: When detecting that the shooting device is started, determining whether there is a photo taking task; If there is a photo-taking task, obtain a photo-taking scene identifier associated with the photo-taking task; If there is no photo-taking task, output a prompt for selecting a photo-taking scene; After the user selects a shooting scene, the shooting scene identifier selected by the user is obtained.

6. The method for prompting a photographic posture according to claim 5, wherein: If no photo-taking task exists, after outputting a prompt for selecting a photo-taking scene, the method further includes: If the user-selected photographing scene identifier is not obtained within the preset time interval, capturing an initial scene image by the photographing device; Comparing the initial scene image with preset scene images in a preset image set; A target scene image having the highest similarity to the initial scene image is obtained, and a scene identifier corresponding to the target scene image is used as a photographing scene identifier.

7. The method for prompting a photographing posture according to any one of claims 1 to 6, characterized in that: If the photographing posture information does not match the standard posture information pre-associated with the photographing scene identifier, after outputting a photographing posture adjustment prompt, the method includes: Acquiring an image of a product captured by the camera; parsing the product image to obtain product shooting information of the product image; Comparing the product shooting information with the product standard information corresponding to the photographing scene identifier; When the product shooting information does not match the product standard information corresponding to the photographing scene identifier, outputting a second shooting prompt; If the product shooting information matches the product standard information corresponding to the photographing scene identifier, the product image is uploaded to a preset inspection platform.

8. A device for prompting a photographic posture, characterized in that: The photographing posture prompting device is provided in an electronic device, wherein the electronic device includes a photographing device and an acceleration sensor, and the photographing posture prompting device includes: A scene acquisition module, configured to acquire a photographing scene identifier when detecting that the photographing device is activated, wherein the photographing scene identifier refers to identification information for determining a photographing scene, and the photographing scene is an inspection scene; An acceleration acquisition module, configured to receive a user's camera operation instruction and collect three-axis acceleration data of the electronic device through the acceleration sensor; An external parameter judgment module is used to judge whether there are pre-calibrated external parameters; an acceleration acquisition module, configured to acquire calibrated acceleration data through the acceleration sensor if the pre-calibrated external parameters do not exist; A matrix construction module is used to construct an extrinsic parameter rotation matrix according to the calibration acceleration data; and use the extrinsic parameter rotation matrix as a pre-calibrated extrinsic parameter; A posture determination module is configured to determine the photographing posture information of the electronic device based on the three-axis acceleration data and pre-calibrated extrinsic parameters, where the extrinsic parameters refer to angular parameters of the camera device and the acceleration sensor in the electronic device on the mounting structure of the electronic device, including: performing modulus calculation on the three-axis acceleration data to obtain a modulus value corresponding to the three-axis acceleration data; comparing the modulus value corresponding to the three-axis acceleration data with a preset static threshold to determine whether the electronic device is in a stationary state; when the electronic device is in a stationary state, constructing a matrix based on the modulus value corresponding to the three-axis acceleration data to obtain a three-axis acceleration matrix; performing a product operation on the three-axis acceleration matrix and the pre-calibrated extrinsic parameters to obtain a device deviation angle, and using the device deviation angle as the photographing posture information of the electronic device; The posture prompt module is used to output a prompt for adjusting the photographing posture if the photographing posture information does not match the standard posture information pre-associated with the photographing scene identifier.

9. An electronic device, characterized in that: The electronic device comprises: Filming device; Accelerometer; one or more processors; Memory; and One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the photographing gesture prompt method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that A computer program is stored thereon, and the computer program is loaded by a processor to execute the photographing posture prompting method according to any one of claims 1 to 7.

Images