Image acquisition reminder processing method and device

Abstract

The embodiments of this specification provide an image acquisition reminder processing method and device, wherein an image acquisition reminder processing method includes: obtaining a device screen image captured by an image sensor set on the device screen facing the visible surface of an imaging object; performing feature recognition processing on the device screen image; determining a collection reminder for image acquisition on the device screen based on the recognition processing result; and displaying a mirror reminder generated based on the collection reminder based on a display device.

Description

[0001] This patent application is a divisional application of the Chinese patent application with application number CN202210117082.2, application date February 7, 2022, and invention name “Image acquisition reminder processing method and device”. Technical Field

[0002] This document relates to the field of data processing technology, and in particular to an image acquisition reminder processing method and device. Background Art

[0003] With the continuous development of Internet information, smart terminals have gradually become a necessity for users to obtain Internet information in their lives and work. However, in the process of using smart terminals, smart terminals often fall or fall due to various accidents, resulting in screen shattering. If a smart terminal with a broken screen is sent for repair, the screen often cannot be repaired and needs to be replaced. Therefore, how to protect the screen of smart terminals from being broken has gradually become the focus of users. Summary of the invention

[0004] One or more embodiments of the present specification provide an image acquisition reminder processing method, including: obtaining a device screen image acquired by an image sensor set on a device screen facing a visible surface of an imaging object. Performing feature recognition processing on the device screen image. Determining a collection reminder for image acquisition on the device screen according to the recognition processing result. Displaying a mirror reminder generated based on the collection reminder based on a display device.

[0005] One or more embodiments of the present specification provide an image acquisition reminder processing device, including: a device screen image acquisition module, configured to acquire a device screen image acquired by an image sensor set on the device screen facing the visible surface of an imaging object. A feature recognition processing module, configured to perform feature recognition processing on the device screen image. A collection reminder determination module, configured to determine a collection reminder for image collection on the device screen based on the recognition processing result. A mirror reminder display module, configured to display a mirror reminder generated based on the collection reminder based on a display device.

[0006] One or more embodiments of the present specification provide an image acquisition reminder processing device, including: a processor; and a memory configured to store computer executable instructions, wherein when the computer executable instructions are executed, the processor: obtains a device screen image acquired by an image sensor that calls the device screen setting facing the visible surface of an imaging object. Performs feature recognition processing on the device screen image. Determines an acquisition reminder for image acquisition of the device screen according to the recognition processing result. Displays a mirror reminder generated based on the acquisition reminder based on a display device.

[0007] One or more embodiments of the present specification provide a storage medium for storing computer executable instructions, which implement the following process when executed by a processor: obtaining a device screen image captured by an image sensor that calls a device screen setting facing a visible surface of an imaging object. Performing feature recognition processing on the device screen image. Determining a collection reminder for image collection on the device screen based on the recognition processing result. Displaying a mirror reminder generated based on the collection reminder based on a display device. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] In order to more clearly illustrate one or more embodiments of the present specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present specification. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative labor. Figure 1 A processing flow chart of an image acquisition reminder processing method provided by one or more embodiments of this specification; Figure 2 A schematic diagram of a collection reminder page provided in one or more embodiments of this specification; Figure 3 A schematic diagram of a collection reminder page displayed on a visual surface provided in one or more embodiments of this specification; Figure 4 A schematic diagram of a collection reminder page including a mirror reminder provided for one or more embodiments of this specification; Figure 5 A schematic diagram of a collection reminder page including a mirror reminder displayed on a visible surface provided in one or more embodiments of this specification; Figure 6 A processing flow chart of an image acquisition reminder processing method applied to a security project scenario provided by one or more embodiments of this specification; Figure 7 A schematic diagram of an image acquisition reminder processing device provided for one or more embodiments of this specification; Figure 8 A schematic diagram of the structure of an image acquisition reminder processing device provided in one or more embodiments of this specification. DETAILED DESCRIPTION

[0009] In order to enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the technical solutions in one or more embodiments of this specification will be clearly and completely described below in conjunction with the drawings in one or more embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of them. Based on one or more embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this document.

[0010] This specification provides an embodiment of an image acquisition reminder processing method: Reference Figure 1 , which shows a processing flow chart of an image acquisition reminder processing method provided by this embodiment, referring to Figure 2 , which shows a page schematic diagram of a collection reminder page provided by this embodiment, referring to Figure 3 , which shows a page schematic diagram of a collection reminder page displayed in a visual surface provided by this embodiment; refer to Figure 4 , which shows a page schematic diagram of a collection reminder page including a mirror reminder provided by this embodiment; refer to Figure 5 , which shows a schematic diagram of a page of a collection reminder page including a mirror reminder displayed in a visible surface provided by this embodiment; refer to Figure 6 , which shows a processing flow chart of an image acquisition reminder processing method applied to a security project scenario provided by this embodiment.

[0011] Reference Figure 1 The image acquisition reminder processing method provided in this embodiment specifically includes steps S102 to S108.

[0012] Step S102, obtaining a device screen image captured by an image sensor configured on the device screen facing the visible surface of the imaging object.

[0013] The image acquisition reminder processing method provided in this embodiment calls the image sensor set on the device screen to capture the device screen image of the device facing the visible surface of the imaging object, performs feature recognition processing on the device screen image, and issues acquisition reminders for adjusting the position and angle during the image acquisition process based on the recognition processing result, and mirrors the acquisition reminder. The display device displays the acquisition reminder after the mirroring process, thereby improving the clarity and effectiveness of the device screen image.

[0014] The device screen described in this embodiment refers to the integral component of the front face of the device or the screen surface; the device screen image refers to the captured image of the integral component of the front face of the device or the screen surface; the imaging object includes a plane mirror, and the imaging object may also be other objects that can be imaged; the visible surface of the imaging object includes the front face of the plane mirror that can be imaged, and also includes the front face of other objects that can be imaged.

[0015] In specific implementation, in the process of capturing the device screen image, the image sensor set on the device screen is facing the visible surface of the imaging object, and the image sensor is used to capture the device screen image formed on the visible surface of the imaging object. For example, the front camera is used to capture the device screen image formed in the plane mirror.

[0016] When using an image sensor to capture the screen image of the device, a problem of mirror-in-mirror image nesting may occur. In view of this, after detecting that the image sensor is called, this embodiment does not display the image captured by the image sensor, but displays a capture reminder page, or uses a replacement page to update the capture reminder page; optionally, after detecting that the image sensor is called, the capture reminder page is displayed in a layer above the image capture page displayed on the display device; wherein the display device refers to a display screen.

[0017] Optionally, the acquisition reminder page contains a mirror operation reminder text of the operation reminder text for performing device screen image acquisition, and / or the acquisition reminder page contains a mirror operation reminder image of the operation reminder image for performing device screen image acquisition.

[0018] The mirrored operation reminder text refers to text that can be normally read by a user when imaged on the visible surface of the imaged object, and the mirrored operation reminder image refers to an image that can be normally read by a user when imaged on the visible surface of the imaged object. The display device displays the mirrored operation reminder text and / or the mirrored operation reminder image, i.e., the mirrored operation reminder text and / or the mirrored operation reminder image, for the purpose of displaying the operation reminder text and / or the operation reminder image that can be normally read by a user on the visible surface of the imaged object, so as to facilitate the user to view the operation reminder through the imaged object.

[0019] For example, Figure 2 The collection reminder page shown in the collection reminder page, the operation reminder text and / or operation reminder image for the device screen image collection is mirrored, specifically, the display device includes 201 mirror operation reminder text "Please face the phone screen to the mirror" and 202 mirror operation reminder image; Figure 3 The collection reminder page displayed in the visible surface shown is specifically displayed in the plane mirror Figure 2The collection reminder page is the mirrored operation reminder text and / or mirrored operation reminder image after mirroring, that is, the operation reminder text and / or operation reminder image that can be read normally by the user, specifically including 301 operation reminder text "Please face the phone screen to the mirror" and 302 operation reminder image.

[0020] Step S104: performing feature recognition processing on the device screen image.

[0021] As described above, the device screen image captured by the image sensor set by the device screen facing the visible surface of the imaging object is obtained. In this step, feature recognition processing is performed on the device screen image to obtain the image features after recognition processing.

[0022] In a specific implementation, the image features in the device screen image are identified, and the pixel distance of the key features of the screen is calculated based on the image features. In an optional implementation provided by this embodiment, feature recognition processing is performed on the device screen image in the following manner: Identifying image features in the device screen image using an image recognition algorithm; If the image features include device screen features, determine screen key features in the device screen features, and calculate pixel distances of the screen key features based on feature position information of the screen key features in the device screen image.

[0023] Among them, the key screen features refer to feature points that can determine the device screen, such as the corner points of the device screen; the feature position information refers to the position information of the feature points that can determine the device screen in the device screen image; the pixel distance refers to the distance between the feature points that can determine the device screen.

[0024] Specifically, the image features in the device screen image are identified according to the image recognition algorithm to determine whether the device screen is mapped in the device screen image; if not, a re-shooting reminder is given; if so, the feature points of the device screen in the device screen image are determined, and the pixel distance between the feature points is calculated based on the position information of the feature points in the device screen image to improve the acquisition quality of the device screen image.

[0025] Step S106: determining a collection reminder for collecting an image of the device screen according to the recognition processing result.

[0026] The above-mentioned image recognition algorithm is used to identify image features in the device screen image and calculate the pixel distance of the key features of the screen. On this basis, in this step, a collection reminder for image collection of the device screen is determined according to the recognition processing result, such as a distance adjustment reminder, an angle adjustment reminder, and a position adjustment reminder.

[0027] In actual applications, the key feature threshold of the screen key features is 4, such as determining the 4 screen corner points of the device screen. However, when the image features in the device screen image identified by the image recognition algorithm contain the device screen features (that is, the device screen is mapped in the device screen image), the number of key features of the screen key features in the determined device screen image may be less than the key feature threshold, indicating that the device screen image is not fully collected and the position and angle can be adjusted; the number of key features of the screen key features in the determined device screen image may also be greater than or equal to the key feature threshold, indicating that the device screen image is fully collected and if other problems have been eliminated, the device screen image can be further tested for clarity.

[0028] During specific implementation, when the number of key features of the screen key features in the determined device screen image is less than the key feature threshold, it means that the acquisition of the device screen image is incomplete. It may be that the device screen only appears partially in the visible surface of the imaging object, resulting in incomplete acquisition. In this case, the position can be translated so that all feature points of the device screen are collected. It may also be that the visible surface angle of the device screen and the imaging object is not suitable, resulting in incomplete acquisition. In this case, the visible surface angle of the device screen and the imaging object can be adjusted.

[0029] In an optional implementation manner provided by this embodiment, when the image features in the device screen image are identified by using the image recognition algorithm, and the image features include the device screen features, the screen key features in the device screen features are determined, and based on the feature position information of the screen key features in the device screen image and the pixel distance of the screen key features is calculated, the collection reminder for image collection is determined in the following manner: If the key feature number of the screen key feature is less than the key feature threshold, a position adjustment reminder for adjusting the image acquisition position and an angle adjustment reminder for adjusting the image acquisition angle are determined based on the key feature number.

[0030] In view of the two situations that the device screen only appears partially in the visible surface of the imaging object, and the angle between the device screen and the visible surface of the imaging object is not suitable, it is possible that the number of key features of the screen key features is less than the key feature threshold. Therefore, based on the number of key features, a position adjustment reminder for adjusting the image acquisition position and an angle adjustment reminder for adjusting the image acquisition angle are determined. The position adjustment reminder includes position translation in the vertical plane, such as "please move the phone upwards"; in addition, if the number of key features of the screen key features is less than the key feature threshold, a position adjustment reminder for adjusting the image acquisition position or an angle adjustment reminder for adjusting the image acquisition angle can also be determined based on the number of key features, so as to achieve targeted and flexible processing of image acquisition reminders and improve the acquisition accuracy of device screen images.

[0031] Further, in the above-mentioned use of the image recognition algorithm to identify the image features in the device screen image, when the image features include the device screen features, the screen key features in the device screen features are determined, and the pixel distance of the screen key features is calculated based on the feature position information of the screen key features in the device screen image. If the number of key features of the screen key features is less than the key feature threshold, based on the position adjustment reminder for adjusting the image acquisition position and / or the angle adjustment reminder for adjusting the image acquisition angle determined based on the number of key features, the image acquisition distance between the device screen and the imaging object can be calculated based on the pixel distance, and a distance adjustment reminder can be further made. In an optional implementation manner provided by this embodiment, the acquisition reminder for image acquisition is determined in the following manner: If the pixel distance is less than a pixel distance threshold, calculating an image acquisition distance between the device screen and the imaging object according to the pixel distance; A distance adjustment reminder for adjusting the image acquisition distance is determined based on the image acquisition distance.

[0032] Specifically, after calculating the pixel distance between feature points according to the device screen features (i.e., the feature points of the device screen can be determined), if the pixel distance is less than the pixel distance threshold, the image acquisition distance between the device screen and the imaging object is calculated according to the pixel distance, and a distance adjustment reminder of whether the device screen is close to the imaging object or the device screen is far away from the imaging object can be determined based on the image acquisition distance. For example, the distance adjustment reminder is "Please move the mobile phone screen close to the mirror"; in addition, in addition to calculating the image acquisition distance between the device screen and the imaging object according to the pixel distance, the image acquisition distance between the device screen and the imaging object can also be calculated according to the current focal length, or the image acquisition distance between the device screen and the imaging object can be calculated in combination with the pixel distance and the current focal length, so as to enrich the calculation method of the image acquisition distance and improve the accuracy of the image acquisition distance.

[0033] The number of key features of the screen key features in the device screen image determined above may be less than the key feature threshold, indicating that the device screen image is not fully collected and the position and angle can be adjusted; the number of key features of the screen key features in the device screen image determined may also be greater than or equal to the key feature threshold, indicating that the device screen image is fully collected; the pixel distance of the calculated screen key features may be less than the pixel distance threshold, indicating that the distance between the device screen and the imaged object is not appropriate, and the pixel distance of the screen key features may also be greater than or equal to the pixel distance threshold, indicating that the distance between the device screen and the imaged object is appropriate. When the number of key features is greater than or equal to the key feature threshold and the pixel distance is greater than or equal to the pixel distance threshold, if other problems have been eliminated, the device screen image can be further tested for clarity.

[0034] Furthermore, when using an image recognition algorithm to identify image features in a device screen image, if the image features include device screen features, determine the screen key features in the device screen features, and calculate the pixel distance of the screen key features based on the feature position information of the screen key features in the device screen image, if the number of key features of the screen key features is less than the key feature threshold, determine a position adjustment reminder for adjusting the image acquisition position and / or an angle adjustment reminder for adjusting the image acquisition angle based on the number of key features, and when the pixel distance is less than the pixel distance threshold, calculate the image acquisition distance between the device screen and the imaging object based on the pixel distance, and determine a distance adjustment reminder for adjusting the image acquisition distance based on the image acquisition distance. In an optional implementation manner provided by this embodiment, if the number of key features is greater than or equal to the key feature threshold and the pixel distance is greater than or equal to the pixel distance threshold, perform the following operations: Calculating the image clarity of the device screen image; If the image definition is greater than or equal to the definition threshold, taking the device screen image as the target device screen image; If the image clarity is less than the clarity threshold, a focus parameter is determined and input into the image sensor for secondary image acquisition to obtain a target device screen image.

[0035] Specifically, when the number of key features is greater than or equal to the key feature threshold and the pixel distance is greater than or equal to the pixel distance threshold, it means that the device screen image basically meets the requirements in terms of image features. If other problems of inappropriate position, angle, and distance are excluded, the image clarity of the device screen image can be further calculated. When the image clarity of the device screen image meets the clarity threshold, the device screen image is used as the target device screen image. When the image clarity of the device screen image does not meet the clarity threshold, it may be a focus problem of the image sensor. The focus parameters are determined to reset the image sensor and perform secondary image acquisition. According to the above-mentioned steps, feature recognition processing is performed on the device screen image, acquisition reminders and mirror reminders are displayed, and the target device screen image is determined. This embodiment will not be repeated here.

[0036] In addition, when the image features in the device screen image identified by the image recognition algorithm do not include the device screen features (i.e., the device screen is not mapped in the device screen image), a collection adjustment reminder for image collection adjustment may also be performed to allow the user to re-collect the device screen image, thereby improving the user's perception and further improving the user's experience. In an optional implementation provided by this embodiment, if the image features do not include the device screen features, the following collection reminder operation is performed: generating a collection adjustment reminder for image collection adjustment corresponding to the device screen features not included in the image features.

[0037] Step S108: displaying the mirror reminder generated based on the acquisition reminder on a display device.

[0038] As described above, the collection reminder for image collection on the device screen is determined based on the recognition processing result. Since the distance between the device screen and the imaging object during the image collection process is relatively close, and the normally generated collection reminder is mapped to the visible surface of the imaging object through the display device as a mirrored collection reminder, it is not convenient for the user to read. In view of this, this embodiment performs mirror processing on the collection reminder, generates a mirrored reminder and displays it on the display device. Correspondingly, the collection reminder displayed on the visible surface of the imaging object is a normally readable collection reminder. In the process of displaying the mirrored reminder generated based on the collection reminder based on the display device, in order to enhance the reminder effect, optionally, a voice reminder of the collection reminder is played through a voice sensor.

[0039] In the specific execution process, after detecting that the image sensor is called, the acquisition reminder page is displayed in the upper layer of the image acquisition page displayed by the display device, and the image reminder generated according to the acquisition reminder is displayed in the acquisition reminder page. In the first optional implementation manner provided by this embodiment, the image reminder generated based on the acquisition reminder is displayed based on the display device in the following manner: Performing mirroring processing on the reminder text and reminder identifier carried in the collection reminder to obtain the mirrored reminder text and the mirrored reminder identifier; The mirror image reminder text and the mirror image reminder logo are displayed in the collection reminder page.

[0040] Specifically, the collection reminder may carry a reminder text, that is, a reminder word, and may also carry a reminder logo. The reminder text and reminder logo carried in the collection reminder are mirrored to obtain the mirrored reminder text and the mirrored reminder logo. The mirrored reminder text and the mirrored reminder logo are displayed in the collection reminder page displayed in the upper layer of the image collection page displayed by the above-mentioned display device to enhance the user's perception, facilitate user observation, and improve the efficiency of collecting device screen images.

[0041] For example, Figure 4 The collection reminder page containing the mirror reminder is displayed. The mirror reminder refers to the mirror reminder text and the mirror reminder logo. 401 displays the mirror reminder text "Please keep the phone screen away from the mirror", and 402 displays the mirror reminder logo, which indicates an arrow for upward position translation; Figure 5 The collection reminder page including the mirror reminder displayed in the visible surface of the display is specifically displayed in the plane mirror Figure 4In the collection reminder page including the mirror reminder, 501 displays the mirror reminder text "Please keep the phone screen away from the mirror" after mirroring, which is a reminder text that can be read normally, and 502 displays the mirror reminder logo after mirroring, which is a reminder logo that can be read normally.

[0042] It should be added that the collection reminder may only carry either a reminder text or a reminder logo. In this case, the reminder text or reminder logo carried in the collection reminder is mirrored to obtain a mirrored reminder text or a mirrored reminder logo; the mirrored reminder text or the mirrored reminder logo is displayed on the collection reminder page.

[0043] In addition, in addition to the above-mentioned implementation method of displaying the mirror reminder text and / or mirror reminder logo in the acquisition reminder page, after detecting that the image sensor has been called, the acquisition reminder page can be displayed on the upper layer of the image acquisition page displayed by the display device, and the image frame can be used as a processing unit to render a second acquisition reminder page, and image mirroring processing can be performed to obtain the mirror reminder page, and the mirror reminder page can be displayed. In an optional implementation method provided by this embodiment, the mirror reminder generated based on the acquisition reminder is displayed based on the display device in the following manner: Rendering a second collection reminder page including the reminder text and reminder identifier carried in the collection reminder; Performing image mirroring processing on the second collection reminder page to obtain a mirrored reminder page; The acquisition reminder page displayed in the upper layer of the image acquisition page is updated to the mirror image reminder page.

[0044] Specifically, a second collection reminder page including reminder text and a reminder mark is rendered first. Similarly, to facilitate user observation, the second collection reminder page is image mirrored to obtain a mirrored reminder page, and the collection reminder page is updated to the mirrored reminder page. What the user observes on the visible surface of the imaged object is the second collection reminder page.

[0045] It should be supplemented that, when using the image frame as the processing unit, rendering a second acquisition reminder page, performing image mirroring processing to obtain a mirrored reminder page, and displaying the mirrored reminder page, a second acquisition reminder page containing the reminder text or reminder logo carried in the acquisition reminder can also be rendered; image mirroring processing is performed on the second acquisition reminder page to obtain a mirrored reminder page; and the acquisition reminder page displayed on the upper layer of the image acquisition page is updated to the mirrored reminder page.

[0046] In specific implementation, after the display device displays a mirror reminder generated based on the acquisition reminder, the user adjusts the acquisition angle, acquisition position and / or acquisition distance for image acquisition according to the acquisition reminder, and faces the image sensor to the visible surface of the imaging object for secondary image acquisition, and the server obtains the second device screen image. In an optional implementation provided by this embodiment, after detecting that the image sensor is called, the acquisition reminder page is displayed in the upper layer of the image acquisition page displayed by the display device, and the acquisition reminder page contains the mirror operation reminder text of the operation reminder text for device screen image acquisition, and / or the acquisition reminder page contains the mirror operation reminder image of the operation reminder image for device screen image acquisition. Based on this, secondary image acquisition is performed in the following manner to determine the target screen image: Calling the image sensor to perform secondary image acquisition on the visible surface of the imaging object to obtain a screen image of a second device; Calculating the image definition of the screen image of the second device; If the image clarity does not meet the clarity threshold, determine the focus parameters and input them into the image sensor to perform three image acquisitions to obtain a target device screen image; If the image definition meets the definition threshold, the second device screen image is used as the target device screen image.

[0047] Specifically, after the mirror reminder generated based on the acquisition reminder is displayed on the display device, the user can view the mirror reminder after the mirroring through the visible surface of the imaging object. In fact, it is an acquisition reminder. Image acquisition adjustment is performed according to the acquisition reminder. The server calls the image sensor to perform a second image acquisition on the visible surface of the imaging object to obtain the second device screen image. According to the above-mentioned steps, feature recognition processing is performed on the device screen image, and acquisition reminders and mirror reminders are displayed. They are not repeated here, and the image clarity of the second device screen image is calculated; when the image clarity of the device screen image does not meet the clarity threshold, it may be a focus problem of the image sensor. The focus parameters are determined to reset the image sensor and perform three image acquisitions to obtain the target device screen image; when the image clarity of the device screen image meets the clarity threshold, the second device screen image is used as the target device screen image.

[0048] In order to improve the calculation accuracy of the image clarity, the image clarity is calculated based on the mirroring operation reminder text and the mirroring operation reminder image. In an optional implementation manner provided by this embodiment, in the specific process of calculating the image clarity of the screen image of the second device, the following operations are performed: Identifying image features corresponding to the mirroring operation reminder text and the mirroring operation reminder image contained in the screen image of the second device; The image clarity is calculated based on the image features and reference image features corresponding to the mirroring operation reminder text and the mirroring operation reminder image.

[0049] It should be supplemented that the image features corresponding to the mirroring operation reminder text or the mirroring operation reminder image contained in the screen image of the second device can also be identified; and the image clarity is calculated based on the image features and the reference image features corresponding to the mirroring operation reminder text or the mirroring operation reminder image.

[0050] In actual applications, due to various accidents, the device may fall or drop, which may subsequently cause the screen to shatter. If it is sent for repair, the repair cost is high, and in the case of a broken screen, most of the time the only option is to replace the screen. In response to this, protection items for the device are provided to users. Users access the protection item by triggering the protection application access portal and capture the device screen image through the image sensor. After the user participates in the protection item, if the screen is damaged, the user can take a picture of the screen and apply for protection, that is, apply for protection resources. The screen here refers to the transparent component set above the display component of the device screen of the protection device participating in the protection item (such as the inner and outer screens of the display screen). Protection items include screen breakage insurance.

[0051] Optionally, the device configured with the device screen is a security device participating in a security project, and the image sensor is called after the security application access portal of the security project is triggered; the security resource issuance condition of the security project is that a transparent component set above the display component of the device screen of the security device is damaged.

[0052] In an optional implementation manner provided in this embodiment, the security resources are issued in the following manner: Based on a target device screen image of a device configured with the device screen, identifying screen features in the target device screen image; If it is determined based on the screen features that the device meets the guaranteed resource issuance condition, the guaranteed resources are issued to the device.

[0053] Specifically, based on the collected target device screen image of the device that configures the device screen, the server identifies the screen features in the target device screen image, that is, identifies the state of the transparent component in the target screen image. If the transparent component state is damaged, it is determined based on the screen features that the security device meets the security resource issuance conditions, and security resources can be issued to the security device to protect the rights and interests of users.

[0054] The following takes the application of an image acquisition reminder processing method provided by this embodiment in a security project scenario as an example to further illustrate the image acquisition reminder processing method provided by this embodiment. Figure 6, an image acquisition reminder processing method applied to a security project scenario, specifically includes steps S602 to S626.

[0055] Step S602: after detecting that the guarantee application access portal of the guarantee project is triggered, calling the image sensor set on the device screen to face the visible surface of the imaging object to capture the device screen image.

[0056] Step S604: using an image recognition algorithm to identify image features in the captured device screen image.

[0057] Step S606: if the image features include device screen features, determine the key screen features in the device screen features, and calculate the pixel distance of the key screen features according to the feature position information of the key screen features in the device screen image.

[0058] If the image feature does not include the device screen feature, a capture adjustment reminder for performing image capture adjustment corresponding to the device screen feature not included in the image feature is generated.

[0059] Step S608: If the key feature number of the screen key features is less than the key feature threshold, a position adjustment reminder for adjusting the image acquisition position and an angle adjustment reminder for adjusting the image acquisition angle are determined based on the key feature number.

[0060] In addition, if the key feature number of the screen key features is less than the key feature threshold, a position adjustment reminder for adjusting the image acquisition position or an angle adjustment reminder for adjusting the image acquisition angle is determined based on the key feature number.

[0061] Step S610: if the pixel distance is less than the pixel distance threshold, the image acquisition distance between the device screen and the imaging object is calculated according to the pixel distance, and a distance adjustment reminder for adjusting the image acquisition distance is determined based on the image acquisition distance.

[0062] If the number of key features is greater than or equal to the key feature threshold and the pixel distance is greater than or equal to the pixel distance threshold, the image clarity of the device screen image is calculated. If the image clarity is greater than or equal to the clarity threshold, the device screen image is used as the target device screen image. If the screen image clarity is less than the clarity threshold, the focus parameters are determined and input into the image sensor for secondary image acquisition to obtain the target device screen image of the security project.

[0063] The collection adjustment reminders, position adjustment reminders, angle adjustment reminders, and distance adjustment reminders mentioned above are all collection reminders.

[0064] Step S612, mirroring the reminder text and reminder mark carried in the collection reminder to obtain the mirrored reminder text and the mirrored reminder mark.

[0065] In addition, the reminder text or reminder mark carried in the collection reminder may also be mirrored to obtain the mirrored reminder text or the mirrored reminder mark.

[0066] Step S614: displaying the mirror reminder text and the mirror reminder logo in the collection reminder page.

[0067] Among them, after detecting that the image sensor is called, the acquisition reminder page is displayed in the upper layer of the image acquisition page displayed by the display device.

[0068] In addition, a mirror reminder text or a mirror reminder logo may also be displayed in the collection reminder page.

[0069] In addition, steps S612 to S614 can also be performed as follows: rendering a second collection reminder page that includes the reminder text and / or reminder logo carried in the collection reminder; performing image mirroring on the second collection reminder page to obtain a mirrored reminder page; and updating the collection reminder page displayed in the upper layer of the image collection page to a mirrored reminder page.

[0070] Step S616: call the image sensor to perform secondary image acquisition on the visible surface of the imaging object to obtain a second device screen image.

[0071] Step S618: identifying image features corresponding to the mirroring operation reminder text and the mirroring operation reminder image contained in the screen image of the second device.

[0072] In addition, image features corresponding to the mirroring operation reminder text or the mirroring operation reminder image contained in the screen image of the second device may also be identified.

[0073] Step S620: Calculate the image clarity based on the image features, the mirroring operation reminder text, and the reference image features corresponding to the mirroring operation reminder image.

[0074] In addition, the image clarity may also be calculated based on the image features and the reference image features corresponding to the mirroring operation reminder text or the mirroring operation reminder image.

[0075] Step S622: If the image clarity does not meet the clarity threshold, the focus parameter is determined and input into the image sensor to perform three image acquisitions to obtain the target device screen image of the security project.

[0076] If the image definition meets the definition threshold, the second device screen image is used as the target device screen image.

[0077] Step S624: Based on the target device screen image of the device for configuring the device screen, identify screen features in the target device screen image.

[0078] Among them, the device that configures the device screen is the protection device participating in the protection project.

[0079] Step S626: If it is determined based on the screen features that the device meets the guarantee resource issuance conditions of the guarantee project, the guarantee resources are issued to the device.

[0080] Among them, the conditions for issuing guarantee resources for the guarantee project are that the transparent component set above the display component of the device screen of the guarantee equipment is damaged, such as damage to the inner screen and / or outer screen of the display screen.

[0081] In summary, the image acquisition reminder processing method provided in this embodiment first obtains a device screen image acquired by calling an image sensor set on the device screen facing the visible surface of the imaging object, and uses an image recognition algorithm to recognize image features in the device screen image. If the image features include device screen features, determine the screen key features in the device screen features, and calculate the pixel distance of the screen key features based on the feature position information of the screen key features in the device screen image; Secondly, if the number of key features of the screen key features is less than the key feature threshold, a position adjustment reminder for adjusting the image acquisition position and / or an angle adjustment reminder for adjusting the image acquisition angle is determined based on the number of key features; if the pixel distance is less than the pixel distance threshold, the image acquisition distance between the device screen and the imaging object is calculated according to the pixel distance, and a distance adjustment reminder for adjusting the image acquisition distance is determined based on the image acquisition distance; if the number of key features is greater than or equal to the key feature threshold and the pixel distance is greater than or equal to the pixel distance threshold, the image clarity of the device screen image is calculated, if the image clarity is greater than or equal to the clarity threshold, the device screen image is used as the target device screen image, if the screen image clarity is less than the clarity threshold, the focus parameters are determined and input into the image sensor for secondary image acquisition to obtain the target device screen image; Again, mirroring the reminder text and / or reminder logo carried in the collection reminder to obtain the mirrored reminder text and / or the mirrored reminder logo, and displaying the mirrored reminder text and / or the mirrored reminder logo in the collected reminder page; or, rendering a second collection reminder page containing the reminder text and / or the reminder logo, mirroring the second collection reminder page to obtain the mirrored reminder page, and updating the collection reminder page displayed in the upper layer of the image collection page to the mirrored reminder page; Finally, the image sensor is called to perform a second image acquisition on the visible surface of the imaging object to obtain the second device screen image, and the image clarity of the second device screen image is calculated; if the image clarity does not meet the clarity threshold, the focus parameters are determined and input into the image sensor to perform three image acquisitions to obtain the target device screen image. If the image clarity meets the clarity threshold, the second device screen image is used as the target device screen image to improve the clarity of the device screen image and realize the automation of device screen image acquisition.

[0082] An embodiment of an image acquisition reminder processing device provided in this specification is as follows: In the above-mentioned embodiment, an image acquisition reminder processing method is provided, and correspondingly, an image acquisition reminder processing device is also provided, which is described below with reference to the accompanying drawings.

[0083] Reference Figure 7 , which shows a schematic diagram of an image acquisition reminder processing device provided in this embodiment.

[0084] Since the device embodiment corresponds to the method embodiment, the description is relatively simple, and the relevant parts can refer to the corresponding description of the method embodiment provided above. The device embodiment described below is only illustrative.

[0085] This embodiment provides an image acquisition reminder processing device, which runs on a server and includes: The device screen image acquisition module 702 is configured to acquire a device screen image captured by an image sensor configured to call the device screen and facing the visible surface of the imaging object; The feature recognition processing module 704 is configured to perform feature recognition processing on the device screen image; The collection reminder determination module 706 is configured to determine a collection reminder to collect an image of the device screen according to the recognition processing result; The mirror image reminder display module 708 is configured to display the mirror image reminder generated based on the collection reminder based on a display device.

[0086] An embodiment of an image acquisition reminder processing device provided in this specification is as follows: Corresponding to the image acquisition reminder processing method described above, based on the same technical concept, one or more embodiments of this specification also provide an image acquisition reminder processing device, which is used to execute the image acquisition reminder processing method provided above. Figure 8 A schematic diagram of the structure of an image acquisition reminder processing device provided in one or more embodiments of this specification.

[0087] This embodiment provides an image acquisition reminder processing device, including: like Figure 8 As shown, the image acquisition reminder processing device may have relatively large differences due to different configurations or performances, and may include one or more processors 801 and memory 802, and the memory 802 may store one or more storage applications or data. Among them, the memory 802 may be a short-term storage or a persistent storage. The application stored in the memory 802 may include one or more modules (not shown in the figure), and each module may include a series of computer executable instructions in the image acquisition reminder processing device. Furthermore, the processor 801 may be configured to communicate with the memory 802, and execute a series of computer executable instructions in the memory 802 on the image acquisition reminder processing device. The image acquisition reminder processing device may also include one or more power supplies 803, one or more wired or wireless network interfaces 804, one or more input / output interfaces 805, one or more keyboards 806, etc.

[0088] In a specific embodiment, the image acquisition reminder processing device includes a memory and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs may include one or more modules, and each module may include a series of computer executable instructions in the image acquisition reminder processing device, and the one or more programs are configured to be executed by one or more processors, including computer executable instructions for performing the following: Obtain a device screen image captured by an image sensor that calls the device screen setting and faces the visible surface of the imaging object; Performing feature recognition processing on the device screen image; Determining a collection reminder to collect an image of the device screen according to the recognition processing result; The mirror reminder generated based on the collection reminder is displayed based on the display device.

[0089] An embodiment of a storage medium provided in this specification is as follows: Corresponding to the image acquisition reminder processing method described above, based on the same technical concept, one or more embodiments of this specification also provide a storage medium.

[0090] The storage medium provided in this embodiment is used to store computer executable instructions, and the computer executable instructions implement the following process when executed by a processor: Obtain a device screen image captured by an image sensor that calls the device screen setting and faces the visible surface of the imaging object; Performing feature recognition processing on the device screen image; Determining a collection reminder to collect an image of the device screen according to the recognition processing result; The mirror reminder generated based on the collection reminder is displayed based on the display device.

[0091] It should be noted that the embodiment of the storage medium in this specification and the embodiment of the image acquisition reminder processing method in this specification are based on the same inventive concept, so the specific implementation of this embodiment can refer to the implementation of the aforementioned corresponding method, and the repeated parts will not be repeated.

[0092] The above is a description of a specific embodiment of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0093] In the 1930s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to circuit structures such as diodes, transistors, switches, etc.) or software improvements (improvements to method flows). However, with the development of technology, many of today's method flow improvements can be regarded as direct improvements to hardware circuit structures. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using hardware entity modules. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask chip manufacturers to design and produce dedicated integrated circuit chips. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one kind of HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0094] The controller may be implemented in any suitable manner, for example, the controller may take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (e.g., software or firmware) executable by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller, examples of which include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320, and the memory controller may also be implemented as part of the control logic of the memory. It is also known to those skilled in the art that, in addition to implementing the controller in a purely computer-readable program code manner, the controller may be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, such a controller may be considered as a hardware component, and the devices for implementing various functions included therein may also be considered as structures within the hardware component. Or even, the devices for implementing various functions may be considered as both software modules for implementing the method and structures within the hardware component.

[0095] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0096] For the convenience of description, the above devices are described in terms of functions and are divided into various units. Of course, when implementing the embodiments of this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0097] It should be understood by those skilled in the art that one or more embodiments of the present specification may be provided as a method, system or computer program product. Therefore, one or more embodiments of the present specification may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0098] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0099] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0100] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0101] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0102] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0103] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined in this article, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0104] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0105] One or more embodiments of the present specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. One or more embodiments of the present specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0106] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0107] The above description is only an embodiment of this document and is not intended to limit this document. For those skilled in the art, this document may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this document should be included in the scope of the claims of this document.

Claims

1. An image acquisition reminder processing method, comprising: Obtain a device screen image captured by an image sensor that calls the device screen setting and faces the visible surface of the imaging object; Performing feature recognition processing on the device screen image, and determining a collection reminder for image collection according to the recognition processing result; Displaying, based on a display device, a mirror reminder generated based on the acquisition reminder; Perform secondary image acquisition and determine the target device screen image.

2. According to the image acquisition reminder processing method of claim 1, the feature recognition processing of the device screen image comprises: Identifying image features in the device screen image using an image recognition algorithm; If the image features include device screen features, determine screen key features in the device screen features, and calculate pixel distances of the screen key features based on feature position information of the screen key features in the device screen image.

3. The image acquisition reminder processing method according to claim 2, wherein determining the image acquisition reminder according to the recognition processing result comprises: If the key feature number of the screen key feature is less than the key feature threshold, a position adjustment reminder for adjusting the image acquisition position and / or an angle adjustment reminder for adjusting the image acquisition angle are determined based on the key feature number.

4. The image acquisition reminder processing method according to claim 3, wherein determining the image acquisition reminder according to the recognition processing result further comprises: If the pixel distance is less than a pixel distance threshold, calculating an image acquisition distance between the device screen and the imaging object according to the pixel distance; A distance adjustment reminder for adjusting the image acquisition distance is determined based on the image acquisition distance.

5. According to the image acquisition reminder processing method of claim 4, if the number of key features is greater than or equal to the key feature threshold and the pixel distance is greater than or equal to the pixel distance threshold, perform the following operations: Calculating the image clarity of the device screen image; If the image definition is greater than or equal to the definition threshold, taking the device screen image as the target device screen image; If the image clarity is less than the clarity threshold, a focus parameter is determined and input into the image sensor for secondary image acquisition to obtain a target device screen image.

6. The image acquisition reminder processing method according to claim 2, if the image features do not include the device screen features; The step of determining a collection reminder for image collection according to the recognition processing result includes: A capture adjustment reminder is generated for performing image capture adjustment corresponding to the device screen feature not included in the image feature.

7. The image acquisition reminder processing method according to claim 1, after detecting that the image sensor is called, a collection reminder page is displayed in a layer above the image acquisition page displayed on the display device.

8. The image acquisition reminder processing method according to claim 7, wherein the display device is used to display the mirror image reminder generated based on the acquisition reminder, comprising: Performing mirroring processing on the reminder text and / or reminder identifier carried in the collection reminder to obtain the mirrored reminder text and / or the mirrored reminder identifier; The mirror image reminder text and / or the mirror image reminder logo are displayed in the collection reminder page.

9. The image acquisition reminder processing method according to claim 7, wherein the display device displays the mirror image reminder generated based on the acquisition reminder, comprising: Rendering a second collection reminder page including the reminder text and / or reminder mark carried in the collection reminder; Performing image mirroring processing on the second collection reminder page to obtain a mirrored reminder page; The acquisition reminder page displayed in the upper layer of the image acquisition page is updated to the mirror image reminder page.

10. The image acquisition reminder processing method according to claim 7, wherein the acquisition reminder page contains a mirror operation reminder text of the operation reminder text for performing device screen image acquisition, and / or the acquisition reminder page contains a mirror operation reminder image of the operation reminder image for performing device screen image acquisition.

11. The image acquisition reminder processing method according to claim 10, wherein the performing secondary image acquisition and determining the target screen image comprises: Calling the image sensor to perform secondary image acquisition on the visible surface of the imaging object to obtain a screen image of a second device; Calculating the image definition of the screen image of the second device; If the image clarity does not meet the clarity threshold, determine the focus parameters and input them into the image sensor to perform three image acquisitions to obtain a target device screen image; If the image definition meets the definition threshold, the second device screen image is used as the target device screen image.

12. The image acquisition reminder processing method according to claim 11, wherein the step of calculating the image clarity of the screen image of the second device comprises: Identifying image features corresponding to the mirroring operation reminder text and / or the mirroring operation reminder image contained in the screen image of the second device; The image clarity is calculated based on the image feature and a reference image feature corresponding to the mirroring operation reminder text and / or the mirroring operation reminder image.

13. The image acquisition reminder processing method according to claim 1, further comprising: The voice reminder of the collection reminder is played through the voice sensor.

14. The image acquisition reminder processing method according to claim 1, wherein the device configured with the device screen is a guarantee device participating in a guarantee project, and the calling of the image sensor is performed after the guarantee application access portal of the guarantee project is triggered; in, The guarantee resource issuance condition for the guarantee project is that a transparent component arranged above a display component of a device screen of the guarantee device is damaged.

15. The image acquisition reminder processing method according to claim 14, further comprising: Based on a target device screen image of a device configured with the device screen, identifying screen features in the target device screen image; If it is determined based on the screen features that the device meets the guaranteed resource issuance condition, the guaranteed resources are issued to the device.

16. The image acquisition reminder processing method according to any one of claims 1 to 15, wherein the imaging object comprises a plane mirror.

17. An image acquisition reminder processing device, comprising: A device screen image acquisition module is configured to acquire a device screen image captured by an image sensor that calls the device screen setting and faces a visible surface of an imaging object; A feature recognition processing module is configured to perform feature recognition processing on the device screen image and determine a collection reminder for image collection according to the recognition processing result; A mirror reminder display module, configured to display a mirror reminder generated based on the acquisition reminder based on a display device; Perform secondary image acquisition and determine the target device screen image.

18. An image acquisition reminder processing device, comprising: processor; and a memory configured to store computer executable instructions that, when executed, cause the processor to: Obtain a device screen image captured by an image sensor that calls the device screen setting and faces the visible surface of the imaging object; Performing feature recognition processing on the device screen image, and determining a collection reminder for image collection according to the recognition processing result; Displaying, based on a display device, a mirror reminder generated based on the acquisition reminder; Perform secondary image acquisition and determine the target device screen image.

19. A storage medium for storing computer executable instructions, wherein the computer executable instructions, when executed by a processor, implement the following process: Obtain a device screen image captured by an image sensor that calls the device screen setting and faces the visible surface of the imaging object; Performing feature recognition processing on the device screen image, and determining a collection reminder for image collection according to the recognition processing result; Displaying, based on a display device, a mirror reminder generated based on the acquisition reminder; Perform secondary image acquisition and determine the target device screen image.

Images