An event confirmation and timing method based on image recognition

Abstract

The present invention discloses an event confirmation and timing method based on image recognition. The method comprises the following steps: S1, the aided person logs in to a terminal with which the aided person has a trusted relationship; S2, a group photo of the aided person and the aided person in the same frame is taken through the terminal, and the group photo is sent to a server; S3, the server matches the group photo with a face feature library; S4, the server sends the identity of B to the aided person's terminal; the aided person confirms the identity of the aided person on the terminal; S5, when the aided person confirms the identity of the aided person in the terminal interface, the terminal interface displays the start of timing; S6, when the mutual aid service ends, the aided person logs in to the terminal with which the aided person has a trusted relationship again; S7, a group photo of the aided person and the aided person in the same frame is taken through the terminal, and the group photo is sent to the server; S8, the server identifies the facial image specified in the group photo, and performs feature matching with the facial image of the identity confirmed in S4.

Description

Technical Field

[0001] The invention relates to a mutual assistance service event confirmation and timing method based on face image recognition. Background Art

[0002] Community mutual aid service is a neighborhood mutual aid service model that mobilizes the community’s surplus labor resources and other voluntary organizations to provide community services.

[0003] Time bank is a paid volunteer service model proposed by Kahn against the backdrop of the stagflation crisis in the United States. In my country, it is mainly used in the field of community elderly care. It can also be applied in broader scenarios such as urban community mutual aid services, new rural mutual aid services, and future community neighborhood mutual aid services.

[0004] In the time bank system, the donor deposits the time of providing mutual aid services into the "bank", which corresponds to the increase of time currency in his time bank account; the recipient needs to pay time currency to enjoy the mutual aid services, which corresponds to the decrease of time currency in his time bank account. For community volunteers, they can accumulate time currency for themselves by participating in volunteer activities, so as to make payments when services are needed in the future. In this model, the accuracy of the measurement of mutual aid service time is related to the vital interests of both the donor and the recipient. It can be said that the confirmation of mutual aid service events and the accurate timing of the service process have become a key technology in the application of community mutual aid service solutions based on time banks. Summary of the invention

[0005] In order to overcome the above-mentioned defects, the present invention provides a mutual assistance service event confirmation and timing method based on face image recognition.

[0006] The present invention proposes an event confirmation and timing method based on image recognition, which is used to confirm mutual assistance service events and accurately time the service process in the process of community mutual assistance service based on time bank. The method at least includes the following steps:

[0007] S0. As a preparatory work, collect a proper amount of facial photo samples from all community members who register to participate in the community mutual aid service, and perform feature analysis on the samples to generate a corresponding feature library. The server can quickly access the feature library. The recipients of the community mutual aid service access the server through the Internet or mobile Internet;

[0008] S1. The recipient logs in to a terminal with which he has a trusted relationship;

[0009] S2. In the service start interface, the terminal takes a photo of the recipient and the donor in the same frame. During the photo, the donor's facial image is displayed in the designated area. The recipient's terminal sends the recipient's identity (recorded as A) and the photo to the server.

[0010] S3. The server performs feature analysis on the facial images in the specified area of the group photo and matches the data with the face feature database; for those with a matching degree greater than or equal to the threshold T, they are arranged in descending order of the matching degree. Denote the identity of the person with the highest matching degree as B, and then denote the M persons with a matching degree lower than B as the substitutes for face recognition (the identities of these M persons are denoted as B1, B2, …, BM). Here, 0 < T < 1, and preferably, T = 0.85;

[0011] If B is not found, that is, no image with a matching degree greater than or equal to T is found in the face database, then return to S2 to take a new photo.

[0012] S4. The server sends the identity of B to the assisted person's terminal. The assisted person's terminal displays the identity information of B on the operation interface. The assisted person needs to confirm the identity of the helper on the terminal:

[0013] S4-1. If it is verified, the assisted person's terminal feeds back the information that the identity of the helper is correctly recognized to the server. After receiving it, the server starts timing for the mutual assistance service time, and at the same time archives and retains the group photo sent in S2 as evidence.

[0014] S4-2. If it is not verified, the assisted person's terminal feeds back the information that the identity of the helper is incorrectly recognized to the server. The server will send at most the identities of M substitutes to the assisted person's terminal until the identity of the helper is selected and confirmed. The specific method can be any one of the following methods:

[0015] (1) The server sends the 1st substitute to the assisted person's terminal and waits for confirmation. If no confirmation is obtained, it sends the 2nd substitute to the assisted person's terminal and waits for the confirmation of the identity of the helper, and so on, until at most the Mth substitute is sent to the assisted person's terminal and waits for the confirmation of the identity of the helper.

[0016] (2) The server sends the identities of M substitutes to the assisted person's terminal at the same time. The assisted person selects and confirms the identity of the helper that is verified on the terminal.

[0017] (3) The server first sends K1 substitutes (1 ≤ K1 < M) to the assisted person's terminal. The assisted person selects the identity of the helper that is verified among them on the terminal; if none of the K1 substitutes is verified, the server then sends K2 substitutes (1 < K2 ≤ M - K1) to the assisted person's terminal; in this way, the K substitutes are sent to the assisted person's terminal in batches until the assisted person selects and confirms the identity of the helper that is verified on the terminal;

[0018] When the server receives the information for confirming the identity of the assisted person from the assisted person's terminal, it starts timing for the mutual assistance service time, and at the same time archives and retains the group photo sent in S2 as evidence;

[0019] If none of the M substitutes receives confirmation from the recipient's terminal, the confirmation of the donor's identity fails this time, and it returns to S2 to take a new photo.

[0020] S5. After the recipient confirms the donor's identity in the terminal interface, the timing starts to be displayed on the terminal interface. Preferably, during the mutual assistance service process, the terminal interface continuously tracks and displays the current timing information.

[0021] S6. When the mutual assistance service ends, the recipient logs in to the terminal with which there is a trusted relationship again.

[0022] S7. In the service end interface, take a group photo of the recipient and the donor together through the terminal. During the photo-taking process, the facial image of the donor appears at the designated position. The recipient's terminal sends the recipient's identity and the group photo to the server.

[0023] S8. The server identifies the designated facial image in the group photo and performs feature matching with the facial image of identity B confirmed in S4. If the degree of conformity is greater than or equal to the preset threshold H, the server will terminate the timing of the mutual assistance service time and archive and retain the group photo in S7 as evidence; here 0 < H < 1, preferably, H = 0.9.

[0024] Preferably, the server also sends the timing result of the mutual assistance service to the recipient's terminal, and the recipient's terminal displays the mutual assistance service end information and duration information on the interface.

[0025] If the degree of conformity is less than the preset threshold H, the server notifies the recipient's terminal that the mutual assistance service ends unsuccessfully. The recipient's terminal needs to take a new group photo and return to S7.

[0026] S9. This method also supports the following special steps involving manual participation:

[0027] S9-1. When the process enters step S3 a cumulative of K1 times and no verified donor identity is found through face matching, the recipient can choose to manually enter the donor's identity information and send it to the server together with the group photo, to be left for later manual verification to confirm the effectiveness of this mutual assistance service. Among them, K1 is a preset positive integer.

[0028] S9-2. When the process enters step S4 a cumulative of K2 times and no verified donor identity is found through face matching, the recipient can choose to manually enter the donor's identity information and send it to the server together with the group photo, to be left for later manual verification to confirm the effectiveness of this mutual assistance service. Among them, K2 is a preset positive integer.

[0029] S9-3, when the process enters step S8 for K3 times, and the consistency of the identity of the helper has not been confirmed by face matching for K3 times, the helpee can choose to manually confirm the end of the mutual assistance service, and send it to the server together with the group photo, leaving it for later manual identification to confirm the validity of this mutual assistance service. K3 is a preset positive integer.

[0030] Preferably, due to the need to protect identity information, the terminal uses a partially hidden incomplete display method when displaying the donor's identity.

[0031] The present invention provides a technical confirmation method for community mutual assistance service events and accurately times the service process, thereby solving the problem of paid volunteer service models represented by time banks in their implementation and promotion. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 This is a schematic diagram of the mutual assistance service event confirmation and process timing steps initiated by the mobile terminal based on image recognition

[0033] Figure 2 This is a schematic diagram of the mutual assistance service event confirmation and process timing steps based on image recognition initiated by the web browser

[0034] Figure 3 This is a diagram showing an incomplete display of the donor's identity. DETAILED DESCRIPTION

[0035] The present invention proposes an event confirmation and timing method based on face image recognition, which is used to confirm mutual assistance service events and accurately time the service process in the process of community mutual assistance service based on time bank, so as to support the settlement and management of paid volunteer services. The technical solution in the embodiment of the present invention is clearly and completely described below in conjunction with the accompanying drawings.

[0036] Example 1

[0037] In this embodiment, a proper amount of facial photo samples are collected in advance from all community members who register to participate in the community mutual aid service, and feature analysis is performed on the samples to generate a corresponding feature library Q. The recipients of the community mutual aid service use mobile phones, PADs or other forms of mobile terminals to establish a communication connection with the server deployed in the cloud through the mobile Internet, and the server can quickly access the feature library Q.

[0038] In the process of community mutual assistance service, the helper provides paid voluntary services to the helpee. The beginning and end of the service process are initiated by the helpee, and the identity verification and service evidence are obtained by taking a photo with the helper. The method includes at least the following steps: Figure 1 As shown:

[0039] S1. The recipient turns on the mobile terminal, logs in to the pre-installed terminal application (APP) with the recipient's own account through reliable verification methods such as passwords and fingerprints.

[0040] S2. On the start interface of the mutual assistance service in the APP, take a group photo of the recipient and the donor in the same frame, and ensure that the facial image of the donor appears in the specified area during the photo-taking process. The APP sends the recipient's account and the group photo to the server in the cloud.

[0041] S3. The server performs feature analysis on the facial image in the specified area of the group photo and matches it with the data in the face feature library Q. For the face images with a matching degree greater than or equal to the threshold T, they are arranged in descending order of the matching degree. Denote the identity of the person with the highest matching degree as B, and then denote the M persons with matching degrees ranked after B as substitutes for face recognition (the identities of these M persons are denoted as B1, B2,..., BM). Here, 0 < T < 1, and preferably, T = 0.85.

[0042] If B is not found, that is, no person with a matching degree greater than or equal to T is found in the feature library Q, then return to S2 and take a photo again.

[0043] S4. The server sends the identity of B to the APP. The identity information of B is displayed on the operation interface of the APP, and the recipient needs to confirm the identity of the donor in the APP:

[0044] S4-1. If it is verified, the APP feeds back the information that the identity of the donor is correctly recognized to the server. After receiving it, the server starts timing for the mutual assistance service time, and at the same time archives and retains the group photo sent in S2 as evidence.

[0045] S4-2. If it is not verified, the APP feeds back the information that the identity of the donor is incorrectly recognized to the server. Next, the server sends at most M substitute identities to the APP until the identity of the donor is selected and confirmed. The specific method can be any one of the following methods:

[0046] (1) The server sends the first substitute to the APP and waits for the APP to feedback whether the first substitute is the donor. If the feedback is that the identity does not match, then send the second substitute to the APP and wait for the APP to feedback whether the second substitute is the donor, and so on, until at most the Mth substitute is sent to the APP and wait for the APP to confirm whether the identity of the donor matches.

[0047] (2) The server sends M substitutes to the APP at the same time, and the recipient selects the identity of the donor that matches and confirms it.

[0048] (3) The server first sends K1 substitutes (1 ≤ K1 < M) to the APP, and the recipient selects the eligible helper identities from them; if none of the above K1 substitutes are eligible, the server then sends K2 substitutes (1 < K2 ≤ M - K1) to the APP. In this way, the K substitutes are sent to the APP in batches until the recipient confirms the eligible helper identity in the APP.

[0049] When the server receives the information that the APP has confirmed the helper identity, it starts timing the mutual assistance service time and archives and retains the group photo image sent by S2 for evidence.

[0050] If none of the M substitutes have been confirmed by the APP, the confirmation of the helper identity for this time fails, and S2 is returned to take a new photo.

[0051] S5. When the recipient confirms the helper identity in the APP, the APP interface shows that the timing starts. Preferably, during the mutual assistance service process, the APP interface continuously tracks and shows the current timing information.

[0052] S6. When the mutual assistance service ends, the recipient logs in to the APP again.

[0053] S7. In the service end interface of the APP, take a group photo of the recipient and the helper in the same frame, and ensure that the facial image of the helper appears in the designated area during the photo-taking process. The APP sends the recipient's account number and the group photo image to the server.

[0054] S8. The server performs feature comparison on the facial image in the designated area of this group photo and the facial image used to confirm identity B in step S4.

[0055] If the degree of conformity is greater than or equal to the preset threshold H, the server will terminate the timing of the mutual assistance service time and archive and retain the group photo image in S7 for evidence. Here, 0 < H < 1, preferably, H = 0.9.

[0056] Preferably, the server also sends the timing result of the mutual assistance service to the APP, and the APP shows the mutual assistance service end information and duration information on the interface.

[0057] If the degree of conformity is less than the preset threshold H, the server notifies the APP that the mutual assistance service ends unsuccessfully. The recipient needs to take a new group photo image on the APP and return to S7.

[0058] In addition, this method also supports the following special steps involving manual participation:

[0059] (1) If step S3 is executed K1 times cumulatively and the identity of the donor is not found through face matching, where K1 is a preset positive integer, the recipient can choose to manually input the donor's identity information and send it to the server together with the photo, leaving it for later manual identification to confirm the effectiveness of this mutual aid service.

[0060] (2) If step S4 is executed K2 times cumulatively and the identity of the donor is not found through face matching, where K2 is a preset positive integer, the recipient can choose to manually input the donor's identity information and send it to the server together with the photo, leaving it for later manual identification to confirm the effectiveness of this mutual aid service.

[0061] (3) If step S8 is executed for a total of K3 times without confirming the consistency of the helper's identity through face matching, where K3 is a preset positive integer, the recipient can choose to manually confirm the end of the mutual aid service and send it to the server together with the group photo, leaving it for later manual identification to confirm the validity of this mutual aid service.

[0062] Preferably, due to the need to protect identity information, this embodiment adopts a partially hidden incomplete display mode when the APP displays the identity of the donor. Figure 2 This is a schematic diagram of an incomplete display method.

[0063] Example 2

[0064] In this embodiment, a proper amount of facial photo samples are collected in advance from all community members who register to participate in the community mutual aid service, and feature analysis is performed on the samples to generate a corresponding feature library Q. The recipients of the community mutual aid service use a personal computer to establish a communication connection with a remote server through a browser, and the server can quickly access the feature library Q.

[0065] In the process of community mutual assistance service, the helper provides paid voluntary services to the helpee. The beginning and end of the service process are initiated by the helpee, and the identity verification and service evidence are obtained by taking a photo with the helper. The method includes at least the following steps: Figure 3 As shown:

[0066] S1. The beneficiary opens the browser and enters the server URL, and logs in to the server with the beneficiary's own account through trusted verification methods such as passwords and random codes.

[0067] S2. In the mutual aid service start interface, take a photo of the aid recipient and the aid provider in the same frame, and ensure that the aid provider's facial image is displayed in the designated area during the photo. The browser sends the aid recipient's account and the photo to the server.

[0068] S3. The server performs feature analysis on the facial images in the specified area of the group photo image and matches them with the data in the face feature database Q. For the face images with a matching degree greater than or equal to the threshold T, they are arranged in descending order of the matching degree. Denote the identity of the person with the highest matching degree as B, and then denote the next M persons with matching degrees after B as the backups for face recognition (the identities of these M persons are denoted as B1, B2, …, BM). Here, 0 < T < 1, and preferably, T = 0.85.

[0069] If B is not found, that is, no person with a matching degree greater than or equal to T is found in the feature database Q, then return to S2 to take a new photo.

[0070] S4. The server sends the identity of B to the browser. The identity information of B is displayed on the browser page, and the recipient needs to confirm the identity of the helper on the page:

[0071] S4-1. If it is verified, the browser sends the information that the identity of the helper is correctly recognized to the server. After receiving it, the server starts timing for the mutual assistance service time, and at the same time archives and retains the group photo image sent in S2 as evidence.

[0072] S4-2. If it is not verified, the browser sends the information that the identity of the helper is incorrectly recognized to the server. Next, the server sends at most M backup identities to the browser until the identity of the helper is selected and confirmed. The specific method can be any one of the following methods:

[0073] (1) The server sends the 1st backup to the browser and waits for its feedback on whether the 1st backup is the helper. If the feedback is that the identity does not match, then send the 2nd backup to the browser and wait for its feedback on whether the 2nd backup is the helper, and so on, until at most the Mth backup is sent to the browser and wait for its confirmation on whether the identity of the helper matches.

[0074] (2) The server sends M backups to the browser at the same time, and the recipient selects and confirms the identity of the helper that matches.

[0075] (3) The server first sends K1 backups (1 ≤ K1 < M) to the browser, and the recipient selects the identity of the helper that matches from them; if none of the above K1 backups match, the server then sends K2 backups (1 < K2 ≤ M - K1) to the browser. In this way, the K backups are sent to the browser in batches until the recipient confirms the identity of the helper that matches on the web page.

[0076] When the server receives the information confirming the identity of the helper from the browser, it starts timing for the mutual assistance service time, and at the same time archives and retains the group photo image sent in S2 as evidence.

[0077] If none of the M substitutes receive confirmation from the browser, the confirmation of the helper's identity fails, and the process returns to S2 to take a new photo.

[0078] S5. After the recipient confirms the helper's identity in the browser, the web page starts displaying the timer. Preferably, during the mutual assistance service, as long as the browser web page is not closed, the current timer information will be continuously tracked and displayed.

[0079] S6. When the mutual assistance service ends, the recipient opens the browser again, enters the server website address, and logs in to the server.

[0080] S7. On the service end interface of the browser web page, take a group photo of the recipient and the helper in the same frame, and ensure that the facial image of the helper is presented in the specified area during the photo-taking process. The browser sends the recipient's account and the group photo to the server.

[0081] S8. The server performs feature comparison on the facial image in the specified area of this group photo and the facial image used to confirm identity B in step S4.

[0082] If the degree of conformity is greater than or equal to the preset threshold H, where 0 < H < 1, preferably H = 0.9, the server will terminate the timing of the mutual assistance service time and archive the group photo in S7 for evidence.

[0083] Preferably, the server also sends the timing result of the mutual assistance service to the browser, and the browser page will display the mutual assistance service end information and the duration information.

[0084] If the degree of conformity is less than the preset threshold H, the server notifies the browser that the mutual assistance service ends unsuccessfully. The recipient needs to retake a group photo on the browser page and return to S7.

[0085] In addition, this method also supports the following special steps involving manual participation:

[0086] (1) If after executing step S3 a cumulative of K1 times, no authentic helper identity is found through face matching, where K1 is a preset positive integer, then the recipient can choose to manually enter the helper's identity information and send it to the server together with the group photo, to be left for later manual verification to confirm the effectiveness of this mutual assistance service.

[0087] (2) If after executing step S4 a cumulative of K2 times, no authentic helper identity is found through face matching, where K2 is a preset positive integer, then the recipient can choose to manually enter the helper's identity information and send it to the server together with the group photo, to be left for later manual verification to confirm the effectiveness of this mutual assistance service.

[0088] (3) If step S8 is executed for a total of K3 times without confirming the consistency of the helper's identity through face matching, where K3 is a preset positive integer, the recipient can choose to manually confirm the end of the mutual aid service and send it to the server together with the group photo, leaving it for later manual identification to confirm the validity of this mutual aid service.

[0089] Preferably, due to the need to protect identity information, this embodiment uses a partially hidden incomplete display method when displaying the donor's identity in the browser web page. Figure 3 This is a schematic diagram of an incomplete display method.

[0090] It should be pointed out that embodiments (I) and (II) not only describe the key steps of the mutual assistance service event confirmation and timing method, but also introduce several key points or recommended methods in the implementation of the above steps. In specific practice, the above steps can derive more changes (such as branch processes); any core process or key steps that are the same as those of the present invention should be regarded as within the scope of the present invention.

Claims

1. An event confirmation and timing method based on image recognition, used to confirm mutual aid service events and accurately time the service process in a community mutual aid service process based on a time bank; characterized in that: In advance, a proper amount of facial photo samples are collected from all community members who have registered to participate in the community mutual aid service, and feature analysis is performed on the samples to generate a corresponding feature library Q; the recipients of the community mutual aid service use a personal computer to establish a communication connection with a remote server through a browser, and the server can quickly access the feature library Q; In the process of community mutual aid service, the helper provides paid voluntary services to the helpee, the start and end of the service process are initiated by the helpee, and the identity verification and service evidence are obtained by taking a photo with the helper; the method includes at least the following steps: S1. The recipient opens the browser and enters the server URL. He / she logs into the server with his / her own account through a trusted authentication method using a password or random code. S2. In the mutual aid service start interface, take a photo of the aid recipient and the aid provider in the same frame, and ensure that the aid provider's facial image is presented in the designated area during the photo; the browser sends the aid recipient's account and the photo to the server; S3. The server performs feature analysis on the facial image of the specified area in the group photo and matches it with the data in the face feature library Q. For the facial images with a matching degree greater than or equal to the threshold T, they are arranged from high to low according to the matching degree, and the person with the highest matching degree is recorded as B, and then the M persons with matching degrees after B are recorded as substitutes for face recognition; where 0 <T<1; If B is not found, that is, no person with a matching degree greater than or equal to T is found in the feature library Q, then return to S2 and retake the photo; S4. The server sends B's identity to the browser. The browser page displays B's identity information, and the recipient needs to confirm the donor's identity on the page: S4-1, if it is true, the browser will feed back the correct information of the helper's identity to the server, and the server will start timing the mutual assistance service time after receiving it, and at the same time archive the group photo sent by S2 as evidence; S4-2. If it is not true, the browser will feedback the information of the helper's identity recognition error to the server, and the server will then send at most M substitute identities to the browser until the helper's identity is selected and confirmed; the specific method is any of the following methods: (1) The server sends substitute No. 1 to the browser and waits for feedback on whether substitute No. 1 is the helper. If the feedback is that the identity is not true, the server sends substitute No. 2 to the browser and waits for feedback on whether substitute No. 2 is the helper. This continues until substitute No. M is sent to the browser and waits for confirmation of whether the helper's identity is true. (2) The server sends M substitutes to the browser at the same time, and the recipient selects the donor who matches his identity and confirms it; (3) The server first sends K1 substitutes to the browser, and the recipient selects the donor who matches the identity. If none of the K1 substitutes match the identity, the server sends K2 substitutes to the browser. In this way, K substitutes are sent to the browser in batches until the recipient confirms the identity of the donor who matches the identity on the web page. Where 1≤K1 <M; 1<K2≤M-K1; When the server receives the information from the browser confirming the identity of the helper, it starts counting the mutual assistance service time and archives the group photo sent by S2 as evidence; If none of the M substitutes are confirmed by the browser, the identity confirmation of the donor fails, and the process returns to S2 to take a new photo; S5. When the recipient confirms the identity of the donor in the browser, the web page displays the start of the timer; S6. When the mutual assistance service ends, the recipient opens the browser again, enters the server URL, and logs into the server; S7. In the service end interface of the browser webpage, a photo of the aid recipient and the aid provider is taken, and the aid provider's facial image is ensured to be presented in the designated area during the photo. The browser sends the aid recipient's account number and the photo to the server. S8, the server compares the facial image of the designated area of ​​the current group photo with the facial image used to confirm identity B in step S4; If the compliance is greater than or equal to the preset threshold H, the server will terminate the timing of the mutual assistance service time and archive the group photo of S7 as evidence; <H<1; The server also sends the timing result of the mutual assistance service to the browser, and the browser page will display the end information and duration information of the mutual assistance service; If the degree of compliance is less than the preset threshold value H, the server notifies the browser that the mutual assistance service has failed to end; the recipient needs to retake the group photo in the browser page, and returns to S7; It also includes special steps for human involvement: (1) If step S3 is performed K1 times cumulatively and the identity of the donor is not found through face matching, where K1 is a preset positive integer, the recipient manually enters the donor's identity information and sends it to the server together with the group photo, leaving it for later manual identification to confirm the validity of this mutual aid service; (2) If step S4 is performed K2 times cumulatively and the identity of the donor is not found through face matching, where K2 is a preset positive integer, the recipient manually enters the donor's identity information and sends it to the server together with the group photo, leaving it for later manual identification to confirm the validity of this mutual aid service; (3) If step S8 is executed a total of K3 times without the consistency of the helper's identity being confirmed by face matching, where K3 is a preset positive integer, the beneficiary manually confirms the end of the mutual aid service and sends it to the server together with the group photo, leaving it to later manual identification to confirm the validity of this mutual aid service.

Images