Inspection card punching method and device, equipment and medium

Abstract

The invention relates to an inspection card punching method and device, equipment and a medium in the technical field of property information, and the method comprises the steps: responding to an inspection task execution event, and when a preset fence range of a corresponding inspection point reaches a preset range, and the position information of a terminal is located in the preset fence range, executing the inspection task; acquiring inspection point clock-in information and inspection real-shot images correspondingly generated when the terminal executes on-site clock-in operation and on-site shooting operation to form inspection point detail information; when the preset fence range of the inspection point is smaller than the preset range, sending on-site playing voice corresponding to the one-time verification code to audio playing equipment on the site of the inspection point, obtaining a to-be-verified code which is uploaded by the terminal and is obtained by collecting and analyzing the corresponding to-be-verified audio, and when the to-be-verified code is consistent with the one-time verification code through comparison, sending the to-be-verified audio to the inspection point. And obtaining the detailed information of the inspection point constructed in a preset time length, and sending the detailed information of the inspection point to a task administrator of the inspection task. According to the invention, card punching in a limited inspection area can be strictly controlled.

Description

Technical Field

[0001] The present application relates to the field of property information technology, and in particular to a patrol inspection and clocking-in method and its corresponding device, computer equipment, and computer-readable storage medium. Background Art

[0002] With the rapid development of informatization and intelligence, the routine inspection and clocking in of property management is facing unprecedented challenges and opportunities. With the deep integration of geographic location information technology (such as GPS, Beidou, etc.) and mobile smart terminals, location-based services (LBS) have been widely used in all walks of life. Especially in the field of property management and personnel monitoring, the traditional "man-to-man" management method can no longer meet the needs of efficient, accurate and flexible management.

[0003] In view of the shortcomings of traditional practices, the applicant has been engaged in research in related fields for a long time and has taken a different approach to solve industry problems in the field of property information technology. Summary of the invention

[0004] The primary purpose of the present application is to solve at least one of the above problems and to provide a patrol inspection punch-in method and its corresponding device, computer equipment, and computer program product.

[0005] In order to meet the various objectives of this application, this application adopts the following technical solutions:

[0006] A patrol inspection punch-in method provided to meet one of the purposes of this application includes the following steps:

[0007] In response to the inspection task execution event, when the preset fence range of the corresponding inspection point reaches the preset range and the location information of the terminal is within the preset fence range, the inspection point punch-in information and the inspection real-shot image generated by the terminal performing the on-site punch-in operation and the on-site shooting operation are obtained to form the inspection point detailed information;

[0008] When the preset fence range of the inspection point is smaller than the preset range, a live playback voice corresponding to the one-time verification code is sent to the audio playback device at the inspection point, and the terminal uploads the verification code obtained by collecting and parsing the corresponding audio to be verified;

[0009] When the code to be verified is consistent with the one-time verification code, the inspection point punch-in information and the inspection real-shot image generated by the terminal performing the on-site punch-in operation and the on-site shooting operation within the preset time period are obtained to form the inspection point detailed information;

[0010] The inspection point detailed information is sent to the task administrator of the inspection task.

[0011] On the other hand, a patrol inspection punching device provided to meet one of the purposes of the present application includes a first patrol inspection punching module, a module for obtaining a code to be verified, a second patrol inspection punching module and an information sending module, wherein the first patrol inspection punching module is used to respond to a patrol inspection task execution event, and when the preset fence range of the corresponding patrol inspection point reaches the preset range, and the location information of the terminal is within the preset fence range, the patrol inspection point punching information and patrol inspection real-shot images generated corresponding to the on-site punching operation and on-site shooting operation performed by the terminal are obtained to constitute patrol inspection point detailed information; the module for obtaining the code to be verified is used to respond to the patrol inspection task execution event when the preset fence range of the corresponding patrol inspection point reaches the preset range and the location information of the terminal is within the preset fence range. When the fence range is smaller than the preset range, a live broadcast voice corresponding to the one-time verification code is sent to the audio playback device at the inspection point, and the terminal uploads the code to be verified obtained by collecting and parsing the corresponding audio to be verified; the second inspection punch-in module is used to obtain the inspection point punch-in information and inspection real-shot images generated by the terminal performing the on-site punch-in operation and on-site shooting operation within the preset time period when the code to be verified is consistent with the one-time verification code, so as to constitute the inspection point detailed information; the information sending module is used to send the inspection point detailed information to the task administrator of the inspection task.

[0012] On the other hand, a computer device is provided to meet one of the purposes of the present application, including a central processing unit and a memory, wherein the central processing unit is used to call and run a computer program stored in the memory to execute the steps of the inspection and clocking-in method described in the present application.

[0013] On the other hand, a computer program product provided to meet another purpose of the present application includes a computer program / instruction, which, when executed by a processor, implements the steps of the method described in any embodiment of the present application.

[0014] The technical solution of this application has many advantages, including but not limited to the following aspects:

[0015] This application first determines whether the terminal location information is within the preset fence range. Only when the patrol personnel actually arrives at the preset fence range can the on-site clock-in and shooting operations be triggered, ensuring that the patrol personnel must personally arrive near the inspection point to complete the clock-in and shooting, thereby avoiding the possibility of remote clock-in and shooting.

[0016] Secondly, when the preset fence range of the inspection point is smaller than the preset range, the system will send a one-time verification code to the on-site audio playback device and collect and parse the corresponding audio to be verified through the terminal. This enhances the randomness and unpredictability of the inspection process, making it impossible for inspectors to prepare or forge on-site verification information in advance, because each verification code is temporarily generated and used once.

[0017] Furthermore, only when the verification code is consistent with the one-time verification code, the inspector needs to complete the on-site clock-in and photo shooting within the preset time. Verify the actual presence of the inspector and complete the clock-in and photo shooting within a limited time to avoid clocking in and taking photos away from the site.

[0018] Finally, the check-in information and real-time images of the inspection points are sent to the task manager, providing detailed data support for the supervision and management of the inspection work. This improves the transparency of the inspection check-in work and provides a reliable basis for checking the inspection check-in work.

[0019] To sum up, without the need for any property management personnel to intervene in supervision, the quality and efficiency of the inspection work can be improved, providing strong support for inspection management and ensuring the fairness and accuracy of the inspection work. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0021] Figure 1 A schematic diagram of a network architecture of an exemplary application environment of the present application;

[0022] Figure 2 This is a flowchart of a typical embodiment of the inspection clocking-in method of the present application;

[0023] Figure 3 This is a principle block diagram of the inspection and punching device of this application;

[0024] Figure 4 A schematic diagram of the structure of a computer device used in this application. DETAILED DESCRIPTION

[0025] The following describes the working principle of the inspection and clocking-in method of the present application and its implementation in the network architecture.

[0026] refer to Figure 1, the network architecture of the present application is composed of a terminal device 80 and a server 82. The terminal device 80 can be a mobile device used by a task manager or an inspector, such as a smart phone or a tablet computer, wherein the task manager can use the terminal device to trigger an inspection task issuance request, and the inspector can use the terminal device to receive inspection task details, take actual inspection images during the inspection task execution, and submit these images as proof of the completion of the inspection task. The server 82 is the core processing unit of the present method, and a property management system is deployed. It is responsible for executing a series of operations such as inspection task issuance, parameter push, actual inspection verification, status update, etc., and provides verification services, evidence storage services, clock-in services, and other possible execution services. The server 82 can be a single server or a front-end server in a distributed deployed service cluster to improve the processing power and reliability of the system.

[0027] The inspection tasks assigned in this application refer to a series of work activities planned in the field of property management to maintain property facilities, improve service quality or respond to the needs of owners. These inspection tasks usually involve inspectors with different work roles, such as property service personnel, maintenance technicians, security monitoring personnel, etc.

[0028] Each inspection task has its corresponding task execution process, which is the core part of the inspection task. It describes in detail a series of steps from task assignment to task completion. This process starts with the task manager submitting an inspection task issuance request through the property management system, and then the system distributes the task details to the corresponding inspectors. The inspectors perform the work according to the assigned task details and collect images at the inspection points of the task to prove the completion of the task. The task execution process also includes the verification of these images, and automatically updates the task status based on the verification results, and sends status update notifications to relevant personnel in real time to ensure the transparency and traceability of task execution.

[0029] Inspection routes can be set in the task execution process, which include at least one inspection point. Inspection points are pre-defined specific stages or milestones in the task execution process, which are crucial to the successful completion of the task. In the inspection task, the inspection route set can include but is not limited to inspection points corresponding to the safety availability of materials, the safety availability of facilities, the construction quality, and the maintenance quality. At these inspection points, the inspectors need to collect and submit the actual inspection images completed at the inspection points. These images will then be verified by the system's built-in image analysis engine to ensure that the inspection task meets the established standards and quality requirements.

[0030] The task administrator of the present application submits an inspection task distribution request corresponding to the inspection task through the server 82, and the server 82 responds to the request and obtains the inspection task, which in one embodiment includes the inspection route, designated inspection personnel, execution start and end time, execution effective time, inspection details, etc. Then, the server 82 pushes the inspection task to the terminal device 80 of the corresponding inspection personnel.

[0031] The inspection personnel use the camera function on their terminal device 80 to collect the inspection real-shot images according to the provided image capture parameters and submit them to the server 82. After receiving the inspection real-shot images, the image verification mechanism of the server 82 calls the image analysis engine for verification. The image analysis engine uses an adaptive learning algorithm to optimize the analysis accuracy and efficiency based on historical verification data, and calls the corresponding functional modules for verification, such as calling the signature recognition functional module to verify whether the signature is signed by the same person, calling the quality inspection functional module to identify whether there is any quality abnormality at the inspection site, and calling the image matching functional module to match whether the image to be verified matches its original image.

[0032] According to the verification results, the status update module of the server 82 automatically updates the task status and sends a task status update notification to the task manager and the inspection personnel. If the verification results of all key nodes are passed, the server 82 can automatically generate a task summary report and send a notification of the task summary report through the property management system.

[0033] From the above principle description, it can be seen that the inspection task processing method of the present application can ensure the transparency and credibility of the inspection task execution, while providing an efficient and reliable task collaboration and management platform, constituting a property collaboration management system, which is suitable for a variety of property management and service scenarios.

[0034] A patrol inspection and clocking-in method of the present application can be programmed as a computer program product and deployed in a client or server for execution. For example, in the exemplary application scenario of the present application, it can be deployed and implemented in a server of an independent station store, thereby accessing an interface opened after the computer program product is running and performing human-computer interaction with the process of the computer program product through a graphical user interface to execute the method.

[0035] See also Figure 2 , a patrol inspection punch-in method of the present application, in its typical embodiment, comprises the following steps:

[0036] Step S1100, in response to the inspection task execution event, when the preset fence range of the corresponding inspection point reaches the preset range, and the location information of the terminal is within the preset fence range, the inspection point punch-in information and the inspection real-shot image generated by the terminal performing the on-site punch-in operation and the on-site shooting operation are obtained to form the inspection point detailed information;

[0037] For each inspection point in the inspection route set for the inspection task whose preset fence range is greater than or equal to the preset threshold, the preset fence range of each inspection point can be pre-set by the task administrator who sets the inspection task.

[0038] It can be understood that the property management system can provide the task administrator with the function of setting up an electronic fence. The task administrator can use this function on his terminal. First, he selects the inspection point in the set inspection route. The system locates and displays the location of the inspection point on the map displayed on the function interface. Then, the task administrator can use the drawing toolbar of the function interface to freely draw the shape and size of the electronic fence of the inspection point. He can also choose to draw a reverse electronic fence within the electronic fence. After all the drawings are completed, the system will use the corresponding range within the electronic fence of the inspection point and outside the reverse electronic fence as the preset fence range of the inspection point. The drawing toolbar includes a variety of box selection tools such as any one or any multiple of irregular shapes, circles, ellipses, rectangles, triangles, etc., and also includes an erase tool, a delete control tool, a copy control tool, a return control tool, etc. Those skilled in the art can flexibly implement the drawing toolbar.

[0039] The preset range refers to a range of 100 meters from the corresponding inspection point.

[0040] The server responds to the on-site clocking-in event of the inspection point during the execution of the inspection task, obtains the location information and the unique identifier of the inspection point uploaded by the inspection personnel operating the terminal, and confirms that the preset fence range of the inspection point is greater than or equal to the preset range, and when the location information is within the preset fence range, obtains the inspection point clocking-in information generated by the terminal uploading the clocking-in interactive operation at the inspection point where the terminal is located. The clocking-in interactive operation can be that the inspection personnel hand-writes the inspector's own name in the handwritten signature panel interface displayed on the terminal. The inspection point clocking-in information includes the inspection point information, clocking-in identity information, and clocking-in time. The clocking-in identity information includes the account information of the clocking-in person and the signature image of the clocking-in person. The account information of the clocking-in person is the account information logged in by the inspection personnel when using the system on the terminal. The clocking-in time is the timestamp corresponding to the completion of the clocking-in interactive operation.

[0041] Furthermore, the server responds to the on-site shooting event of the inspection point during the execution of the inspection task, obtains the on-site real-shot image uploaded by the terminal at the inspection point, and determines the corresponding on-site shooting information, generates an on-site shooting watermark image containing the on-site shooting information, and synthesizes the on-site shooting watermark image to the preset watermark position in the on-site real-shot image to obtain the inspection real-shot image. The on-site shooting information includes the shooting time, shooting point information, and the account information of the photographer. The shooting time is the corresponding timestamp when the on-site real-shot image is uploaded. The shooting point information indicates the location of the corresponding shooting point, which can be located using GPS positioning or a publicly recognized address description text. The photographer's account information is the account information logged in by the inspection personnel when using the system on the terminal.

[0042] The inspection point punch-in information generated by executing the punch-in interactive operation. The punch-in interactive operation can be that the inspection personnel brings the near-field communication device of their terminal close to the near-field communication device equipped at the inspection point, and then hand-writes the inspector's own name on the handwritten signature panel interface displayed on the terminal; it can also be that the inspection personnel uses the camera of their terminal to take a picture at the inspection point to identify the inspection point feature code, and then hand-writes the inspector's own name on the handwritten signature panel interface displayed on the terminal. The inspection point punch-in information includes the inspection point location information, punch-in identity information, and punch-in time. The punch-in identity information includes the puncher account information and the puncher signature image.

[0043] Step S1200: When the preset fence range of the inspection point is smaller than the preset range, a live playback voice corresponding to the one-time verification code is sent to the audio playback device at the inspection point, and the terminal uploads the verification code obtained by collecting and parsing the corresponding audio to be verified;

[0044] For each inspection point in the inspection route whose preset fence range is less than the preset threshold, the system may set the preset fence range of each inspection point to be a preset length from the inspection point by default, and the preset length is less than 100 meters, and can be set as needed by a person skilled in the art, such as 10 meters. In addition, an audio playback device for performing on-site verification may be provided at the site of each inspection point.

[0045] In one embodiment, after the inspection personnel complete all matters that need to be inspected at the inspection point on site, they can scan or directly enter the unique identification code of the inspection point through the operating terminal, and then the terminal sends the unique identification code to the server to trigger an on-site verification event.

[0046] The server responds to the on-site verification event and receives the unique identification code of the inspection point. A random number generation algorithm can be used to generate a verification code consisting of multiple characters. The verification code is only valid for this on-site verification event, so the verification code is used as a one-time verification code, that is, the verification code corresponding to each on-site verification event is different to prevent cheating. The number of characters, character type, and random number generation algorithm of the one-time verification code can be flexibly set by technicians in this field. For example, the number of characters can be sixteen, the character type can include any one or any combination of uppercase letters, lowercase letters, and numbers, and the random number generation algorithm can be any one or any combination of random number generation algorithms, API-charAit, etc.

[0047] An open source speech synthesis model is used to generate a speech corresponding to the one-time verification code according to a pre-learned pronunciation voiceprint, as the on-site playback voice, then, an audio playback instruction corresponding to the audio playback device is constructed to enable the audio playback device to play the audio, and an audio collection instruction corresponding to the audio device that enables the terminal to collect sound and complete the collection after the preset collection time is constructed. After that, according to the unique identification code of the inspection point, that is, the unique identification of the audio playback device set at the inspection point, and the communication address of the terminal, the audio playback instruction and the on-site playback voice are sent to the audio playback device, and the audio collection instruction is sent to the terminal, so that the control audio playback device starts to play the on-site playback voice after receiving the audio playback instruction and the on-site playback voice. At the same time, the control terminal starts to collect sound after receiving the audio collection instruction, and completes the collection after the preset collection time, and obtains the corresponding audio as the audio to be verified. The preset collection time is slightly longer than the playback time of the on-site playback voice to ensure that the on-site playback voice can be fully collected. Those skilled in the art can pre-set the preset collection time as needed according to the disclosure herein.

[0048] After obtaining the audio to be verified, the terminal can use an open source speech recognition algorithm to identify the speech content in the audio to be verified, obtain the corresponding speech content text, and upload it to the server. The server receives the speech content text as the code to be verified.

[0049] The speech synthesis model is pre-trained to a convergence state, learns at least one pronunciation voiceprint, and generates speech corresponding to the input text pronounced with various pronunciation voiceprints. Those skilled in the art can flexibly use the existing open source speech synthesis model.

[0050] The speech recognition algorithm can be implemented as an open source model that has been pre-trained to a convergence state and has learned the ability to recognize text in speech, such as SeamlessM4T, Whisper, Buzz, etc. Those skilled in the art can choose to implement one as needed, or call an open source speech recognition interface that can recognize text in speech, such as IBM Watson Speech to Text, Google Speech-to-Text, etc.

[0051] Step S1300: when the code to be verified is consistent with the one-time verification code, obtaining the inspection point punch-in information and the inspection real-shot image corresponding to the on-site punch-in operation and on-site shooting operation performed by the terminal within a preset time period, forming the inspection point detailed information;

[0052] The preset time length is used to restrict the patrol personnel to complete the on-site clock-in interaction and on-site photography operations within a limited time after they have completed all matters that need to be inspected at the corresponding patrol point. If all operations are completed beyond this limited time, it is deemed that the patrol personnel did not complete these operations on-site; otherwise, it is deemed that the patrol personnel completed these operations on-site. Technical personnel in this field can pre-set the preset time length as needed according to the disclosure herein.

[0053] The server compares the code to be verified with the one-time verification code. When the comparison is consistent, it is confirmed that the on-site verification of the inspection point has passed. Then, within the preset time, the server responds to the on-site clocking event of the inspection point during the execution of the inspection task, obtains the location information and the unique identifier of the inspection point uploaded by the inspection personnel operating the terminal, and confirms that the preset fence range of the inspection point is greater than or equal to the preset range, and the location information is within the preset fence range, and obtains the inspection point clocking information generated by the terminal uploading the inspection point where the inspection personnel perform the clocking interaction operation. The clocking interaction operation can be that the inspection personnel hand-writes the name of the inspection personnel in the handwritten signature panel interface displayed on the terminal. The inspection point clocking information includes the inspection point information, clocking identity information, and clocking time. The clocking identity information includes the account information of the clocking person and the signature image of the clocking person. The clocking person account information is the account information logged in by the inspection personnel when using the system on the terminal. The clocking time is the timestamp corresponding to the completion of the clocking interaction operation.

[0054] Furthermore, the server responds to the on-site shooting event of the inspection point during the execution of the inspection task, obtains the on-site real-shot image uploaded by the terminal at the inspection point, and determines the corresponding on-site shooting information, generates an on-site shooting watermark image containing the on-site shooting information, and synthesizes the on-site shooting watermark image to the preset watermark position in the on-site real-shot image to obtain the inspection real-shot image. The on-site shooting information includes the shooting time, shooting point information, and the account information of the photographer. The shooting time is the corresponding timestamp when the on-site real-shot image is uploaded. The shooting point information indicates the location of the corresponding shooting point, which can be located using GPS positioning or a publicly recognized address description text. The photographer's account information is the account information logged in by the inspection personnel when using the system on the terminal.

[0055] Step S1400: Send the inspection point detailed information to the task administrator of the inspection task.

[0056] The inspection point detailed information is sent to the task administrator of the inspection task, so that the task administrator can be informed of the detailed execution status of the inspection point during the execution of the inspection task.

[0057] It can be known from the typical embodiments of the present application that the technical solution of the present application has many advantages, including but not limited to the following aspects:

[0058] This application first determines whether the terminal location information is within the preset fence range. Only when the patrol personnel actually arrives at the preset fence range can the on-site clock-in and shooting operations be triggered, ensuring that the patrol personnel must personally arrive near the inspection point to complete the clock-in and shooting, thereby avoiding the possibility of remote clock-in and shooting.

[0059] Secondly, when the preset fence range of the inspection point is smaller than the preset range, the system will send a one-time verification code to the on-site audio playback device and collect and parse the corresponding audio to be verified through the terminal. This enhances the randomness and unpredictability of the inspection process, making it impossible for inspectors to prepare or forge on-site verification information in advance, because each verification code is temporarily generated and used once.

[0060] Furthermore, only when the verification code is consistent with the one-time verification code, the inspector needs to complete the on-site clock-in and photo shooting within the preset time. Verify the actual presence of the inspector and complete the clock-in and photo shooting within a limited time to avoid clocking in and taking photos away from the site.

[0061] Finally, the check-in information and real-time images of the inspection points are sent to the task manager, providing detailed data support for the supervision and management of the inspection work. This improves the transparency of the inspection check-in work and provides a reliable basis for checking the inspection check-in work.

[0062] To sum up, without the need for any property management personnel to intervene in supervision, the quality and efficiency of the inspection work can be improved, providing strong support for inspection management and ensuring the fairness and accuracy of the inspection work.

[0063] In a further embodiment, step S1300, obtaining the inspection point punch-in information and the inspection real-shot image corresponding to the on-site punch-in operation and the on-site shooting operation performed by the terminal within a preset time period, includes the following steps:

[0064] Step S1310: respond to the on-site clock-in event of the inspection point during the execution of the inspection task, and obtain the inspection point clock-in information generated by the terminal uploading the on-site clock-in interactive operation performed at the inspection point within a preset time period;

[0065] The preset time length is used to restrict the patrol personnel to complete the on-site clock-in interaction and on-site photography operations within a limited time after they have completed all matters that need to be inspected at the corresponding patrol point. If all operations are completed beyond this limited time, it is deemed that the patrol personnel did not complete these operations on-site; otherwise, it is deemed that the patrol personnel completed these operations on-site. Technical personnel in this field can pre-set the preset time length as needed according to the disclosure herein.

[0066] When the code to be verified is consistent with the one-time verification code, it is confirmed that the on-site verification of the inspection point has passed. Then, within the preset time, the inspector enters the name of the inspector by handwriting on the handwritten signature panel interface displayed on his terminal to implement the on-site clock-in interactive operation at the inspection point. The terminal generates a signature image containing the signature as the clock-in signature image by enabling the pre-packaged image generation code of the handwritten signature panel interface, and uses the account information corresponding to the property management system used by the inspector on the terminal as the clock-in account information, and uses the current timestamp as the clock-in time. Then, the terminal uses the clock-in account information and the clock-in signature image to form the clock-in identity information, and uses the clock-in identity information, clock-in time and the unique identification code of the inspection point previously submitted by the terminal to the server to form the clock-in information of the inspection point, which is uploaded to the server to trigger the on-site clock-in event of the inspection point.

[0067] The server responds to the checkpoint punch-in event during the execution of the inspection task, receives the checkpoint punch-in information uploaded by the terminal, queries the checkpoint location table according to the unique identification code of the checkpoint in the checkpoint punch-in information, determines the checkpoint location information associated with the unique identification code mapping, appends the checkpoint location information to the checkpoint punch-in information, returns the checkpoint punch-in information and the punch-in success notification to the terminal, and updates the checkpoint status of the checkpoint during the inspection task execution of the terminal to punch-in success. The checkpoint location table includes the unique identification code of each checkpoint and the checkpoint location information associated with its mapping. The checkpoint location information indicates the location of the corresponding checkpoint, which can be located using GPS or a publicly recognized address description text.

[0068] Step S1320, responding to the on-site shooting event of the inspection point during the execution of the inspection task, obtaining the on-site real-time image shot and uploaded by the terminal at the inspection point within the preset time, and determining the corresponding on-site shooting information, wherein the on-site shooting information includes the shooting time, the shooting point information, and the account information of the shooter;

[0069] Furthermore, the inspector uses the camera of his terminal to take real-life images at the inspection point and then uploads the real-life images to the server, as well as the current shooting location information and the account information corresponding to the property management system used by the inspector on the terminal.

[0070] The server receives the on-site real-shot image, current shooting location information and account information, responds to the on-site shooting event of the inspection point during the execution of the inspection task, uses the current timestamp as the shooting time, the current shooting location information as the shooting point information, and the account information as the account information of the photographer, and the shooting time, shooting point information, and photographer account information constitute the on-site shooting information.

[0071] Step S1330: Generate a live shooting watermark image containing the live shooting information, and synthesize the live shooting watermark into the live real shooting image according to a preset watermark position to obtain the inspection real shooting image.

[0072] The rendering interface provided by OpenGL is called to generate a live shooting watermark image containing the live shooting information, the live shooting watermark image is synthesized to a preset watermark position in the live real-shot image, the inspection real-shot image is obtained, and the inspection real-shot image is returned to the terminal.

[0073] In this embodiment, by responding to the on-site clocking-in events and on-site shooting events of the inspection points during the execution of the inspection task, the corresponding inspection point clocking-in information and the inspection real-shot images are constructed, so that the on-site shooting information is placed in the watermark of the inspection real-shot image, so that it can be visualized for inspection personnel and task managers to check at any time. Secondly, the clocking-in interactive operation is flexible and convenient, which improves the clocking-in efficiency and information processing efficiency.

[0074] In a further embodiment, step S1310, responding to the on-site clock-in event of the inspection point during the execution of the inspection task, obtaining the inspection point clock-in information generated by the terminal uploading the on-site clock-in interactive operation performed at the inspection point within a preset time, includes the following steps:

[0075] Step S1311, comparing the account information of the designated person in the pre-designated identity information of the inspection task with the account information of the clock-in person in the clock-in identity information in the inspection point clock-in information to obtain a first comparison result;

[0076] The designated person's account information is compared with the card-punching person's account information for consistency. When the comparison is consistent, the first comparison result is confirmed to be consistent; when not consistent, the first comparison result is confirmed to be inconsistent.

[0077] Step S1312: using a preset signature recognition model to determine the confidence level between the signature image of the designated person in the pre-designated identity information and the signature image of the clock-in person in the clock-in identity information;

[0078] The signature recognition model is pre-trained to a convergence state and learns the ability to determine the confidence of the signature of the same person between two input signature images.

[0079] The signature image of the designated person and the signature image of the clock-in person are input into the two image branch networks in the signature recognition model respectively, and the image semantic information of the corresponding signature images is extracted by the two image branch networks respectively. It is not difficult to understand that the image semantic information includes but is not limited to the shape, font type, size, writing habits, etc. of the handwriting, and the corresponding handwriting feature vector is obtained. The classification layer in the signature recognition model determines the vector distance between the two image feature vectors as the confidence of the signature of the same person.

[0080] Step S1313: When the first comparison result indicates that there is a difference and / or the confidence of the fan signature is lower than a preset threshold, it is confirmed that the identity authentication of the punch-in identity information has failed, and an identity anomaly report describing the first comparison result and / or the confidence of the fan signature is generated accordingly.

[0081] If the first comparison result is inconsistent, that is, there is a difference in the representation, it can be concluded that the inspector who punched in is not the same person as the designated inspector, so it is confirmed that the identity verification of the punch-in identity information fails, and an abnormal description text describing the first comparison result is generated accordingly. Those skilled in the art can flexibly implement the abnormal description text based on the disclosure herein.

[0082] If the confidence of the same person's signature is lower than the preset threshold, it means that the signature of the inspector who punched in and the signature of the designated inspector are not signed by the same person, so the identity verification of the punch-in identity information is confirmed to fail, and an abnormal description text describing the first comparison result is generated accordingly. Those skilled in the art can flexibly implement the abnormal description text and the preset threshold according to the disclosure herein.

[0083] When the first comparison result indicates that there are differences and / or the fan signature confidence is lower than a preset threshold, the abnormal description text describing the first comparison result and / or the abnormal description text of the fan signature confidence constitutes an identity anomaly report.

[0084] In this embodiment, identity verification is performed by matching the account information and signature image of the person punching the card with the account information and signature image of the designated person. When the identity verification fails, a corresponding identity anomaly report is generated to achieve efficient and accurate identity verification with high explainability, thereby ensuring the recognition of property management system users.

[0085] In a further embodiment, before step S1312, using a preset signature recognition model to determine the same person signature confidence between the signature image of the designated person in the pre-designated identity information and the signature image of the punching person in the punching identity information, includes:

[0086] Step S2312: Use a first training set to train the signature recognition model to a convergent state, and learn the ability to determine the confidence of the signature between two input signature images. The first training set includes multiple first training samples and their first supervisory labels. The first training sample includes two signature images, and the first supervisory label marks the confidence of the signature between the two signature images.

[0087] The signature recognition model can be a dual-tower model, which includes two image branch networks and a classification layer in terms of the model structure. The two image branch networks are the same image feature extraction model. The image feature extraction model can adopt a model used to extract image features. The recommended model is EffectiveNetV2. Other models such as the YOLO series model, Resnet model, Vi T (Vision Transformer) model, etc. can also be adopted. The classification layer is used for binary classification tasks and can be any vector distance algorithm such as the Euclidean distance algorithm, the cosine similarity algorithm, the inner product, etc.

[0088] Collect signature images obtained by hand-signing multiple times by multiple people in advance, split these signature images into two groups, each group contains the same number of signature images, then randomly extract one signature image from each of the two groups each time, and obtain the corresponding two signature images to form a single first training sample, until all signature images are extracted from the two groups, thereby obtaining multiple first training samples, and then annotate the supervisory label of the first training sample according to whether the two signature images in each first training sample are signed by the same person, for example, if the two signature images in the first training sample are signed by the same person, annotate the first supervisory label of the first training sample as 1; otherwise, annotate the first supervisory label of the first training sample as 0. Collect all first training samples and their first supervisory labels to form the first training set.

[0089] A single first training sample and its first supervision label in the first training set are called, and two signature images in the first training sample are input into two image branch networks in the signature recognition model respectively. The two image branch networks respectively extract image semantic information of the corresponding signature images. It is not difficult to understand that the image semantic information includes but is not limited to the shape, font type, size, writing habits, etc. of the handwriting, and the corresponding handwriting feature vector is obtained. The classification layer in the signature recognition model determines the vector distance between the two image feature vectors as the confidence of predicting the signature of the same person, and calls the preset cross entropy loss function. The technical personnel in this field can flexibly adapt it according to prior knowledge or experimental experience. The method is configured to calculate a cross entropy loss value corresponding to the predicted confidence of the same person's signature based on the supervisory label of the first training sample. When the cross entropy loss value is lower than or equal to a preset threshold, it indicates that the signature recognition model has been trained to a convergence state, thereby terminating the signature recognition model training; when the cross entropy loss value exceeds the preset threshold, it indicates that the signature recognition model has not converged, and then the model is gradient updated according to the cross entropy loss value, and the weight parameters of each link of the model are usually corrected through back propagation to make the model further close to convergence, and then, other first training samples and their first supervisory labels are continuously called to perform iterative training on the signature recognition model until the model is trained to a convergence state.

[0090] In this embodiment, the implementation process of training the signature recognition model to a convergence state is disclosed to ensure the accuracy and reliability of the confidence of the same person's signature obtained by the signature recognition model.

[0091] In a further embodiment, step S1330, generating a live shooting watermark image containing the live shooting information, synthesizing the live shooting watermark into the live shooting image according to a preset watermark position, and obtaining the inspection real shooting image, comprises the following steps:

[0092] Step S1331, determining the original image representation of the inspection real-shot image, and symmetrically encrypting the original image representation according to the watermark position information of the on-site watermark image in the inspection real-shot image to obtain an information encapsulation ciphertext;

[0093] A preset watermark detection model is used to perform target detection on the inspection real-shot image. By performing convolution operations on the inspection real-shot image, the image semantic features corresponding to the inspection real-shot image are extracted, including the color, texture, edge, contour, and deep, complex and abstract semantic information of the inspection real-shot image, and a corresponding multi-scale feature map is generated. The on-site shot watermark image in the inspection real-shot image is used as the target detection object to predict multiple candidate frame areas on the feature map. The overlapping candidate frame areas are filtered through non-maximum suppression, and the anchor frame area of ​​the target detection object in the frame is inferred. The position information of the anchor frame area in the inspection real-shot image is obtained as the watermark position information of the on-site shot watermark image in the inspection real-shot image.

[0094] A preset image representation model is used to take the inspection real-shot image as input, extract the color, texture, edge, contour, deep complex and abstract semantic information of the inspection real-shot image, and obtain an image feature vector that vectorizes the semantic information as the original image representation.

[0095] Furthermore, the encryption function in the AES (Advanced Encryption Standard) symmetric encryption and decryption algorithm is used to use the watermark position information as a symmetric encryption and decryption key to symmetric encrypt the original image representation and obtain a corresponding ciphertext as the information encapsulation ciphertext.

[0096] The image representation model is pre-trained to a convergence state and acquires the ability to represent the image semantic features of the input image. The specific selection of the image representation model can be any readily available open source model such as the Resnet series, VGG, Mobile series, etc.

[0097] The watermark detection model is pre-trained to a convergent state and acquires the ability to detect the watermark position information of the live-shot watermark image in the input image. The specific training process can be flexibly implemented by those skilled in the art according to the reasoning process disclosed herein. The watermark detection model can adopt a model suitable for the target detection task in terms of model selection. The YOLOV8 model is recommended. Other models such as U-Net model, Ret i naNet model, CornerNet model, etc. can also be adopted.

[0098] Step S1332: Generate an anti-tampering code including the information encapsulation ciphertext, and synthesize the anti-tampering code into the inspection real-shot image.

[0099] Generate the tamper-proof code containing the information encapsulation ciphertext, the tamper-proof code may be a two-dimensional code, and call the rendering interface provided by OpenGL to generate a watermark image containing the tamper-proof code.

[0100] In one embodiment, the watermark image is adjusted to a preset size and then its transparency is adjusted to a preset transparency, and the adjusted watermark image is synthesized to a preset position in the inspection real-shot image. The preset size, preset transparency, and preset position can be set by those skilled in the art as needed.

[0101] In another embodiment, a nearest neighbor interpolation algorithm is used to scale the watermark image to the size of the actual inspection image and binarize it to obtain a binary watermark image of the size with only 0 and 1, which is directly written into the LSB (Least Significant Bit) of the actual inspection image to implicitly embed the watermark image in the actual inspection image, so that the watermark image is hidden in the actual inspection image and it is difficult to distinguish the existence of the watermark image with the naked eye.

[0102] In this embodiment, by constructing the original image representation of the actual inspection image and the corresponding information encapsulation ciphertext, the tamper-proof code containing the information encapsulation ciphertext is synthesized into the actual inspection image, which can ensure the security of image information and prevent information tampering. At the same time, it also provides a reliable basis for information traceability verification, greatly improving the accuracy and reliability of inspection information. Secondly, the tamper-proof code does not need additional storage but can be retrieved and called when needed, which greatly improves information processing efficiency and does not consume additional storage space.

[0103] In a further embodiment, step S1332, generating an anti-tampering code including the information encapsulation ciphertext, and synthesizing the anti-tampering code into the inspection real-shot image, comprises the following steps:

[0104] Step S13320: In response to the image authenticity verification event, obtain the inspection real-shot image to be verified corresponding to the event and the information encapsulation ciphertext contained in the anti-tampering code therein;

[0105] The task administrator selects the inspection real-shot images provided by the inspection personnel during the inspection task execution on his terminal, and uses the inspection real-shot images as the inspection real-shot images to be verified. The terminal uploads the inspection real-shot images to be verified to the server, triggering an image authenticity verification event.

[0106] The server responds to the image authenticity verification event, receives the patrol real-shot image to be verified, identifies the tamper-proof code in the patrol real-shot image to be verified, and obtains the information encapsulation ciphertext pre-encapsulated therein.

[0107] Step S13321, according to the watermark position information of the on-site watermark image in the inspection real-shot image to be verified, symmetrically decrypt the information encapsulation ciphertext, and when the symmetrical decryption is successful, obtain the original image representation;

[0108] The watermark detection model is used to perform target detection on the real-shot inspection image to be verified. By performing a convolution operation on the real-shot inspection image to be verified, the image semantic features corresponding to the real-shot inspection image to be verified are extracted, including the color, texture, edge, contour, and deep, complex and abstract semantic information of the real-shot inspection image to be verified, and a corresponding multi-scale feature map is generated. The on-site watermark image in the real-shot inspection image to be verified is used as the target detection object to predict multiple candidate frame areas on the feature map. The overlapping candidate frame areas are filtered through non-maximum suppression, and the anchor frame area of ​​the target detection object in the frame is inferred. The position information of the anchor frame area in the real-shot inspection image to be verified is obtained as the watermark position information of the on-site watermark image in the real-shot inspection image to be verified.

[0109] Furthermore, the decryption function in the AES (Advanced Encryption Standard) symmetric encryption and decryption algorithm is used to use the watermark position information as the symmetric encryption and decryption key to symmetrically decrypt the information encapsulation ciphertext. When the symmetric decryption is successful, the plaintext corresponding to the information encapsulation ciphertext is the original image representation. The original image representation is the image representation of the inspection real-shot image before encryption corresponding to the inspection real-shot image to be verified.

[0110] Step S13322, determining the degree of match between the original image representation and the image representation to be verified corresponding to the patrol inspection real-shot image to be verified;

[0111] The image representation model is used to take the real-shot inspection image to be verified as input, extract the color, texture, edge, contour, deep complex and abstract semantic information of the real-shot inspection image to be verified, and obtain an image feature vector that vectorizes the semantic information as the representation of the image to be verified.

[0112] By traversing the original image representation and the image representation to be verified, it is determined whether the original image representation and the image representation to be verified are exactly the same. When they are exactly the same, it is confirmed that the matching degree between the original image representation and the image representation to be verified is 1; otherwise, it is confirmed that the matching degree between the original image representation and the image representation to be verified is 0.

[0113] Step S13323: When the symmetric decryption fails and / or the matching degree meets the preset conditions, it is confirmed that the authenticity verification of the real-shot inspection image to be verified fails, and an image anomaly report describing the reason for the symmetric decryption failure and / or the matching degree is generated accordingly.

[0114] If the matching degree is 0, it is confirmed that the matching degree meets the preset conditions. It can be proved that the visual effect presented by the inspection real-shot image to be verified is different from that of the inspection real-shot image before encryption, that is, the inspection real-shot image to be verified has been tampered with before encryption. Therefore, it is confirmed that the authenticity check of the inspection real-shot image to be verified has failed, and an abnormal description text describing the matching degree is generated accordingly. The abnormal description text includes an explanation of the origin of the matching degree, and the reason why the inspection real-shot image to be verified has been tampered with based on the matching degree. Those skilled in the art can flexibly implement the abnormal description text according to the disclosure herein.

[0115] If the symmetric decryption fails, it can be proved based on the principle of symmetric encryption and decryption that the watermark position information of the on-site watermark image in the real-shot inspection image to be verified has changed relative to the watermark position information before encryption, that is, the real-shot inspection image to be verified has tampered with the on-site watermark image of the corresponding real-shot inspection image before encryption. Therefore, it is confirmed that the authenticity check of the real-shot inspection image to be verified has failed, and an exception description text describing the reason for the failure of the symmetric decryption is generated accordingly. Those skilled in the art can flexibly implement the exception description text based on the disclosure herein.

[0116] When the symmetric decryption fails and / or the matching degree meets a preset condition, the abnormal description text describing the reason for the symmetric decryption failure and / or the abnormal description text of the matching degree constitute an image abnormality report.

[0117] In this embodiment, the task quality inspection model is used to perform quality verification on the actual inspection images. When the quality verification fails, a corresponding image abnormality report is generated to achieve efficient and accurate quality verification with high interpretability, thereby ensuring the recognition of property management system users.

[0118] In a further embodiment, step S1200, sending a live playback voice corresponding to the one-time verification code to the audio playback device at the inspection point, and obtaining the code to be verified obtained by the terminal uploading and parsing the corresponding audio to be verified, includes the following steps:

[0119] Step S1210, responding to the on-site verification event of the inspection point, randomly generating a one-time verification code, using a preset speech synthesis model to generate a speech corresponding to the one-time verification code pronounced with the target voiceprint as the on-site playback voice;

[0120] The speech synthesis model is pre-trained to a convergence state and acquires the ability to generate speech corresponding to the input text pronounced with the target voiceprint.

[0121] For each inspection point in the inspection route set for the inspection task, where the preset fence range is smaller than the preset threshold, an audio playback device for performing on-site verification can be provided at the inspection point, and a unique identification code is marked on the audio playback device. Since the unique identification code is unique, the unique identification code is used as the unique identification code of the inspection point. The unique identification code can be expressed as a QR code, bar code, device ID, etc. that can uniquely refer to the audio playback device. A communication connection is pre-established between the audio playback device and the server, and the communication connection can be any one of Bluetooth connection, WiFi connection, data cable connection, etc., and those skilled in the art can select one to implement as needed.

[0122] After the inspection personnel complete all matters that need to be inspected at the inspection point on site, they can scan or directly enter the unique identification code of the inspection point through the operation terminal, and then the terminal sends the unique identification code to the server to trigger an on-site verification event.

[0123] The server responds to the on-site verification event and receives the unique identification code of the inspection point. A random number generation algorithm can be used to generate a verification code consisting of multiple characters. The verification code is only valid for this on-site verification event, so the verification code is used as a one-time verification code, that is, the verification code corresponding to each on-site verification event is different to prevent cheating. The number of characters, character type, and random number generation algorithm of the one-time verification code can be flexibly set by those skilled in the art. For example, the number of characters can be sixteen, the character type can include any one or any combination of uppercase letters, lowercase letters, and numbers, and the random number generation algorithm can be any one or any combination of random number generation algorithms, API-charAit, etc.

[0124] The speech synthesis model is used to take the one-time verification code as input text to generate a speech corresponding to the target voiceprint, which is used to play to the audio playback device, so the speech is used as the live playback voice. The target voiceprint can be the voiceprint of the task manager or the voiceprint of a person other than the patrol personnel.

[0125] Step S1220: Send the on-site voice playback and audio playback instructions to the audio playback device at the inspection point, and send an audio collection instruction to the terminal to control the audio playback device to play the on-site voice playback and control the terminal to collect the corresponding audio to be verified;

[0126] The server constructs an audio playback instruction corresponding to enabling the audio playback device to play audio, and constructs an audio collection instruction corresponding to enabling the audio device of the terminal to collect sound and complete the collection after the preset collection time. After that, according to the unique identification code of the inspection point, that is, the unique identification of the audio playback device set at the inspection point, and the communication address of the terminal, the audio playback instruction and the on-site playback voice are sent to the audio playback device, and the audio collection instruction is sent to the terminal, so that the control audio playback device starts to play the on-site playback voice after receiving the audio playback instruction and the on-site playback voice. At the same time, the control terminal starts to collect sound after receiving the audio collection instruction, and completes the collection after the preset collection time to obtain the corresponding audio. Since the audio may be collected by the inspection personnel not at the scene, the audio is used as the audio to be verified. The preset collection time is slightly longer than the playback time of the on-site playback voice to ensure that the on-site playback voice can be fully collected. Those skilled in the art can pre-set the preset collection time as needed according to the disclosure herein.

[0127] Step S1230: When the terminal uses a preset voice recognition model to confirm that the pronunciation voiceprint corresponding to the audio to be verified belongs to the target voiceprint, the terminal uploads a voice content text corresponding to the audio to be verified that is parsed using a preset voice recognition algorithm as a code to be verified.

[0128] The speech recognition model is pre-trained to a convergence state and acquires the ability to determine whether the pronunciation voiceprint corresponding to the input audio belongs to the target voiceprint. The speech recognition algorithm can be implemented as an open source model that has been pre-trained to a convergence state and acquires the ability to recognize text in speech, such as Seaml essM4T, Whisper, Buzz, etc. Those skilled in the art can choose one to implement as needed, or call an open source speech recognition interface that can recognize text in speech, such as IBM Watson Speech to Text, Google Speech-to-Text, etc.

[0129] After the terminal obtains the audio to be verified, it uses the speech recognition model to take the audio to be verified as the input audio to determine whether the pronunciation voiceprint corresponding to the input audio belongs to the target voiceprint. When the pronunciation voiceprint belongs to the target voiceprint, the terminal further uses the speech recognition algorithm to identify the voice content in the audio to be verified, obtains the corresponding voice content text, and uploads it to the server. When the pronunciation voiceprint does not belong to the target voiceprint, it is directly confirmed that the on-site verification of the inspection point has failed.

[0130] The server receives the voice content text and uses it as a code to be verified.

[0131] In this embodiment, by responding to the on-site verification event of the inspection point, a one-time verification code is generated corresponding to the on-site played voice pronounced with the target voiceprint, the audio playback device is controlled to play the on-site played voice, and the terminal is controlled to collect the corresponding audio to be verified. When the terminal confirms that the pronunciation voiceprint corresponding to the audio to be verified belongs to the target voiceprint, the corresponding code to be verified is obtained by the terminal uploading the terminal. This can realize efficient and accurate verification of whether the inspectors are actually at the corresponding inspection points without the inspectors being aware of it, thereby preventing the inspectors from cheating.

[0132] In a further embodiment, step S1200, in response to the on-site verification event of the inspection point, randomly generating a one-time verification code, using a preset speech synthesis model to generate a one-time verification code corresponding to the speech pronounced with the target voiceprint, as before playing the speech on-site, includes the following steps:

[0133] Step S2200, respectively training the speech generator and the speech discriminator to a convergence state, and correspondingly learning the ability to generate speech corresponding to the input text pronounced with the target voiceprint, and learning the ability to discriminate whether the pronunciation voiceprint corresponding to the input audio belongs to the target voiceprint;

[0134] Call the prepared first training set, train the speech generator, train it to a convergent state, and learn the ability to generate speech corresponding to the input text and pronounced with the target voiceprint. The speech generator can be any one of Tacotron2, GST, DeepVoice3, ClearNet, LPCNet, Transformer-TTS, Glow-TTS, Flow-TTS, cVAE+Flow+GAN, PnG BERT, etc., and those skilled in the art can flexibly implement it.

[0135] Multiple one-time verification codes can be randomly generated as individual first training samples. Then, for each first training sample, a person with the target voiceprint can read the first training sample clearly and completely at a certain speaking speed, and record the corresponding voice as the first supervision label of the first training sample. The target voiceprint can be the voiceprint of the task manager or the voiceprint of a person other than the inspection personnel. All the first training samples and their first supervision labels are collected to form the first training set.

[0136] The prepared second training set is called to train the speech discriminator, and the speech discriminator is trained to a convergence state to learn the ability to discriminate the pronunciation voiceprint corresponding to the input audio. The speech discriminator can be any one of DeepSpeech, ESPnet, etc., and those skilled in the art can flexibly implement it.

[0137] Multiple first training samples can be selected from the first training set and used as the single second training samples. Then, the second supervisory label of each second training sample is marked to indicate that the pronunciation voiceprint corresponding to the second training sample belongs to the target voiceprint. In addition, for each one-time verification code in each selected first training sample, multiple people with the target voiceprint each read out the one-time verification code clearly and completely at a certain speaking speed, and record the corresponding voice as a single second training sample, thereby obtaining multiple second training samples. Then, the second supervisory label of each second training sample is marked to indicate that the pronunciation voiceprint corresponding to the second training sample does not belong to the target voiceprint. All the second training samples and their second supervisory labels are collected to form the second training set.

[0138] Step S2210: Connect the speech generator trained to convergence to the speech discriminator trained to convergence to construct an adversarial neural network, and use the speech generator and speech discriminator in the adversarial neural network after adversarial training as a speech synthesis model and a speech recognition model respectively.

[0139] According to the above-mentioned construction method of the first training sample and its first supervisory label, and the construction method of the second training sample and its second supervisory label, the first training set and the second training set are reconstructed accordingly.

[0140] The first training set and the second training set are called to perform at least one round of adversarial training on the adversarial neural network, so that during the adversarial training, the speech generator generates speech that is increasingly difficult to be distinguished by the speech discriminator, and the discriminator attempts to more accurately identify whether the pronunciation voiceprints of these speech belong to the target voiceprint. The speech generator and speech discriminator in the adversarial neural network after adversarial training are used as a speech synthesis model and a speech recognition model respectively.

[0141] In this embodiment, the implementation process of constructing a speech synthesis model and a speech recognition model that are pre-trained to a convergence state is disclosed to ensure the accuracy and reliability of the inference results of each model.

[0142] See also Figure 3 , a patrol punching device provided to meet one of the purposes of the present application is a functional embodiment of the patrol punching method of the present application. On the other hand, the device is a patrol punching device provided to meet one of the purposes of the present application, including a first patrol punching module 1100, a module 1200 for obtaining a code to be verified, a second patrol punching module 1300 and an information sending module 1400, wherein the first patrol punching module 1100 is used to respond to patrol task execution events, when the preset fence range of the corresponding patrol point reaches the preset range, and the location information of the terminal is within the preset fence range, the patrol point punching information and patrol real-shot images generated by the terminal performing on-site punching operations and on-site shooting operations are obtained, forming Inspection point detailed information; a module for obtaining the code to be verified 1200, which is used to send the on-site playback voice corresponding to the one-time verification code to the audio playback device at the inspection point when the preset fence range of the inspection point is smaller than the preset range, and obtain the code to be verified obtained by the terminal uploading and collecting and parsing the corresponding audio to be verified; a second inspection punch-in module 1300, which is used to obtain the inspection point punch-in information and inspection real-shot images corresponding to the on-site punch-in operation and on-site shooting operation performed by the terminal within a preset time period when the code to be verified is consistent with the one-time verification code, so as to constitute the inspection point detailed information; an information sending module 1400, which is used to send the inspection point detailed information to the task administrator of the inspection task.

[0143] In a further embodiment, the second inspection punch-in module 1300 includes: a first event response submodule, which is used to respond to the on-site punch-in event of the inspection point during the execution of the inspection task, and obtain the inspection point punch-in information generated by the terminal uploading the on-site punch-in interaction operation at the inspection point within a preset time period; a second event response submodule, which is used to respond to the on-site shooting event of the inspection point during the execution of the inspection task, and obtain the on-site real-shot image uploaded by the terminal at the inspection point within the preset time period, and determine the corresponding on-site shooting information, wherein the on-site shooting information includes the shooting time, shooting point information, and the account information of the photographer; an image watermarking submodule, which is used to generate an on-site shooting watermark image containing the on-site shooting information, and synthesize the on-site shooting watermark into the on-site real-shot image according to the preset watermark position to obtain the inspection real-shot image.

[0144] In a further embodiment, the first event response submodule includes: a first comparison submodule, which is used to compare the account information of the designated person in the pre-designated identity information of the inspection task with the account information of the clocking-in person in the clocking-in identity information in the inspection point clocking-in information to obtain a first comparison result; a confidence determination submodule, which is used to determine the signature confidence between the signature image of the designated person in the pre-designated identity information and the signature image of the clocking-in person in the clocking-in identity information using a preset signature recognition model; and an exception report submodule, which is used to confirm that the identity authentication of the clocking-in identity information has failed when the first comparison result indicates that there is a difference and / or the confidence of the signature is lower than a preset threshold, and to generate an identity exception report describing the first comparison result and / or the confidence of the signature.

[0145] In a further embodiment, before the confidence determination submodule, it includes: a first model training submodule, which is used to adopt a first training set to train the signature recognition model to a convergence state, and learn the ability to determine the confidence of the same person's signature between two input signature images, the first training set includes multiple first training samples and their first supervisory labels, the first training sample includes two signature images, and the first supervisory label marks the confidence of the same person's signature between the two signature images.

[0146] In a further embodiment, after the image watermarking submodule, it includes: a ciphertext acquisition submodule, used to determine the original image representation of the inspection real-shot image, and symmetrically encrypt the original image representation according to the watermark position information of the on-site watermark image in the inspection real-shot image to obtain the information encapsulation ciphertext; an image synthesis submodule, used to generate an anti-tampering code containing the information encapsulation ciphertext, and synthesize the anti-tampering code into the inspection real-shot image.

[0147] In a further embodiment, the module for obtaining the code to be verified 1200 includes: a third event response submodule, which is used to respond to the on-site verification event of the inspection point, randomly generate a one-time verification code, and use a preset speech synthesis model to generate a voice corresponding to the one-time verification code pronounced with the target voiceprint as the on-site playback voice; a voice collection submodule, which is used to send the on-site playback voice and audio playback instructions to the audio playback device at the inspection point, and send an audio collection instruction to the terminal to control the audio playback device to play the on-site playback voice, and control the terminal to collect the corresponding audio to be verified; a submodule for parsing the code to be verified, which is used to obtain the voice content text corresponding to the audio to be verified uploaded by the terminal using a preset speech recognition algorithm to parse it when the terminal uses a preset speech recognition model to confirm that the pronunciation voiceprint corresponding to the audio to be verified belongs to the target voiceprint.

[0148] In a further embodiment, before the third event response submodule, it includes: a second model training submodule, which is used to train the speech generator and the speech discriminator to a convergence state respectively, and correspondingly learn the ability to generate speech pronounced with the target voiceprint corresponding to the input text, and learn the ability to discriminate whether the pronunciation voiceprint corresponding to the input audio belongs to the target voiceprint; an adversarial training submodule, which is used to connect the speech generator trained to convergence to the speech discriminator trained to convergence to construct an adversarial neural network, and use the speech generator and speech discriminator in the adversarial neural network after adversarial training as a speech synthesis model and a speech recognition model respectively.

[0149] In order to solve the above technical problems, the present application also provides a computer device. Figure 4 As shown, a schematic diagram of the internal structure of a computer device. The computer device includes a processor, a computer-readable storage medium, a memory, and a network interface connected via a system bus. Among them, the computer-readable storage medium of the computer device stores an operating system, a database, and computer-readable instructions. The database may store a control information sequence. When the computer-readable instructions are executed by the processor, the processor can implement a patrol inspection and punching method. The processor of the computer device is used to provide computing and control capabilities to support the operation of the entire computer device. The memory of the computer device may store computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor can execute the patrol inspection and punching method of the present application. The network interface of the computer device is used to connect and communicate with a terminal. Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0150] In this embodiment, the processor is used to execute Figure 3 The memory stores the program code and various data required to execute the above modules or submodules. The network interface is used to transmit data between user terminals or servers. The memory in this embodiment stores the program code and data required to execute all modules / submodules in the inspection and punching device of this application, and the server can call the program code and data of the server to execute the functions of all submodules.

[0151] The present application also provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the inspection and clocking-in method of any embodiment of the present application.

[0152] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments of the present application can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a computer-readable storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0153] In summary, this application can strictly control clocking in within the restricted inspection area and prevent inspectors from cheating.

[0154] It will be understood by those skilled in the art that the various operations, methods, steps, measures, and schemes in the processes discussed in this application may be alternated, changed, combined, or deleted. Furthermore, other steps, measures, and schemes in the various operations, methods, and processes discussed in this application may also be alternated, changed, rearranged, decomposed, combined, or deleted. Furthermore, the steps, measures, and schemes in the various operations, methods, and processes in the prior art that are open source and in this application may also be alternated, changed, rearranged, decomposed, combined, or deleted.

[0155] The above description is only a partial implementation method of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A patrol inspection clocking method, characterized in that: The steps include: In response to the inspection task execution event, when the preset fence range of the corresponding inspection point reaches the preset range and the location information of the terminal is within the preset fence range, the inspection point punch-in information and the inspection real-shot image generated by the terminal performing the on-site punch-in operation and the on-site shooting operation are obtained to form the inspection point detailed information; When the preset fence range of the inspection point is smaller than the preset range, a live playback voice corresponding to the one-time verification code is sent to the audio playback device at the inspection point, and the terminal uploads the verification code obtained by collecting and parsing the corresponding audio to be verified; When the code to be verified is consistent with the one-time verification code, the inspection point punch-in information and the inspection real-shot image generated by the terminal performing the on-site punch-in operation and the on-site shooting operation within the preset time period are obtained to form the inspection point detailed information; The inspection point detailed information is sent to the task administrator of the inspection task.

2. The inspection clocking-in method according to claim 1, characterized in that: Obtaining the inspection point check-in information and the inspection real-shot image corresponding to the on-site check-in operation and on-site shooting operation performed by the terminal within a preset time period, including the following steps: In response to the on-site clock-in event of the inspection point during the execution of the inspection task, the terminal uploads the on-site clock-in information generated by the interactive operation of performing the on-site clock-in at the inspection point within the preset time period; In response to the on-site shooting event of the inspection point during the execution of the inspection task, the on-site real-time image uploaded by the terminal at the inspection point within the preset time period is obtained, and the corresponding on-site shooting information is determined, and the on-site shooting information includes the shooting time, shooting point information, and the account information of the photographer; A live shooting watermark image containing the live shooting information is generated, and the live shooting watermark is synthesized into the live real shooting image according to a preset watermark position to obtain an inspection real shooting image.

3. The inspection clocking-in method according to claim 2, characterized in that: In response to the on-site clock-in event of the inspection point during the execution of the inspection task, after the terminal uploads the on-site clock-in information generated by the interactive operation of performing the on-site clock-in at the inspection point within the preset time, the following steps are included: Compare the account information of the designated person in the pre-designated identity information of the inspection task with the account information of the clock-in person in the clock-in identity information in the inspection point clock-in information to obtain a first comparison result; Using a preset signature recognition model to determine the confidence level between the signature image of the designated person in the pre-designated identity information and the signature image of the punching person in the punching identity information; When the first comparison result indicates that there is a difference and / or the confidence of the fan signature is lower than a preset threshold, it is confirmed that the identity authentication of the punch-in identity information has failed, and an identity anomaly report describing the first comparison result and / or the confidence of the fan signature is generated accordingly.

4. The inspection clocking-in method according to claim 3, characterized in that: Before using a preset signature recognition model to determine the same person signature confidence between the signature image of the designated person in the pre-designated identity information and the signature image of the punching person in the punching identity information, the method includes: A signature recognition model is trained to a convergent state using a first training set to acquire the ability to determine the confidence of a signature between two input signature images. The first training set includes a plurality of first training samples and their first supervisory labels. The first training sample includes two signature images. The first supervisory label annotates the confidence of a signature between the two signature images.

5. The inspection clocking-in method according to claim 2, characterized in that: Generate a live shooting watermark image containing the live shooting information, synthesize the live shooting watermark into the live shooting image according to the preset watermark position, and obtain the inspection real shooting image, including the following steps: Determine the original image representation of the inspection real-shot image, and symmetrically encrypt the original image representation according to the watermark position information of the on-site watermark image in the inspection real-shot image to obtain an information encapsulation ciphertext; Generate an anti-tampering code containing the information encapsulation ciphertext, and synthesize the anti-tampering code into the inspection real-shot image.

6. The inspection clocking-in method according to claim 1, characterized in that: Sending a live playback voice corresponding to the one-time verification code to the audio playback device at the inspection point, and obtaining the code to be verified obtained by the terminal uploading and parsing the corresponding audio to be verified, including the following steps: In response to the on-site verification event of the inspection point, a one-time verification code is randomly generated, and a preset speech synthesis model is used to generate a speech corresponding to the one-time verification code pronounced with the target voiceprint as the on-site playback voice; Send the on-site voice playback and audio playback instructions to the audio playback device at the inspection point, and send an audio collection instruction to the terminal to control the audio playback device to play the on-site voice playback and control the terminal to collect the corresponding audio to be verified; When the terminal uses a preset voice recognition model to confirm that the pronunciation voiceprint corresponding to the audio to be verified belongs to the target voiceprint, the terminal uploads the voice content text corresponding to the audio to be verified parsed by a preset voice recognition algorithm as the code to be verified.

7. The inspection clocking-in method according to claim 6, characterized in that: In response to the on-site verification event of the inspection point, a one-time verification code is randomly generated, and a preset speech synthesis model is used to generate a speech corresponding to the one-time verification code pronounced with the target voiceprint, as before the on-site voice playback, including the following steps: The speech generator and the speech discriminator are trained to convergence state respectively, and the ability to generate speech corresponding to the input text pronounced with the target voiceprint is learned, and the ability to discriminate whether the pronunciation voiceprint corresponding to the input audio belongs to the target voiceprint is learned; The speech generator trained to convergence is connected to the speech discriminator trained to convergence to construct an adversarial neural network, and the speech generator and speech discriminator in the adversarial neural network after adversarial training are used as a speech synthesis model and a speech recognition model respectively.

8. A patrol inspection punch card device, characterized in that: include: The first inspection clock-in module is used to respond to the inspection task execution event. When the preset fence range of the corresponding inspection point reaches the preset range and the location information of the terminal is within the preset fence range, the inspection point clock-in information and the inspection real-shot image generated by the terminal performing the on-site clock-in operation and the on-site shooting operation are obtained to form the inspection point detailed information; The module for acquiring the code to be verified is used to send the on-site playback voice corresponding to the one-time verification code to the audio playback device at the inspection point when the preset fence range of the inspection point is smaller than the preset range, and to obtain the code to be verified obtained by the terminal uploading the collected and parsed corresponding audio to be verified; The second inspection punching module is used to obtain the inspection point punching information and inspection real-shot images generated by the terminal performing the on-site punching operation and on-site shooting operation within a preset time period when the code to be verified is consistent with the one-time verification code, so as to form the inspection point detailed information; The information sending module is used to send the inspection point detailed information to the task administrator of the inspection task.

9. A computer device comprising a central processing unit and a memory, characterized in that: The central processing unit is used to call and run the computer program stored in the memory to execute the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: It stores a computer program implemented according to the method described in any one of claims 1 to 7 in the form of computer-readable instructions, and when the computer program is called and executed by a computer, the steps included in the corresponding method are executed.

Images