Inspection evidence storage method and device, equipment and medium

Abstract

The invention relates to an inspection evidence storage method and device, equipment and medium in the technical field of property information, and the method comprises the steps: responding to an inspection task execution event, and obtaining inspection point clock-in information and inspection point real shooting information correspondingly generated when a terminal executes a field clock-in operation and a field shooting operation at a corresponding inspection point, the method comprises the steps of forming inspection point detail information, performing symmetric encryption on original image representation corresponding to an inspection real-shot image according to watermark position information in the inspection real-shot image contained in the inspection point real-shot information to obtain an information encapsulation ciphertext, generating a tamper-proof code containing the information encapsulation ciphertext, and synthesizing the tamper-proof code into the inspection real-shot image to obtain the tamper-proof information of the inspection real-shot image. And generating an information check code corresponding to the detailed information of the inspection point, submitting the inspection point information and the information check code to the block chain platform, and obtaining a block chain evidence storage certificate returned by the block chain platform. According to the invention, highly safe and non-tampering inspection information evidence storage is realized.

Description

Technical Field

[0001] The present application relates to the field of property information technology, and in particular to a patrol inspection and evidence storage method and its corresponding device, computer equipment, and computer-readable storage medium. Background Art

[0002] With the rapid development of digital technology, the importance of electronic evidence in legal proceedings, legal disputes and arbitration mediation has become increasingly prominent. However, the authenticity, integrity and immutability of electronic evidence have always been issues that need to be urgently resolved in judicial practice.

[0003] Traditional electronic evidence preservation methods often rely on centralized storage systems, which pose the risk of data being tampered with or lost, and are difficult to meet the high legal requirements for evidence. Other electronic evidence preservation methods also use watermark cameras to add real time and location watermarks to photos, providing additional verification information for photos and enhancing the authenticity and credibility of photos. However, watermark technology alone cannot completely solve all problems in electronic evidence preservation, especially when faced with advanced tampering methods, its protection effect is limited.

[0004] In view of the shortcomings of traditional practices, the applicant has been engaged in research in related fields for a long time and has taken a different approach to solve industry problems in the field of property information technology. Summary of the invention

[0005] The primary purpose of the present application is to solve at least one of the above problems and to provide a patrol inspection and evidence storage method and its corresponding device, computer equipment, and computer program product.

[0006] In order to meet the various objectives of this application, this application adopts the following technical solutions:

[0007] A patrol inspection and evidence storage method provided to meet one of the purposes of this application includes the following steps:

[0008] In response to the inspection task execution event, the inspection point punch-in information and inspection point real-time shooting information generated by the terminal performing the on-site punch-in operation and on-site shooting operation at the corresponding inspection point are obtained to form the inspection point detailed information;

[0009] According to the watermark position information in the inspection real-shot image contained in the inspection point real-shot information, the original image representation corresponding to the inspection real-shot image is symmetrically encrypted to obtain the information encapsulation ciphertext, and the tamper-proof code containing the information encapsulation ciphertext is generated, and the tamper-proof code is synthesized into the inspection real-shot image;

[0010] Generate an information verification code corresponding to the inspection point details, submit the inspection point information and the information verification code to the blockchain platform, and obtain the blockchain evidence certificate returned by the blockchain platform.

[0011] On the other hand, a patrol inspection evidence storage device provided to meet one of the purposes of the present application includes an event response module, an image synthesis module and a blockchain evidence storage module, wherein the event response module is used to respond to patrol inspection task execution events, obtain patrol inspection point punch-in information and patrol inspection point real-shot information generated by the terminal performing on-site punch-in operations and on-site shooting operations at the corresponding patrol inspection points, and constitute patrol inspection point detailed information; the image synthesis module is used to symmetrically encrypt the original image representation corresponding to the patrol inspection real-shot image according to the watermark position information in the patrol inspection real-shot image contained in the patrol inspection point real-shot information to obtain an information encapsulation ciphertext, generate an anti-tampering code containing the information encapsulation ciphertext, and synthesize the anti-tampering code into the patrol inspection real-shot image; the blockchain evidence storage module is used to generate an information verification code corresponding to the patrol inspection point detailed information, submit the patrol inspection point information and the information verification code to the blockchain platform, and obtain the blockchain evidence storage certificate returned by the blockchain platform.

[0012] On the other hand, a computer device provided to meet one of the purposes of the present application includes a central processing unit and a memory, wherein the central processing unit is used to call and run a computer program stored in the memory to execute the steps of the inspection and evidence storage method described in the present application.

[0013] On the other hand, a computer program product provided to meet another purpose of the present application includes a computer program / instruction, which, when executed by a processor, implements the steps of the method described in any embodiment of the present application.

[0014] The technical solution of this application has many advantages, including but not limited to the following aspects:

[0015] Firstly, by responding to inspection task execution events, this application can obtain in real time the inspection point check-in information and inspection real-time images, wherein the inspection real-time images also include on-site watermark images, thereby ensuring the timeliness and integrity of information processing.

[0016] Secondly, watermark technology and symmetric encryption technology are used to perform on-site information watermark processing and anti-tampering processing on the actual inspection images to enhance the security and integrity of the image information and ensure that the image information is not illegally tampered with during transmission and storage.

[0017] Finally, by submitting the inspection point details and information verification code to the blockchain platform, a blockchain evidence certificate is obtained. By utilizing the immutability and transparency of blockchain technology, strong legal support is provided for all details of the inspection process, and authentic and reliable electronic evidence of the inspection is obtained.

[0018] To sum up, ensuring the orderly and efficient execution of inspection work and converting all inspection information produced into legally reliable electronic evidence provides strong technical support for legal proceedings and arbitration mediation. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0020] Figure 1 A schematic diagram of a network architecture of an exemplary application environment of the present application;

[0021] Figure 2 A flowchart of a typical embodiment of the inspection and evidence storage method of the present application;

[0022] Figure 3 This is a principle block diagram of the inspection and evidence storage device of this application;

[0023] Figure 4 A schematic diagram of the structure of a computer device used in this application. DETAILED DESCRIPTION

[0024] The following will describe the working principle of the inspection and evidence storage method of the present application and its implementation in the network architecture.

[0025] refer to Figure 1 , the network architecture of the present application is composed of a terminal device 80 and a server 82. The terminal device 80 can be a mobile device used by a task manager or an inspector, such as a smart phone or a tablet computer, wherein the task manager can use the terminal device to trigger an inspection task issuance request, and the inspector can use the terminal device to receive inspection task details, take actual inspection images during the inspection task execution, and submit these images as proof of the completion of the inspection task. The server 82 is the core processing unit of the present method, and a property management system is deployed. It is responsible for executing a series of operations such as inspection task issuance, parameter push, actual inspection verification, status update, etc., and provides verification services, evidence storage services, clock-in services, and other possible execution services. The server 82 can be a single server or a front-end server in a distributed deployed service cluster to improve the processing power and reliability of the system.

[0026] The inspection tasks assigned in this application refer to a series of work activities planned in the field of property management to maintain property facilities, improve service quality or respond to the needs of owners. These inspection tasks usually involve inspectors with different work roles, such as property service personnel, maintenance technicians, security monitoring personnel, etc.

[0027] Each inspection task has its corresponding task execution process, which is the core part of the inspection task. It describes in detail a series of steps from task assignment to task completion. This process starts with the task manager submitting an inspection task issuance request through the property management system, and then the system distributes the task details to the corresponding inspectors. The inspectors perform the work according to the assigned task details and collect images at the inspection points of the task to prove the completion of the task. The task execution process also includes the verification of these images, and automatically updates the task status based on the verification results, and sends status update notifications to relevant personnel in real time to ensure the transparency and traceability of task execution.

[0028] Inspection routes can be set in the task execution process, which include at least one inspection point. Inspection points are specific pre-defined stages or milestones in the task execution process, which are crucial to the successful completion of the task. In the inspection task, the inspection route set can include but is not limited to inspection points corresponding to the safety availability of materials, the safety availability of facilities, the construction quality, and the maintenance quality. At these inspection points, the inspectors need to collect and submit the actual inspection images completed at the inspection points. These images will then be verified by the system's built-in image analysis engine to ensure that the inspection task meets the established standards and quality requirements.

[0029] The task administrator of the present application submits an inspection task distribution request corresponding to the inspection task through the server 82, and the server 82 responds to the request and obtains the inspection task, which in one embodiment includes the inspection route, designated inspection personnel, execution start and end time, execution effective time, inspection details, etc. Then, the server 82 pushes the inspection task to the terminal device 80 of the corresponding inspection personnel.

[0030] The inspection personnel use the camera function on their terminal device 80 to collect the inspection real-shot images according to the provided image capture parameters and submit them to the server 82. After receiving the inspection real-shot images, the image verification mechanism of the server 82 calls the image analysis engine for verification. The image analysis engine uses an adaptive learning algorithm to optimize the analysis accuracy and efficiency based on historical verification data, and calls the corresponding functional modules for verification, such as calling the signature recognition functional module to verify whether the signature is signed by the same person, calling the quality inspection functional module to identify whether there is any quality abnormality at the inspection site, and calling the image matching functional module to match whether the image to be verified matches its original image.

[0031] According to the verification results, the status update module of the server 82 automatically updates the task status and sends a task status update notification to the task manager and the inspection personnel. If the verification results of all key nodes are passed, the server 82 can automatically generate a task summary report and send a notification of the task summary report through the property management system.

[0032] From the above principle description, it can be seen that the inspection task processing method of the present application can ensure the transparency and credibility of the inspection task execution, while providing an efficient and reliable task collaboration and management platform, constituting a property collaboration management system, which is suitable for a variety of property management and service scenarios.

[0033] A patrol inspection and evidence storage method of the present application can be programmed as a computer program product and deployed in a client or server for execution. For example, in the exemplary application scenario of the present application, it can be deployed and implemented in the server of an independent station store, thereby accessing the interface opened after the computer program product is running and performing human-computer interaction with the process of the computer program product through a graphical user interface to execute the method.

[0034] See also Figure 2 In a typical embodiment of the present application, a patrol inspection and evidence storage method includes the following steps:

[0035] Step S1100, responding to the inspection task execution event, obtaining the inspection point punch-in information and inspection point real-time shooting information generated by the terminal performing the on-site punch-in operation and on-site shooting operation at the corresponding inspection point, and forming the inspection point detailed information;

[0036] The server responds to the on-site clocking-in event of the inspection point during the execution of the inspection task, and obtains the inspection point clocking-in information generated by the terminal uploading the clocking-in interactive operation at the inspection point. The clocking-in interactive operation can be that the inspection personnel puts the near-field communication device of their terminal close to the near-field communication device equipped at the inspection point, and then hand-writes the inspector's own name on the handwritten signature panel interface displayed on the terminal; it can also be that the inspection personnel uses the camera of their terminal to take a picture at the inspection point to identify the inspection point feature code of the inspection point, and then hand-writes the inspector's own name on the handwritten signature panel interface displayed on the terminal. The inspection point clocking-in information includes the inspection point information, clocking-in identity information, and clocking-in time. The clocking-in identity information includes the account information of the clocking-in person and the signature image of the clocking-in person.

[0037] The server responds to the on-site shooting event of the inspection point during the execution of the inspection task, obtains the on-site real-shot image uploaded by the terminal at the inspection point, determines the corresponding on-site shooting information, generates an on-site shooting watermark image containing the on-site shooting information, synthesizes the on-site shooting watermark image to the preset watermark position in the on-site real-shot image, obtains the inspection real-shot image, and then, the inspection real-shot image and the on-site shooting information constitute the inspection point real-shot information. The on-site shooting information includes shooting time, shooting point information, and photographer account information.

[0038] Step S1200: according to the watermark position information in the inspection real-shot image contained in the inspection point real-shot information, symmetrically encrypt the original image representation corresponding to the inspection real-shot image to obtain the information encapsulation ciphertext, generate the tamper-proof code containing the information encapsulation ciphertext, and synthesize the tamper-proof code into the inspection real-shot image;

[0039] A preset watermark detection model is used to perform target detection on the inspection real-shot image. By performing convolution operations on the inspection real-shot image, the image semantic features corresponding to the inspection real-shot image are extracted, including the color, texture, edge, contour, and deep, complex and abstract semantic information of the inspection real-shot image, and a corresponding multi-scale feature map is generated. The on-site shot watermark image in the inspection real-shot image is used as the target detection object to predict multiple candidate frame areas on the feature map. The overlapping candidate frame areas are filtered through non-maximum suppression, and the anchor frame area of ​​the target detection object in the frame is inferred. The position information of the anchor frame area in the inspection real-shot image is obtained as the watermark position information of the on-site shot watermark image in the inspection real-shot image.

[0040] A preset image representation model is used to take the inspection real-shot image as input, extract the color, texture, edge, contour, deep complex and abstract semantic information of the inspection real-shot image, and obtain an image feature vector that vectorizes the semantic information as the original image representation.

[0041] Furthermore, the encryption function in the AES (Advanced Encryption Standard) symmetric encryption and decryption algorithm is used to use the watermark position information as the symmetric encryption and decryption key to symmetrically encrypt the original image representation, obtain the corresponding ciphertext as the information encapsulation ciphertext, and generate a tamper-proof code containing the information encapsulation ciphertext. The tamper-proof code can be a two-dimensional code. The rendering interface provided by OpenGL is called to generate a watermark image containing the tamper-proof code. In one embodiment, the watermark image is adjusted to a preset size and then its transparency is adjusted to a preset transparency. The watermark image after all adjustments are completed is synthesized to a preset position in the inspection real-shot image. The preset size, preset transparency, and preset position can be set by a person skilled in the art as needed. In another embodiment, a nearest neighbor interpolation algorithm is used to scale the watermark image to the size of the actual inspection image and binarize it to obtain a binary watermark image of the size with only 0 and 1, which is directly written into the LSB (Least Significant Bit) of the actual inspection image to implicitly embed the watermark image in the actual inspection image, so that the watermark image is hidden in the actual inspection image and it is difficult to distinguish the existence of the watermark image with the naked eye.

[0042] The image representation model is pre-trained to a convergence state and acquires the ability to represent the image semantic features of the input image. The specific selection of the image representation model can be any readily available open source model such as the Resnet series, VGG, Mobile series, etc.

[0043] The watermark detection model is pre-trained to a convergent state and acquires the ability to detect the watermark position information of the live-shot watermark image in the input image. The specific training process can be flexibly implemented by those skilled in the art according to the reasoning process disclosed herein. The watermark detection model can adopt a model suitable for the target detection task in terms of model selection. The YOLOV8 model is recommended. Other models such as U-Net model, Ret i naNet model, CornerNet model, etc. can also be adopted.

[0044] Step S1300: Generate an information verification code corresponding to the inspection point details information, submit the inspection point information and the information verification code to the blockchain platform, and obtain the blockchain evidence certificate returned by the blockchain platform.

[0045] The inspection point detailed information includes inspection point punch-in information and inspection point real-shot information. The inspection point punch-in information includes inspection point location information, punch-in identity information, and punch-in time. The punch-in identity information includes the punch-in person account information and the punch-in person signature image. The inspection point real-shot information includes the inspection real-shot image and on-site shooting information. The on-site shooting information includes the shooting time, shooting location information, and the photographer account information.

[0046] The signature image of the card puncher and the inspection real-shot image are converted into corresponding card punching signature byte stream and inspection real-shot byte stream, and then, a preset hash algorithm is used for the card punching signature byte stream, the inspection real-shot byte stream, the card puncher account information and the on-site shooting information to generate corresponding information verification codes. The hash algorithm is recommended to use sha256, and other hash algorithms such as md5, sha1, sha384, sha512, sha3_224, sha3_256, sha3_384, sha3_512, etc. can also be used. Those skilled in the art can choose one to implement as needed. A connection is established with the blockchain platform by enabling the communication interface provided by the blockchain platform, and then the inspection point information and information verification code are packaged into transaction information and submitted to the blockchain platform to obtain the blockchain evidence certificate returned by it.

[0047] Usually, after the network nodes of the distributed architecture of the blockchain platform confirm that the transaction information is correct, they perform corresponding chain hash calculations on the transaction information and write the obtained hash value and transaction information into a new block of the blockchain. Furthermore, the blockchain platform automatically generates an electronic preservation certificate based on the successfully stored transaction data, and returns it to the server as a blockchain evidence certificate. The blockchain evidence certificate contains information verification code, inspection point information, card puncher account information, card puncher signature image, card punching time, shooting time, shooting point information, photographer account information, and inspection real-shot image containing tamper-proof code.

[0048] It can be known from the typical embodiments of the present application that the technical solution of the present application has many advantages, including but not limited to the following aspects:

[0049] Firstly, by responding to inspection task execution events, this application can obtain in real time the inspection point check-in information and inspection real-time images, wherein the inspection real-time images also include on-site watermark images, thereby ensuring the timeliness and integrity of information processing.

[0050] Secondly, watermark technology and symmetric encryption technology are used to perform on-site information watermark processing and anti-tampering processing on the actual inspection images to enhance the security and integrity of the image information and ensure that the image information is not illegally tampered with during transmission and storage.

[0051] Finally, by submitting the inspection point details and information verification code to the blockchain platform, a blockchain evidence certificate is obtained. By utilizing the immutability and transparency of blockchain technology, strong legal support is provided for all details of the inspection process, and authentic and reliable electronic evidence of the inspection is obtained.

[0052] To sum up, ensuring the orderly and efficient execution of inspection work and converting all inspection information produced into legally reliable electronic evidence provides strong technical support for legal proceedings and arbitration mediation.

[0053] In a further embodiment, step S1100, obtaining the checkpoint check-in information and the checkpoint real-time shooting information generated by the terminal performing the on-site check-in operation and the on-site shooting operation at the corresponding checkpoint, includes the following steps:

[0054] Step S1110, responding to an on-site clock-in event at a patrol point during the execution of a patrol task, obtaining the patrol point clock-in information generated by the terminal when performing an on-site clock-in interactive operation at the patrol point;

[0055] For each inspection point in the inspection route set for the inspection task, a near-field communication device and / or an inspection point feature code for performing punch-in interaction operations may be provided at the inspection point. The inspection point feature code may be a QR code (QR CODE) or a barcode, etc., and the near-field communication device may be an NFC device.

[0056] In one embodiment, the inspector can place the near field communication device of the terminal close to the near field communication device equipped at the inspection point, and then hand-write the inspector's own name on the handwritten signature panel interface displayed on the terminal to implement the on-site clock-in interactive operation at the inspection point. The terminal obtains the unique identification code corresponding to the inspection point pre-stored in the near field communication device equipped at the inspection point through its near field communication device, generates a signature image containing the signature as the clock-in signature image by enabling the pre-packaged image generation code of the handwritten signature panel interface, uses the account information corresponding to the property management system used by the inspector on the terminal as the clock-in account information, and uses the current timestamp as the clock-in time. Then, the terminal forms the clock-in identity information with the clock-in account information and the clock-in signature image, and forms the clock-in information of the inspection point with the clock-in identity information, the clock-in time and the unique identification code of the inspection point, and uploads it to the server to trigger the on-site clock-in event of the inspection point. The unique identification code of the inspection point is used to uniquely identify the inspection point to distinguish it from other inspection points. Specifically, it can be the factory unique identification code of the near-field communication device equipped on site at the inspection point, or it can be flexibly set by technicians in this field, such as ID.

[0057] In another embodiment, the inspector can use the camera of his terminal to take a photo at the inspection point to identify the inspection point feature code of the inspection point, and then hand-write the inspector's own name on the handwritten signature panel interface displayed on the terminal to implement the on-site clock-in interactive operation at the inspection point. The terminal obtains the corresponding pre-packaged unique identification code of the inspection point by identifying the inspection point feature code, generates a signature image containing the signature as the puncher's signature image by enabling the pre-packaged image generation code of the handwritten signature panel interface, uses the corresponding account information of the property management system used by the inspector on the terminal as the puncher's account information, and uses the current timestamp as the punch-in time. Then, the terminal forms the punch-in identity information with the puncher's account information and the puncher's signature image, and forms the punch-in identity information, the punch-in time and the unique identification code of the inspection point as the inspection point punch-in information, which is uploaded to the server to trigger the on-site clock-in event of the inspection point. The unique identification code of the inspection point is used to uniquely identify the inspection point to distinguish it from other inspection points, and can be flexibly set by those skilled in the art, such as ID.

[0058] The server responds to the on-site clocking-in event of the inspection point during the execution of the inspection task, receives the inspection point clocking-in information uploaded by the terminal, queries the inspection point location table according to the unique identification code of the inspection point in the inspection point clocking-in information, determines the inspection point location information associated with the mapping of the unique identification code, appends the inspection point location information to the inspection point clocking-in information, returns the inspection point clocking-in information and the punching success notification to the terminal, and updates the clocking-in status of the inspection point during the execution of the terminal's inspection task to the punching success. The inspection point location table includes the unique identification code of each inspection point and the inspection point location information associated with its mapping. The inspection point location information indicates the location of the corresponding inspection point, which can be located using GPS positioning or a publicly recognized address description text.

[0059] Step S1120, responding to the on-site shooting event of the inspection point during the execution of the inspection task, obtaining the on-site real-time shooting image uploaded by the terminal at the inspection point, and determining the corresponding on-site shooting information, wherein the on-site shooting information includes the shooting time, the shooting point information, and the account information of the shooter;

[0060] Furthermore, the inspector uses the camera of his terminal to take real-life images at the inspection point and then uploads the real-life images to the server, as well as the current shooting location information and the account information corresponding to the property management system used by the inspector on the terminal.

[0061] The server receives the on-site real-shot image, current shooting location information and account information, responds to the on-site shooting event of the inspection point during the execution of the inspection task, uses the current timestamp as the shooting time, the current shooting location information as the shooting point information, and the account information as the account information of the photographer, and the shooting time, shooting point information, and photographer account information constitute the on-site shooting information.

[0062] Step S1130, generate a field shooting watermark image containing the field shooting information, synthesize the field shooting watermark into the field real-shot image according to the preset watermark position, obtain the inspection real-shot image, and form the inspection point real-shot information with the inspection real-shot image and the field shooting information.

[0063] The rendering interface provided by OpenGL is called to generate a live shooting watermark image containing the live shooting information, the live shooting watermark image is synthesized to the preset watermark position in the live real-shot image, the inspection real-shot image is obtained, and the inspection real-shot image is returned to the terminal. The preset watermark position can be set as needed by those skilled in the art, such as the upper left corner, upper right corner, lower right corner, lower left corner of the image, etc., which does not affect the position of the main content in the image.

[0064] In this embodiment, by responding to on-site clocking-in events and on-site shooting events at inspection points during the execution of inspection tasks, corresponding inspection point clocking-in information and inspection real-shot images are constructed, so that the on-site shooting information is placed in the watermark of the inspection real-shot image, so that it can be displayed visually for inspection personnel and task managers to check at any time. Secondly, the clocking-in interactive operation is flexible and convenient, which improves the clocking-in efficiency and information processing efficiency.

[0065] In a further embodiment, step S1200, according to the watermark position information in the inspection real-shot image contained in the inspection point real-shot information, symmetrically encrypting the original image representation corresponding to the inspection real-shot image to obtain the information encapsulation ciphertext, generating a tamper-proof code containing the information encapsulation ciphertext, and synthesizing the tamper-proof code into the inspection real-shot image, includes the following steps:

[0066] Step S1210, determining the watermark position information in the inspection real-shot image contained in the inspection point real-shot information, and the original image representation of the inspection real-shot image;

[0067] A preset watermark detection model is used to perform target detection on the inspection real-shot image. By performing convolution operations on the inspection real-shot image, the image semantic features corresponding to the inspection real-shot image are extracted, including the color, texture, edge, contour, and deep, complex and abstract semantic information of the inspection real-shot image, and a corresponding multi-scale feature map is generated. The on-site shot watermark image in the inspection real-shot image is used as the target detection object to predict multiple candidate frame areas on the feature map. The overlapping candidate frame areas are filtered through non-maximum suppression, and the anchor frame area of ​​the target detection object in the frame is inferred. The position information of the anchor frame area in the inspection real-shot image is obtained as the watermark position information of the on-site shot watermark image in the inspection real-shot image.

[0068] A preset image representation model is used to take the inspection real-shot image as input, extract the color, texture, edge, contour, deep complex and abstract semantic information of the inspection real-shot image, and obtain an image feature vector that vectorizes the semantic information as the original image representation.

[0069] The image representation model is pre-trained to a convergence state and acquires the ability to represent the image semantic features of the input image. The specific selection of the image representation model can be any readily available open source model such as the Resnet series, VGG, Mobi l enet series, etc.

[0070] The watermark detection model is pre-trained to a convergent state and acquires the ability to detect the watermark position information of the live-shot watermark image in the input image. The specific training process can be flexibly implemented by those skilled in the art according to the reasoning process disclosed herein. The watermark detection model can adopt a model suitable for the target detection task in terms of model selection. The YOLOV8 model is recommended. Other models such as U-Net model, Ret i naNet model, CornerNet model, etc. can also be adopted.

[0071] Step S1220: symmetrically encrypt the original image representation according to the watermark position information to obtain information encapsulation ciphertext;

[0072] Furthermore, the encryption function in the AES (Advanced Encryption Standard) symmetric encryption and decryption algorithm is used to use the watermark position information as a symmetric encryption and decryption key to symmetrically encrypt the original image representation, obtain the corresponding ciphertext as the information encapsulation ciphertext, generate a tamper-proof code containing the information encapsulation ciphertext, and the tamper-proof code can be a two-dimensional code. The rendering interface provided by OpenGL is called to generate a watermark image containing the tamper-proof code.

[0073] Step S1230: Generate a watermark image corresponding to the tamper-proof code containing the information encapsulation ciphertext, and synthesize the watermark image into the inspection real-shot image.

[0074] After adjusting the watermark image to a preset size, adjust its transparency to a preset transparency, and synthesize the adjusted watermark image to a preset position in the inspection real-shot image. The preset size, preset transparency, and preset position can be set by those skilled in the art as needed.

[0075] In this embodiment, by constructing the original image representation of the actual inspection image and the corresponding information encapsulation ciphertext, the tamper-proof code containing the information encapsulation ciphertext is synthesized into the actual inspection image, which can ensure the security of image information and prevent information tampering. At the same time, it also provides a reliable basis for information traceability verification, greatly improving the accuracy and reliability of inspection information. Secondly, the tamper-proof code does not need additional storage but can be retrieved and called when needed, which greatly improves information processing efficiency and does not consume additional storage space.

[0076] In a further embodiment, step S1200, according to the watermark position information in the inspection real-shot image contained in the inspection point real-shot information, symmetrically encrypt the original image representation corresponding to the inspection real-shot image to obtain the information encapsulation ciphertext, generate the tamper-proof code containing the information encapsulation ciphertext, and synthesize the tamper-proof code into the inspection real-shot image, comprises the following steps:

[0077] Step S2200: In response to an image authenticity verification event, obtain the inspection real-shot image to be verified corresponding to the event and the information encapsulation ciphertext contained in the anti-tampering code therein;

[0078] The task administrator selects the inspection real-shot images provided by the inspection personnel during the inspection task execution on his terminal, and uses the inspection real-shot images as the inspection real-shot images to be verified. The terminal uploads the inspection real-shot images to be verified to the server, triggering an image authenticity verification event.

[0079] The server responds to the image authenticity verification event, receives the patrol real-shot image to be verified, identifies the tamper-proof code in the patrol real-shot image to be verified, and obtains the information encapsulation ciphertext pre-encapsulated therein.

[0080] Step S2210: symmetrically decrypt the information encapsulation ciphertext according to the watermark position information of the on-site watermark image in the inspection real-shot image to be verified, and when the symmetrical decryption is successful, obtain the original image representation;

[0081] The watermark detection model is used to perform target detection on the real-shot inspection image to be verified. By performing a convolution operation on the real-shot inspection image to be verified, the image semantic features corresponding to the real-shot inspection image to be verified are extracted, including the color, texture, edge, contour, and deep, complex and abstract semantic information of the real-shot inspection image to be verified, and a corresponding multi-scale feature map is generated. The on-site watermark image in the real-shot inspection image to be verified is used as the target detection object to predict multiple candidate frame areas on the feature map. The overlapping candidate frame areas are filtered through non-maximum suppression, and the anchor frame area of ​​the target detection object in the frame is inferred. The position information of the anchor frame area in the real-shot inspection image to be verified is obtained as the watermark position information of the on-site watermark image in the real-shot inspection image to be verified.

[0082] Furthermore, the decryption function in the AES (Advanced Encryption Standard) symmetric encryption and decryption algorithm is used to use the watermark position information as the symmetric encryption and decryption key to symmetrically decrypt the information encapsulation ciphertext. When the symmetric decryption is successful, the plaintext corresponding to the information encapsulation ciphertext is the original image representation. The original image representation is the image representation of the inspection real-shot image before encryption corresponding to the inspection real-shot image to be verified.

[0083] Step S2220, determining the degree of match between the original image representation and the image representation to be verified corresponding to the patrol inspection real-shot image to be verified;

[0084] The image representation model is used to take the real-shot inspection image to be verified as input, extract the color, texture, edge, contour, deep complex and abstract semantic information of the real-shot inspection image to be verified, and obtain an image feature vector that vectorizes the semantic information as the representation of the image to be verified.

[0085] By traversing the original image representation and the image representation to be verified, it is determined whether the original image representation and the image representation to be verified are exactly the same. When they are exactly the same, it is confirmed that the matching degree between the original image representation and the image representation to be verified is 1; otherwise, it is confirmed that the matching degree between the original image representation and the image representation to be verified is 0.

[0086] Step S2230: When the symmetric decryption fails and / or the matching degree meets the preset conditions, it is confirmed that the authenticity verification of the real-shot inspection image to be verified fails, and an image anomaly report describing the reason for the symmetric decryption failure and / or the matching degree is generated accordingly.

[0087] If the matching degree is 0, it is confirmed that the matching degree meets the preset conditions. It can be proved that the visual effect presented by the inspection real-shot image to be verified is different from that of the inspection real-shot image before encryption, that is, the inspection real-shot image to be verified has been tampered with before encryption. Therefore, it is confirmed that the authenticity check of the inspection real-shot image to be verified has failed, and an abnormal description text describing the matching degree is generated accordingly. The abnormal description text includes an explanation of the origin of the matching degree, and the reason why the inspection real-shot image to be verified has been tampered with based on the matching degree. Those skilled in the art can flexibly implement the abnormal description text according to the disclosure herein.

[0088] If the symmetric decryption fails, it can be proved based on the principle of symmetric encryption and decryption that the watermark position information of the on-site watermark image in the real-shot inspection image to be verified has changed relative to the watermark position information before encryption, that is, the real-shot inspection image to be verified has tampered with the on-site watermark image of the corresponding real-shot inspection image before encryption. Therefore, it is confirmed that the authenticity check of the real-shot inspection image to be verified has failed, and an exception description text describing the reason for the failure of the symmetric decryption is generated accordingly. Those skilled in the art can flexibly implement the exception description text based on the disclosure herein.

[0089] When the symmetric decryption fails and / or the matching degree meets a preset condition, the abnormal description text describing the reason for the symmetric decryption failure and / or the abnormal description text of the matching degree constitute an image abnormality report.

[0090] In this embodiment, the task quality inspection model is used to perform quality verification on the actual inspection images. When the quality verification fails, a corresponding image abnormality report is generated to achieve efficient and accurate quality verification with high interpretability, thereby ensuring the recognition of property management system users.

[0091] In a further embodiment, step S1230, synthesizing the watermark image into the inspection real-shot image, comprises the following steps:

[0092] Step S1231, scaling the watermark image to the size of the inspection real-shot image and binarizing it to obtain a corresponding binary watermark image;

[0093] The watermark image is scaled to the size of the inspection real-shot image using the nearest neighbor interpolation algorithm and binarized to obtain a binary watermark image of the size with only 0 and 1.

[0094] Step S1232: write the binary watermark image into the least significant bit of the patrol image to embed the tamper-proof code into the patrol image.

[0095] The binary watermark image is directly written into the LSB (Least Significant Bit) of the inspection real-shot image to implicitly embed the watermark image in the inspection real-shot image, so that the watermark image is hidden in the inspection real-shot image and it is difficult to distinguish the existence of the watermark image with the naked eye.

[0096] In this embodiment, the watermark image of the anti-tampering code is synthesized relatively hidden into the inspection real-shot image, which can minimize the visual interference of the watermark image on the inspection real-shot image while ensuring that the anti-tampering code can be identified.

[0097] In a further embodiment, step S1300, generating an information verification code corresponding to the inspection point detailed information, includes the following steps:

[0098] Step S1310: convert the signature image of the person punching the card in the checkpoint punching information contained in the checkpoint detailed information, and the inspection real shot image containing the tamper-proof code in the inspection point real shot information, into a punching signature byte stream and an inspection real shot byte stream respectively;

[0099] For the signature image of the card-puncher and the real-shot inspection image, in one embodiment, based on the Pillow library, use with open(image_path,'rb')as image_file to open the image, where 'rb' means reading the image in binary mode, and then use image_file.read() to read the content of the image and assign it to a variable. Finally, use the return statement to return the variable, that is, the byte stream of the image.

[0100] In another embodiment, based on the Pillow library, use Image.open(image_path) to open the image, and assign the corresponding returned image object to the variable image. Then, we use image.tobytes() to convert the image into a byte stream and assign it to the variable. Finally, we use the return statement to return the variable, i.e., the byte stream of the image.

[0101] The byte stream refers to a continuous data stream in bytes, which can be used to represent any binary data, including pictures, audio, video, etc., that is, the byte stream can be used to represent the binary data of a picture.

[0102] Step S1320: Use a preset hash algorithm to generate a corresponding information verification code for the punch-in signature byte stream, the inspection real-shot byte stream, the punch-in person account information and the on-site shooting information in the inspection point details information.

[0103] The hash algorithm is also called a hash function. It compresses the message or data into a summary to reduce the amount of data and fix the format of the data. The function shuffles and mixes the data to recreate a fingerprint called a hash value (or hash value). The hash value is usually represented by a short string of random letters and numbers. The hash algorithm is recommended to use sha256, and other hash algorithms such as md5, sha1, sha384, sha512, sha3_224, sha3_256, sha3_384, sha3_512, etc. can also be used. Those skilled in the art can choose one to implement as needed.

[0104] In one embodiment, sha256 is used as the hash algorithm. For messages or data of any length, sha256 generates a 256-bit long hash value. The hash algorithm is used to shuffle and mix the punch-in signature byte stream, the patrol real-shot byte stream, the punch-in account information and the on-site shooting information in the patrol point details information, and recreate a corresponding 256-bit long hash value as an information verification code.

[0105] In this embodiment, the image information in the inspection process is converted into a corresponding byte stream, and a preset hash algorithm is used to generate a corresponding information check code for the image information and other key information in the inspection process. Since the hash algorithm has a one-way characteristic, that is, irreversibility, even a small change in information will cause a significant change in the generated hash value, which makes any modification easily detectable, thereby ensuring the integrity and authenticity of all information in the inspection process.

[0106] In a further embodiment, after step S1100, obtaining the checkpoint check-in information and the checkpoint real-shot information generated by the terminal performing the on-site check-in operation and the on-site shooting operation at the corresponding checkpoint to form the checkpoint detailed information, the following steps are included:

[0107] Step S2100, comparing the account information of the designated person in the pre-designated identity information of the inspection task with the account information of the clock-in person in the clock-in identity information in the inspection point clock-in information to obtain a first comparison result;

[0108] The designated person's account information is compared with the card-punching person's account information for consistency. When the comparison is consistent, the first comparison result is confirmed to be consistent; when not consistent, the first comparison result is confirmed to be inconsistent.

[0109] Step S2110: using a preset signature recognition model to determine the confidence level between the signature image of the designated person in the pre-designated identity information and the signature image of the punching person in the punching identity information;

[0110] The signature recognition model is pre-trained to a convergence state and acquires the ability to determine the confidence of the signatures of the same person between the two input signature images. The signature recognition model can be a dual-tower model, which includes two image branch networks and a classification layer in terms of model structure. The two image branch networks are the same image feature extraction model. The image feature extraction model can use a model used to extract image features. The recommended model is EffectiveNetV2. Other models such as the YOLO series model, Resnet model, Vi T (Vision Transformer) model, etc. can also be used. The classification layer is used for binary classification tasks and can be any vector distance algorithm such as the Euclidean distance algorithm, the cosine similarity algorithm, the inner product, etc.

[0111] Collect signature images obtained by hand-signing multiple times by multiple people in advance, split these signature images into two groups, each group contains the same number of signature images, then randomly extract one signature image from each of the two groups each time, and obtain the corresponding two signature images to form a single first training sample, until all signature images are extracted from the two groups, thereby obtaining multiple first training samples, and then annotate the supervisory label of the first training sample according to whether the two signature images in each first training sample are signed by the same person, for example, if the two signature images in the first training sample are signed by the same person, annotate the first supervisory label of the first training sample as 1; otherwise, annotate the first supervisory label of the first training sample as 0. Collect all first training samples and their first supervisory labels to form the first training set.

[0112] A single first training sample and its first supervision label in the first training set are called, and two signature images in the first training sample are input into two image branch networks in the signature recognition model respectively. The two image branch networks respectively extract image semantic information of the corresponding signature images. It is not difficult to understand that the image semantic information includes but is not limited to the shape, font type, size, writing habits, etc. of the handwriting, and the corresponding handwriting feature vector is obtained. The classification layer in the signature recognition model determines the vector distance between the two image feature vectors as the confidence of predicting the signature of the same person, and calls the preset cross entropy loss function. The technical personnel in this field can flexibly adapt it according to prior knowledge or experimental experience. The method is configured to calculate a cross entropy loss value corresponding to the predicted confidence of the same person's signature based on the supervisory label of the first training sample. When the cross entropy loss value is lower than or equal to a preset threshold, it indicates that the signature recognition model has been trained to a convergence state, thereby terminating the signature recognition model training; when the cross entropy loss value exceeds the preset threshold, it indicates that the signature recognition model has not converged, and then the model is gradient updated according to the cross entropy loss value, and the weight parameters of each link of the model are usually corrected through back propagation to make the model further close to convergence, and then, other first training samples and their first supervisory labels are continuously called to perform iterative training on the signature recognition model until the model is trained to a convergence state.

[0113] The signature image of the designated person and the signature image of the clock-in person are input into the two image branch networks in the signature recognition model trained to convergence respectively, and the image semantic information of the corresponding signature images is extracted by the two image branch networks respectively. It is not difficult to understand that the image semantic information includes but is not limited to the shape, font type, size, writing habits, etc. of the handwriting, and the corresponding handwriting feature vector is obtained. The classification layer in the signature recognition model determines the vector distance between the two image feature vectors as the confidence of the signature of the same person.

[0114] Step S2120: When the first comparison result indicates that there is a difference and / or the confidence of the signature of the same person is lower than a preset threshold, it is confirmed that the identity verification of the punch-in identity information fails, and an identity anomaly report describing the first comparison result and / or the confidence of the signature of the same person is generated accordingly;

[0115] If the first comparison result is inconsistent, that is, there is a difference in the representation, it can be concluded that the inspector who punched in is not the same person as the designated inspector, so it is confirmed that the identity verification of the punch-in identity information fails, and an abnormal description text describing the first comparison result is generated accordingly. Those skilled in the art can flexibly implement the abnormal description text based on the disclosure herein.

[0116] If the confidence of the same person's signature is lower than the preset threshold, it means that the signature of the inspector who punched in and the signature of the designated inspector are not signed by the same person, so the identity verification of the punch-in identity information is confirmed to fail, and an abnormal description text describing the first comparison result is generated accordingly. Those skilled in the art can flexibly implement the abnormal description text and the preset threshold according to the disclosure herein.

[0117] When the first comparison result indicates that there are differences and / or the fan signature confidence is lower than a preset threshold, the abnormal description text describing the first comparison result and / or the abnormal description text of the fan signature confidence constitutes an identity anomaly report.

[0118] Step S2130: append the identity anomaly report to the inspection point details information and send it to the task administrator.

[0119] The identity anomaly report is appended to the inspection point detail information to ensure the integrity and traceability of the information, and then the appended inspection point detail information is sent to the task administrator.

[0120] In this embodiment, identity verification is performed by matching the account information and signature image of the person punching the card with the account information and signature image of the designated person. When the identity verification fails, a corresponding identity anomaly report is generated and appended to the inspection point details and the task administrator is notified, thereby achieving efficient and accurate identity verification with high explainability, thereby ensuring the recognition of property management system users.

[0121] In a further embodiment, after step S1100, obtaining the checkpoint check-in information and the checkpoint real-shot information generated by the terminal performing the on-site check-in operation and the on-site shooting operation at the corresponding checkpoint to form the checkpoint detailed information, the following steps are included:

[0122] Step S3100: Use a preset task quality inspection model to perform target detection on the inspection real-shot images contained in the inspection point real-shot information to obtain the confidence and abnormality type of the images with quality abnormality signs in the inspection real-shot images;

[0123] The task quality inspection model is pre-trained to a convergence state and learns the ability to detect quality abnormality signs and abnormality types in input images. The task quality inspection model can use a model suitable for target detection tasks. The recommended model is the YOLOV8 model. Other models such as the U-Net model, the Ret i naNet model, the CornerNet model, etc. can also be used.

[0124] Before the execution of each inspection task, for each inspection point in the inspection route set in advance for the inspection task, all possible quality abnormalities at the inspection point are considered, and corresponding historical inspection real-shot images containing at least one image showing quality abnormality are collected by taking multiple shots on-site at the inspection point. For example, at an inspection point for verifying the safety and feasibility of a waterproof plywood, waterproof plywood with quality abnormalities is photographed on-site to obtain corresponding historical inspection real-shot images. A single historical inspection real-shot image contains images showing quality abnormalities corresponding to one or more of the following image contents: cracks, hollows or curling of waterproof plywood, water leakage at seams of waterproof plywood, and waterproof plywood material types not meeting national standards.

[0125] Each of the obtained historical inspection images is used as the second training sample. For each second training sample, the label img marking tool is used to frame the specific position, size, and abnormality type of each corresponding quality abnormality sign image in the historical inspection image contained in the second training sample, and the corresponding annotation file is obtained as the second supervision label of the second training sample. All the second training samples and their second supervision labels are collected to form the second training set. The abnormality type represents the quality abnormality condition corresponding to the corresponding quality abnormality sign image, and can be set as needed by technicians in this field.

[0126] A single second training sample in the second training set and its second supervision label are called, and the task quality inspection model uses the second training sample as input to perform target detection on the second training sample. The task quality inspection model extracts the image semantic features corresponding to the second training sample by performing a convolution operation on the second training sample, including the color, texture, edge, contour, deep complex and abstract semantic information of the second training sample, and generates a corresponding multi-scale feature map. The image with quality abnormality signs in the second training sample is used as the target detection object to predict multiple candidate box areas on the feature map, and the overlapping candidate box areas are filtered by non-maximum suppression to predict the predicted anchor box area of ​​the target detection object in the box. The cross entropy loss function is used to The supervisory label of the training sample is used to calculate the loss value corresponding to the cross entropy loss of the predicted anchor box area. When the loss values ​​are all lower than the preset threshold, it indicates that the task quality inspection model has been trained to a convergence state, thereby terminating the training of the task quality inspection model; when the loss value is greater than or equal to the preset threshold, it indicates that the task quality inspection model has not converged, and then the task quality inspection model is gradient updated according to the loss value. Usually, the weight parameters of each link in the task quality inspection model are corrected by back propagation to make the task quality inspection model further close to convergence, and then, other second training samples in the second training set and their second supervisory labels are continued to be called to perform iterative training on the task quality inspection model until the task quality inspection model is trained to a convergence state.

[0127] The task quality inspection model trained to convergence performs convolution operations on the inspection real-shot images to extract the image semantic features corresponding to the inspection real-shot images, including the color, texture, edge, contour, deep complex and abstract semantic information of the inspection real-shot images, etc., to generate corresponding multi-scale feature maps, and predicts multiple candidate frame areas on the feature maps using the quality abnormality signs images in the inspection real-shot images as the target detection objects, and infers the anchor frame area of ​​the target detection object in the frame and its confidence and abnormality type through non-maximum suppression filtering of overlapping candidate frame areas. The abnormality type characterizes the quality abnormality condition corresponding to the corresponding quality abnormality signs images.

[0128] Step S3110: when the confidence level of the image with signs of quality abnormality exceeds a preset threshold, confirm that the image with signs of quality abnormality exists in the inspection real-shot image, and generate a corresponding quality inspection abnormality report according to the abnormality type of the image with signs of quality abnormality;

[0129] At this time, it means that the credibility of the image with quality abnormality signs in the inspection real-shot image meets the standard, so the existence of the image with quality abnormality signs is confirmed, and the abnormality details description text associated with the abnormality type mapping of the image with quality abnormality signs is called from the preset abnormality description set as the quality inspection abnormality report. The abnormality description set contains multiple abnormality types and their mapped abnormality details description texts, and the abnormality details description text describes in detail the quality abnormality status represented by the corresponding abnormality type. Those skilled in the art can flexibly preset the abnormality description set according to the disclosure herein.

[0130] Step S3120: append the quality inspection exception report to the inspection point details information and send it to the task administrator.

[0131] The quality inspection exception report is appended to the inspection point detail information to ensure the integrity and traceability of the information, and then the appended inspection point detail information is sent to the task administrator.

[0132] In this embodiment, the confidence and abnormality type of the image with signs of quality abnormality in the actual inspection image are determined by adopting the task quality inspection model. When the confidence exceeds the preset threshold, it is confirmed that the sign of quality abnormality exists in the actual inspection image, and then a quality inspection abnormality report describing the quality abnormality condition corresponding to the image with signs of quality abnormality is generated and appended to the inspection point details information and the task administrator is notified. This can comprehensively and without omission detect the image with signs of quality abnormality and its abnormality type in the input image, which is conducive to making up for omissions and / or errors in manual inspection.

[0133] See also Figure 3, a patrol evidence storage device provided to meet one of the purposes of the present application is a functional embodiment of the patrol evidence storage method of the present application. On the other hand, the device is a patrol evidence storage device provided to meet one of the purposes of the present application, including an event response module 1100, an image synthesis module 1200 and a blockchain evidence storage module 1300, wherein the event response module 1100 is used to respond to patrol task execution events, obtain patrol point punch-in information and patrol point real-shot information generated by the terminal performing on-site punch-in operations and on-site shooting operations at the corresponding patrol points, and construct The inspection point details information is synthesized; the image synthesis module 1200 is used to symmetrically encrypt the original image representation corresponding to the inspection real-shot image according to the watermark position information in the inspection real-shot image contained in the inspection point real-shot information to obtain the information encapsulation ciphertext, generate an anti-tampering code containing the information encapsulation ciphertext, and synthesize the tamper-proof code into the inspection real-shot image; the blockchain evidence module 1300 is used to generate an information verification code corresponding to the inspection point details information, submit the inspection point information and the information verification code to the blockchain platform, and obtain the blockchain evidence certificate returned by the blockchain platform.

[0134] In a further embodiment, the event response module 1100 includes: a first event response sub-module, which is used to respond to the on-site punch-in event of the inspection point during the execution of the inspection task, and obtain the inspection point punch-in information generated by the terminal uploading the on-site punch-in interactive operation at the inspection point; a second event response sub-module, which is used to respond to the on-site shooting event of the inspection point during the execution of the inspection task, obtain the on-site real-shot image uploaded by the terminal at the inspection point, and determine the corresponding on-site shooting information, wherein the on-site shooting information includes the shooting time, shooting point information, and the account information of the photographer; a first watermark synthesis sub-module, which is used to generate an on-site shooting watermark image containing the on-site shooting information, synthesize the on-site shooting watermark into the on-site real-shot image according to the preset watermark position, obtain the inspection real-shot image, and form the inspection point real-shot information with the inspection real-shot image and the on-site shooting information.

[0135] In a further embodiment, the image synthesis module 1200 includes: an image detection and representation sub-module, used to determine the watermark position information in the inspection real-shot image contained in the inspection point real-shot information, and the original image representation of the inspection real-shot image; a symmetric encryption sub-module, used to symmetrically encrypt the original image representation according to the watermark position information to obtain an information encapsulation ciphertext; a second watermark synthesis sub-module, used to generate a watermark image corresponding to the tamper-proof code containing the information encapsulation ciphertext, and synthesize the watermark image into the inspection real-shot image.

[0136] In a further embodiment, the second watermark synthesis submodule includes: an image acquisition unit, used to scale the watermark image to the size of the actual inspection image and binarize it to obtain a corresponding binary watermark image; an image writing unit, used to write the binary watermark image into the least significant bit of the actual inspection image to embed the anti-tampering code into the actual inspection image.

[0137] In a further embodiment, the blockchain evidence storage module 1300 includes: an image conversion sub-module, which is used to convert the signature image of the person punching in the inspection point punch-in information contained in the inspection point details information, and the inspection real-shot image containing the tamper-proof code in the inspection point real-shot information, into a punch-in signature byte stream and an inspection real-shot byte stream respectively; a hash acquisition sub-module, which is used to use a preset hash algorithm to generate corresponding information verification codes for the punch-in signature byte stream, the inspection real-shot byte stream, the punch-in person account information in the inspection point details information, and the on-site shooting information.

[0138] In a further embodiment, the event response module 1100 includes: a first comparison submodule, which is used to compare the account information of the designated person in the pre-designated identity information of the inspection task with the account information of the clock-in person in the clock-in identity information in the inspection point clock-in information to obtain a first comparison result; a signature comparison submodule, which is used to determine the signature confidence between the signature image of the designated person in the pre-designated identity information and the signature image of the clock-in person in the clock-in identity information using a preset signature recognition model; a first report generation submodule, which is used to confirm that the identity authentication of the clock-in identity information has failed when the first comparison result indicates that there is a difference and / or the signature confidence is lower than a preset threshold, and to generate an identity anomaly report describing the first comparison result and / or the signature confidence; a first anomaly report submodule, which is used to append the identity anomaly report to the inspection point detail information and send it to the task administrator.

[0139] In a further embodiment, the event response module 1100 includes: a target detection submodule, which is used to use a preset task quality inspection model to perform target detection on the inspection real-shot images contained in the inspection point real-shot information, and obtain the confidence and abnormality type of the image with signs of quality abnormalities in the inspection real-shot images; a second report generation submodule, which is used to confirm the existence of the image with signs of quality abnormalities in the inspection real-shot images when the confidence of the image with signs of quality abnormalities exceeds a preset threshold, and generate a corresponding quality inspection abnormality report according to the abnormality type of the image with signs of quality abnormalities; a second abnormality report submodule, which is used to append the quality inspection abnormality report to the inspection point details information and send it to the task administrator.

[0140] In order to solve the above technical problems, the present application also provides a computer device. Figure 4 As shown, a schematic diagram of the internal structure of a computer device. The computer device includes a processor, a computer-readable storage medium, a memory, and a network interface connected via a system bus. Among them, the computer-readable storage medium of the computer device stores an operating system, a database, and computer-readable instructions. The database may store a control information sequence. When the computer-readable instructions are executed by the processor, the processor may implement a patrol inspection and evidence storage method. The processor of the computer device is used to provide computing and control capabilities to support the operation of the entire computer device. The memory of the computer device may store computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor may execute the patrol inspection and evidence storage method of the present application. The network interface of the computer device is used to connect and communicate with a terminal. Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0141] In this embodiment, the processor is used to execute Figure 3 The memory stores the program code and various data required to execute the above modules or submodules. The network interface is used to transmit data between user terminals or servers. The memory in this embodiment stores the program code and data required to execute all modules / submodules in the inspection and evidence storage device of this application, and the server can call the program code and data of the server to execute the functions of all submodules.

[0142] The present application also provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the inspection and evidence storage method of any embodiment of the present application.

[0143] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments of the present application can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a computer-readable storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0144] In summary, this application realizes highly secure and tamper-proof inspection information storage.

[0145] It will be understood by those skilled in the art that the various operations, methods, steps, measures, and schemes in the processes discussed in this application may be alternated, changed, combined, or deleted. Furthermore, other steps, measures, and schemes in the various operations, methods, and processes discussed in this application may also be alternated, changed, rearranged, decomposed, combined, or deleted. Furthermore, the steps, measures, and schemes in the various operations, methods, and processes in the prior art that are open source and in this application may also be alternated, changed, rearranged, decomposed, combined, or deleted.

[0146] The above description is only a partial implementation method of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A patrol inspection evidence storage method, characterized in that: The steps include: In response to the inspection task execution event, the inspection point punch-in information and inspection point real-time shooting information generated by the terminal performing the on-site punch-in operation and on-site shooting operation at the corresponding inspection point are obtained to form the inspection point detailed information; According to the watermark position information in the inspection real-shot image contained in the inspection point real-shot information, the original image representation corresponding to the inspection real-shot image is symmetrically encrypted to obtain the information encapsulation ciphertext, and the tamper-proof code containing the information encapsulation ciphertext is generated, and the tamper-proof code is synthesized into the inspection real-shot image; Generate an information verification code corresponding to the inspection point details, submit the inspection point information and the information verification code to the blockchain platform, and obtain the blockchain evidence certificate returned by the blockchain platform.

2. The inspection and evidence storage method according to claim 1 is characterized in that: Obtaining the checkpoint punch-in information and the checkpoint real-time shooting information generated by the terminal performing the on-site punch-in operation and the on-site shooting operation at the corresponding checkpoint includes the following steps: Respond to the on-site clock-in event of the inspection point during the inspection task execution, and obtain the inspection point clock-in information generated by the terminal uploading the on-site clock-in interactive operation performed at the inspection point; In response to the on-site shooting event of the inspection point during the execution of the inspection task, the on-site real-time shooting image uploaded by the terminal at the inspection point is obtained, and the corresponding on-site shooting information is determined, and the on-site shooting information includes the shooting time, shooting point information, and the account information of the photographer; Generate a live shooting watermark image containing the live shooting information, synthesize the live shooting watermark into the live shooting image according to the preset watermark position, obtain the inspection real shooting image, and form the inspection point real shooting information with the inspection real shooting image and the live shooting information.

3. The inspection and evidence storage method according to claim 1 is characterized in that: According to the watermark position information in the inspection real-shot image contained in the inspection point real-shot information, the original image representation corresponding to the inspection real-shot image is symmetrically encrypted to obtain the information encapsulation ciphertext, and the tamper-proof code containing the information encapsulation ciphertext is generated, and the tamper-proof code is synthesized into the inspection real-shot image, including the following steps: Determine the watermark position information in the inspection real-shot image contained in the inspection point real-shot information, and the original image representation of the inspection real-shot image; Symmetrically encrypt the original image representation according to the watermark position information to obtain information encapsulation ciphertext; A watermark image corresponding to the tamper-proof code containing the information encapsulation ciphertext is generated, and the watermark image is synthesized into the inspection real-shot image.

4. The inspection and evidence storage method according to claim 3 is characterized in that: The watermark image is synthesized into the inspection real-shot image, comprising the following steps: Scaling the watermark image to the size of the inspection real-shot image and binarizing it to obtain a corresponding binary watermark image; The binary watermark image is written into the least significant bit of the inspection real-shot image to embed the tamper-proof code into the inspection real-shot image.

5. The inspection and evidence storage method according to claim 1 is characterized in that: Generating an information verification code corresponding to the inspection point details information includes the following steps: Convert the signature image of the person punching the card in the checkpoint punching information contained in the checkpoint detailed information, and the inspection real shot image containing the tamper-proof code in the checkpoint real shot information, into a punching signature byte stream and an inspection real shot byte stream respectively; A preset hash algorithm is used to generate corresponding information verification codes for the punch-in signature byte stream, the inspection real-shot byte stream, the punch-in person account information and the on-site shooting information in the inspection point details information.

6. The inspection and evidence storage method according to claim 1 is characterized in that: After obtaining the checkpoint punch-in information and the checkpoint real-time shooting information generated by the terminal performing the on-site punch-in operation and the on-site shooting operation at the corresponding checkpoint to form the checkpoint detailed information, the following steps are included: Compare the account information of the designated person in the pre-designated identity information of the inspection task with the account information of the clock-in person in the clock-in identity information in the inspection point clock-in information to obtain a first comparison result; Using a preset signature recognition model to determine the confidence level between the signature image of the designated person in the pre-designated identity information and the signature image of the punching person in the punching identity information; When the first comparison result indicates that there is a difference and / or the confidence of the fan signature is lower than a preset threshold, it is confirmed that the identity verification of the punch-in identity information fails, and an identity anomaly report describing the first comparison result and / or the confidence of the fan signature is generated accordingly; The identity anomaly report is appended to the inspection point details information and sent to the task administrator.

7. The inspection and evidence storage method according to claim 1 is characterized in that: After obtaining the checkpoint punch-in information and the checkpoint real-time shooting information generated by the terminal performing the on-site punch-in operation and the on-site shooting operation at the corresponding checkpoint to form the checkpoint detailed information, the following steps are included: Using a preset task quality inspection model to perform target detection on the inspection real-shot images contained in the inspection point real-shot information, and obtaining the confidence and abnormality type of images with signs of quality abnormality in the inspection real-shot images; When the confidence level of the image with signs of quality abnormality exceeds a preset threshold, it is confirmed that the image with signs of quality abnormality exists in the real inspection image, and a corresponding quality inspection abnormality report is generated according to the abnormality type of the image with signs of quality abnormality; The quality inspection exception report is appended to the inspection point details information and sent to the task administrator.

8. A patrol inspection evidence storage device, characterized in that: include: The event response module is used to respond to the inspection task execution event, obtain the inspection point punch-in information and inspection point real-time shooting information generated by the terminal performing the on-site punch-in operation and on-site shooting operation at the corresponding inspection point, and constitute the inspection point detailed information; An image synthesis module is used to symmetrically encrypt the original image representation corresponding to the inspection real-shot image according to the watermark position information in the inspection real-shot image contained in the inspection point real-shot information to obtain the information encapsulation ciphertext, generate a tamper-proof code containing the information encapsulation ciphertext, and synthesize the tamper-proof code into the inspection real-shot image; The blockchain evidence storage module is used to generate an information verification code corresponding to the inspection point details, submit the inspection point information and the information verification code to the blockchain platform, and obtain the blockchain evidence storage certificate returned by the blockchain platform.

9. A computer device comprising a central processing unit and a memory, characterized in that: The central processing unit is used to call and run the computer program stored in the memory to execute the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: It stores a computer program implemented according to the method described in any one of claims 1 to 7 in the form of computer-readable instructions, and when the computer program is called and executed by a computer, the steps included in the corresponding method are executed.

Images