Urban governance construction image authenticity form automatic generation method and system

Abstract

The invention provides an urban governance construction image authenticity form automatic generation method and system, and belongs to the field of urban governance. According to the technical scheme, the method comprises the steps of collecting construction site image data and site affiliated information, embedding a tamper-proof mark, generating preprocessed image data with an authenticity mark in combination with multi-dimensional authenticity verification and preprocessing, constructing a candidate field set and a sample set by using each recognition model, training various models, and establishing a candidate field set and a sample set; and calling the trained model on newly acquired field image data to automatically generate and associatively store an electronic business form. The method has the beneficial effects that the quantization and credibility of the construction site image are realized by introducing the multi-source tamper-proof watermark at the acquisition end and performing multi-dimensional authenticity scoring; through a candidate field set in linkage with text recognition, object recognition and table structure recognition, form template selection and a field mapping model, an electronic business form conforming to industry specifications is automatically generated, and the manual input workload and the error rate are obviously reduced.

Description

Technical Field

[0001] This invention relates to the field of urban governance, and in particular to a method and system for automatically generating forms for verifying the authenticity of construction images in urban governance. Background Technology

[0002] With the advancement of refined and digitalized urban governance, urban management departments have a continuously growing demand for on-site supervision of construction projects, including safety production, civilized construction, dust and noise pollution, road occupation, nighttime construction, waste transportation, and hazardous and major projects. Construction site image data has become a crucial evidence carrier in urban governance, used for law enforcement, risk warning, progress verification, and closed-loop problem rectification. In current urban governance practices, on-site law enforcement personnel, third-party inspectors, or construction units typically use ordinary mobile phones to take photos or simple watermarked cameras to preserve image data, and then manually input key information from the photos into relevant urban governance forms or monitoring platforms. This model has the following shortcomings: First, ordinary photos and simple text watermarks are insufficient to effectively resist the risk of time, location, or image content being tampered with, and the authenticity of images lacks a reliable quantitative evaluation mechanism, affecting the validity of evidence and the traceability of responsibility. Second, images and urban governance business forms have long been separated, requiring repetitive manual identification and data entry in various scenarios such as law enforcement inspections, hidden danger investigations, dust control, and road occupation approval verification, resulting in low efficiency, high error rates, and problems such as missing fields and inconsistencies between forms and images. Third, existing general OCR or form recognition solutions mostly do not combine the regulatory elements of urban governance with the constraints of on-site authenticity, making it difficult to form an integrated technical link of "authentic and verifiable - structured and usable - closed-loop disposal". Summary of the Invention

[0003] The purpose of this invention is to provide a method and system for automatically generating authenticity forms of urban construction images for urban governance scenarios, in order to improve the efficiency and credibility of urban construction supervision and comprehensive law enforcement.

[0004] This invention is achieved through the following measures: A method for automatically generating a form for verifying the authenticity of urban governance construction images, characterized by the following steps: S1. Collect on-site image data and related information, generate on-site image data with anti-tampering marks and associate it with corresponding on-site related information; S2. Perform multi-dimensional authenticity verification and preprocessing on the on-site image data with anti-tampering marks and on-site ancillary information to generate preprocessed image data with authenticity marks and authenticity verification results. S3. Input the preprocessed image data with authenticity markers into each recognition model to obtain on-site recognition information, generate a candidate field set, and combine it with the authenticity markers to form a sample set; S4. Train the authenticity scoring model, form template selection model, field mapping model and interaction optimization model based on the sample set to obtain the corresponding trained models; S5. For the newly acquired on-site image data, use the trained model to generate an electronic business form and store it in association, based on the candidate field set and authenticity label obtained from steps S1 to S3.

[0005] The invention also has the following specific features: S1 includes the following steps: S11. At the construction site, a watermark camera on a mobile terminal is used to collect construction site image data, and at the same time, the site-related information corresponding to the construction site image data is collected. The site-related information includes at least the project number, construction location, construction time, geographical location information, and the identification of the collection device. S12. Perform anti-tampering processing on the construction site image data based on the aforementioned site-related information, embedding anti-tampering tags that reflect the site-related information into the construction site image data, thereby obtaining site image data with anti-tampering tags and corresponding site-related information. The steps for performing anti-tampering processing on the construction site image data are as follows: (1) Perform hash and encryption operations on the project number, construction time and geographical location information in the site auxiliary information to generate verification information for integrity verification; (2) Visual watermark information and invisible digital watermark information carrying the site-related information and the verification information are embedded in multiple spatial areas of the construction site image data, so that cropping, smearing or recoding operations on the construction site image data can be detected by parsing the watermark information.

[0006] S2 includes the following steps: S21. Taking the tamper-proof marked on-site image data and corresponding on-site ancillary information as input, perform digital signature verification, time consistency comparison, location consistency comparison, image content consistency detection, and sharpness and quality assessment respectively to obtain time consistency features, location consistency features, digital signature verification features, image content consistency features, and sharpness and quality features. Based on the time consistency features, location consistency features, digital signature verification features, image content consistency features, and sharpness and quality features, perform multi-dimensional authenticity verification on the tamper-proof marked on-site image data to obtain authenticity verification results indicating the authenticity level of each on-site image data. Specifically: (1) Obtain the device time when collecting the construction site image data, the server time when uploading the construction site image data, and the authoritative time center time, and compare the difference between the above times to obtain the time consistency characteristics. (2) Obtain the spatial distance between the acquisition location and the upload location to obtain location consistency features; (3) Analyze the anti-tampering mark in the on-site image data with anti-tampering mark, and perform digital signature verification on the integrity verification information corresponding to the anti-tampering mark to obtain digital signature verification features; (4) Perform watermark integrity detection and content tampering detection on the on-site image data with anti-tampering marks to obtain image content consistency features; (5) Obtain the sharpness and quality characteristics based on the resolution, contrast and gradient changes of the on-site image data with anti-tampering markings; (6) Input the pre-trained authenticity scoring model with time consistency features, location consistency features, digital signature verification features, image content consistency features and clarity and quality features to obtain an authenticity score that characterizes the comprehensive authenticity of construction site image data. Then, classify the authenticity score according to a preset threshold to obtain the authenticity verification result. The authenticity verification result includes at least three authenticity levels: credible level, doubtful level and suspicious level. S22. Determine the on-site image data that has passed the authenticity verification according to the authenticity verification result, perform image preprocessing on the on-site image data that has passed the authenticity verification to obtain preprocessed image data, and associate the authenticity verification result corresponding to the preprocessed image data as an authenticity marker to obtain preprocessed image data with authenticity marker. The recognition models include a text recognition model, an object recognition model, and a table structure recognition model. Step S3 includes the following steps: S31. Input the preprocessed image data with authenticity markers into the text recognition model, object recognition model and table structure recognition model to obtain on-site recognition information including text content, object category, spatial location information and table structure information; The text recognition model is a deep learning-based optical character recognition model. Specifically, it first locates the text region through a text detection sub-network, then reads the text content through a sequence recognition network, and outputs the text content string, the position of the text region in the image, and the recognition confidence for each recognized text segment. The object recognition model is a deep learning-based target detection model. Specifically, it identifies common objects in construction sites, such as steel bars, formwork, and safety protection facilities. The model outputs the object category, the rectangular position of the object in the image, and the detection confidence for each object. The table structure recognition model is used to identify the row boundaries, column boundaries, and cell regions of a table from preprocessed image data, and outputs the cell position, cell text content, and cell text recognition confidence score of each cell. S32. Based on the on-site identification information and the on-site auxiliary information, aggregate the text fragments, table cells, object labels and auxiliary information belonging to the same on-site image data to generate a candidate field set that corresponds one-to-one with each on-site image data. The candidate field set is used to represent multiple candidate field entries in each on-site image data that can be used to fill in the business form. Each candidate field entry in the candidate field set includes at least field name information, field value information, spatial location information of the field in the preprocessed image data, and field identification confidence level. The field name information is generated based on text content and on-site ancillary information, the field value information is generated based on text content and table structure information, and the field identification confidence level is used to indicate the reliability of the field value information. S33. Combine each candidate field set with its corresponding realism label to form a sample set for model training.

[0007] The post-trained model in step S4 includes a post-trained authenticity scoring model, a post-trained form template selection model, a post-trained field mapping model, and a post-trained interaction optimization model. The interaction optimization model is a reinforcement learning-based policy model, and its form filling status includes a set of confirmed fields, a set of fields to be confirmed, the number of times the user has modified each field, the user's operation sequence, the current business form type, the current authenticity marker and authenticity score, and summary information of the most recent steps in the current user operation sequence. Actions used for reinforcement learning include selecting the next field to present among multiple fields to be confirmed, choosing whether to autofill a field, and choosing whether to issue a prompt for abnormal fields. Reward signals used for reinforcement learning include at least one or more of the following: total form filling time, number of user operation steps, and one-time approval of the business form. Through reinforcement learning training on historical filling logs, the interaction optimization model outputs an optimized field presentation order and autofill strategy given the form filling status, candidate field set, and authenticity marker.

[0008] S5 includes the following steps: S51. For the newly acquired on-site image data, execute steps S1 and S2 to obtain preprocessed image data with authenticity markers corresponding to the newly acquired on-site image data. S52. Input the preprocessed image data with authenticity markers into the text recognition model, object recognition model and table structure recognition model to generate a set of candidate fields corresponding to the newly acquired on-site image data. The set of candidate fields, together with the authenticity score, serves as the features of the new sample. S53. Call the trained authenticity scoring model to score the authenticity of the newly acquired on-site image data, and use the obtained authenticity score as the authenticity mark of the newly acquired on-site image data. S54. Input the candidate field set into the trained form template selection model to determine the target business form template corresponding to the newly collected on-site image data; S55. Input the candidate field set into the trained field mapping model to obtain the initial fill values ​​of each field in the target business form template; S56. During the process of the user confirming and modifying the initial fill value, the current form filling status, the candidate field set, and the authenticity marker are input into the trained interaction optimization model to obtain the field presentation order and automatic filling strategy for the target business form template. The target business form template is dynamically adjusted to obtain the finally confirmed electronic business form. S57. The final confirmed electronic business form is associated with the corresponding newly collected on-site image data and authenticity mark for storage or output, and can be synchronized to the engineering management system or other business systems to realize the automatic conversion of construction site image data into standardized business forms.

[0009] An automatic generation system for urban governance construction image authenticity forms, characterized in that it includes: The data acquisition module includes a watermark camera application installed on a mobile terminal and a server connected to the mobile terminal. The watermark camera application is used to acquire construction time and geographical location information from time synchronization service and location service while acquiring construction site image data, and sends the construction site image data, site auxiliary information and intermediately generated anti-tampering marks to the server. The server is used to perform anti-tampering processing on the construction site image data and store the site image data with anti-tampering marks. The authenticity verification module takes tamper-proof marked on-site image data and corresponding on-site ancillary information as input, and performs digital signature verification, time consistency comparison, location consistency comparison, image content consistency detection, and sharpness and quality assessment to obtain time consistency features, location consistency features, digital signature verification features, image content consistency features, and sharpness and quality features. Based on the time consistency features, location consistency features, digital signature verification features, image content consistency features, and sharpness and quality features, the module performs multi-dimensional authenticity verification on the tamper-proof marked on-site image data to obtain authenticity verification results that indicate the authenticity level of the on-site image data. The module also performs image preprocessing on the on-site image data that passes the authenticity verification to generate preprocessed image data with authenticity marks. The recognition and candidate field generation module is used to input preprocessed image data with authenticity markers into text recognition models, object recognition models, and table structure recognition models to generate on-site recognition information, and generate a set of candidate fields based on the on-site recognition information and on-site ancillary information. The model training module is used to construct a sample set based on the candidate field set and the authenticity label, and to train the authenticity scoring model, form template selection model, field mapping model and interaction optimization model to obtain the trained model; The business form generation module is used to call the trained model to process the newly acquired on-site image data, automatically generate electronic business forms corresponding to the newly acquired on-site image data, and associate and store or output the electronic business forms with the newly acquired on-site image data and authenticity markers.

[0010] The beneficial effects of this invention are as follows: 1. Enhance the credibility of evidence collection in urban governance: By embedding tamper-proof tags into construction site image data and combining it with multi-dimensional features such as time consistency, location consistency, digital signature verification, image content consistency, and clarity for unified modeling, authenticity scoring and grading results are generated. This enables image evidence related to urban governance to have quantifiable and verifiable authenticity, enhancing the reliability of law enforcement evidence collection and accountability.

[0011] 2. Improve the efficiency of urban construction supervision: Candidate field sets are generated based on text recognition, object recognition, and table structure recognition. The automatic conversion of construction images into urban governance business forms is achieved through form template selection model and field mapping model, reducing manual data entry and repetitive verification work, and improving the efficiency of handling scenarios such as safety inspection, civilized construction, and dust and noise inspection.

[0012] 3. Enhance the quality and closed-loop capability of regulatory data: The authenticity marker and candidate field set work together to construct the sample set and train the model, which can continuously optimize the accuracy of form generation in multiple scenarios of urban governance. Combined with the interactive optimization model, it reduces the number of operation steps and errors in the process of on-site personnel confirmation and modification, improves the integrity and consistency of the form, and facilitates the formation of a regulatory closed loop of "discovery-recording-rectification-review".

[0013] 4. Promote standardized data access to urban governance platforms: The automatically generated electronic business forms are associated with and stored or output with corresponding construction site image data and authenticity markers, which facilitates connection with urban governance platforms, housing and construction supervision systems, and comprehensive law enforcement systems, thereby achieving cross-departmental sharing of structured evidence and regulatory data. Attached Figure Description

[0014] Figure 1 This is a flowchart of an embodiment of the present invention. Detailed Implementation

[0015] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0016] To clearly illustrate the technical features of this solution, the following detailed implementation method will be used to explain the solution.

[0017] Example 1 See Figure 1 A method for automatically generating a form for verifying the authenticity of construction images in urban governance includes the following steps: S1. Collect on-site image data and related information, generate on-site image data with anti-tampering marks and associate it with corresponding on-site related information; S2. Perform multi-dimensional authenticity verification and preprocessing on the on-site image data with anti-tampering marks and on-site ancillary information to generate preprocessed image data with authenticity marks and authenticity verification results. S3. Input the preprocessed image data with authenticity markers into each recognition model to obtain on-site recognition information, generate a candidate field set, and combine it with the authenticity markers to form a sample set; S4. Train the authenticity scoring model, form template selection model, field mapping model and interaction optimization model based on the sample set to obtain the corresponding trained models; S5. For the newly acquired on-site image data, use the trained model to generate an electronic business form and store it in association, based on the candidate field set and authenticity label obtained from steps S1 to S3.

[0018] Example 2 See Figure 1 A method for automatically generating a form for verifying the authenticity of construction images in urban governance includes the following steps: S1. Collect on-site image data and related information, generate on-site image data with anti-tampering marks and associate it with corresponding on-site related information, specifically including the following steps; S11. At the construction site, a watermark camera on a mobile terminal is used to collect image data of the construction site, and at the same time, on-site ancillary information corresponding to the construction site image data is collected. The on-site ancillary information includes at least the project number. Construction site Construction time Geographic location information and data acquisition device identification Then the overall on-site ancillary information can be represented as: ; S12. Perform anti-tampering processing on the construction site image data based on the aforementioned site-related information, embedding anti-tampering tags that reflect the site-related information into the construction site image data, thereby obtaining site image data with anti-tampering tags and corresponding site-related information. The steps for performing anti-tampering processing on the construction site image data are as follows: (1) Perform hash and encryption operations on the project number, construction time, and geographical location information in the site auxiliary information to generate verification information for integrity verification. Specifically, if the original construction site image data is a grayscale image, it is denoted as a two-dimensional matrix. ,in Here are the pixel coordinates, and the pixel values ​​are integers from 0 to 255. Define the hash function as follows: The encryption function is ,in Generate an anti-counterfeiting code for integrity verification using the encryption key. The process is as follows: 1) Combine the raw image data with on-site ancillary information:

[0019] 2) Perform a hash operation on the combined result:

[0020] 3) Perform encryption operations on the hash value to obtain the anti-counterfeiting code:

[0021] in, This is the combined input. A hash value of fixed length. This is the encrypted verification information, used for digital signature verification in subsequent steps.

[0022] (2) Visual watermark information and invisible digital watermark information carrying the site-related information and the verification information are embedded in multiple spatial areas of the construction site image data, so that cropping, smearing or recoding operations on the construction site image data can be detected by parsing the watermark information. Specifically: For invisible digital watermarks, this embodiment uses the least significant bit embedding method. Let the watermark bit sequence be denoted as... ,in And select a pixel position for each watermark bit. For 8-bit pixel values Embedded pixel values for:

[0023] in, The least significant bit of the original pixel. This is the construction site image data after watermarking. Watermark bit sequence. From on-site supplementary information and anti-counterfeiting codes The encoding process allows the watermark extracted from the image to simultaneously recover both on-site ancillary information and integrity verification information. This embodiment achieves local watermarking by embedding watermark information carrying on-site information and anti-counterfeiting codes into multiple spatial regions of the same on-site image data. This makes local cropping, overlaying, or recoding of the image appear as missing or inconsistent local watermarks during subsequent watermark analysis, thus enabling detection.

[0024] S2. Perform multi-dimensional authenticity verification and preprocessing on the tamper-proof marked on-site image data and on-site ancillary information to generate preprocessed image data with authenticity marks and authenticity verification results. The specific steps include the following: S21. Taking the tamper-proof marked on-site image data and corresponding on-site ancillary information as input, perform digital signature verification, time consistency comparison, location consistency comparison, image content consistency detection, and sharpness and quality assessment respectively to obtain time consistency features, location consistency features, digital signature verification features, image content consistency features, and sharpness and quality features. Based on the time consistency features, location consistency features, digital signature verification features, image content consistency features, and sharpness and quality features, perform multi-dimensional authenticity verification on the tamper-proof marked on-site image data to obtain authenticity verification results indicating the authenticity level of each on-site image data. Specifically: (1) Obtain the device time when collecting the construction site image data, the server time when uploading the construction site image data, and the time of the authoritative time service center, and compare the differences between the above times to obtain time consistency characteristics. Specifically, when collecting construction site image data, the watermark camera will record the device time. At the same time, the time of the time synchronization center is obtained through the time synchronization service. When an image is uploaded to the server, the server records the upload time. To quantify time consistency, the time difference is defined as:

[0025] Set the maximum allowable time deviation threshold Then the time consistency feature Defined as:

[0026] in, This represents the actual maximum time deviation. The value ranges from 0 to 1, with a larger value indicating more consistent time.

[0027] (2) Obtain the spatial distance between the collection location and the upload location to obtain location consistency features. Specifically, the collection location is obtained through the location service during collection. Record upload location during upload Location is expressed in latitude and longitude. and The distance between two locations can be calculated using spherical distance (e.g., the Haversine formula):

[0028] in, For the Earth's radius, This represents the spherical distance between the data acquisition location and the upload location. A maximum distance threshold is set. Location consistency feature Defined as:

[0029] The value ranges from 0 to 1, with a larger value indicating more consistent positions.

[0030] (3) Parse the anti-tampering mark in the on-site image data with the anti-tampering mark, and perform digital signature verification on the integrity verification information corresponding to the anti-tampering mark to obtain the digital signature verification feature. Specifically, parse the on-site image data with the anti-tampering mark, extract the on-site auxiliary information and anti-counterfeiting code from the watermark, and obtain the extraction result. and Subsequently, the current image and extracted on-site ancillary information were analyzed. Perform the hash and encryption operations again to obtain .like This indicates that the image and accompanying information have not been tampered with during transmission, thus fulfilling the digital signature verification feature. :

[0031] in, This indicates that the digital signature verification has passed. This indicates that the digital signature verification failed or the extraction was abnormal.

[0032] (4) Perform watermark integrity detection and content tampering detection on the on-site image data with anti-tampering marks to obtain image content consistency features. Specifically, in this embodiment, a trained image tampering detection network is used to analyze the on-site image data with anti-tampering marks. The probability that the image has been artificially modified is output by the tampering detection network. Then the image content consistency feature Defined as:

[0033] in, The value range is from 0 to 1. The larger the value, the closer the image content is to its unaltered state; The image tampering detection network is a pre-trained model. The key point of this invention is to unify the modeling of the network output with temporal consistency features, positional consistency features, digital signature verification features, and sharpness and quality features for authenticity scoring and subsequent processing.

[0034] (5) Sharpness and quality characteristics are obtained based on the resolution, contrast, and gradient changes of the tamper-proof image data. Specifically, to reflect image resolution and sharpness, this embodiment adopts a gradient-based sharpness evaluation method. The average gradient magnitude is calculated for the image before preprocessing. and with preset threshold Normalization is performed to obtain sharpness features. :

[0035] in, It can be calculated by the Sobel operator or other gradient operators. The value ranges from 0 to 1, with a larger value indicating higher image clarity.

[0036] (6) Input the time consistency feature, location consistency feature, digital signature verification feature, image content consistency feature, and sharpness and quality feature into the pre-trained authenticity scoring model (i.e., the trained authenticity scoring model, the training process is described in step S4), to obtain an authenticity score that characterizes the comprehensive authenticity of the construction site image data, and classify the authenticity score according to a preset threshold to obtain the authenticity verification result. The authenticity verification result includes at least three authenticity levels: credible level, doubtful level, and questionable level. Specifically: In this embodiment, the authenticity scoring model adopts a logistic regression model; The above five characteristics Combined into eigenvectors, the linear combination result is defined. for:

[0037] in, For feature weight parameters, This is the bias term. The authenticity score is obtained through a logistic function. :

[0038] in, The value ranges from 0 to 1, and the larger the value, the higher the overall authenticity of the construction site image data.

[0039] Two thresholds are set based on the authenticity score. and ,in The authenticity verification results are divided into: when At that time, the authenticity verification result is rated as trustworthy. when At that time, the authenticity verification result was classified as questionable. when At that time, the authenticity verification result was classified as suspicious.

[0040] S22. Based on the authenticity verification results, determine the on-site image data that has passed the authenticity verification. Perform image preprocessing on the on-site image data that has passed the authenticity verification to obtain preprocessed image data. Associate the authenticity verification results corresponding to the preprocessed image data as authenticity markers to obtain preprocessed image data with authenticity markers. Specifically, in this embodiment, for on-site image data with a credibility verification result, enter the image preprocessing process, generate preprocessed image data, and assign the corresponding authenticity score. The authenticity level is used as an authenticity marker and associated with the preprocessed image data to form preprocessed image data with authenticity markers. Preprocessing operations include, but are not limited to, noise suppression, binarization, tilt correction, and contrast enhancement, in order to improve the accuracy of subsequent text recognition, object recognition, and table structure recognition.

[0041] S3. Input the preprocessed image data with authenticity markers into each recognition model to obtain on-site recognition information, generate a candidate field set, and combine it with the authenticity markers to form a sample set. The specific steps include the following: S31. Input the preprocessed image data with authenticity markers into the text recognition model, object recognition model and table structure recognition model to obtain on-site recognition information including text content, object category, spatial location information and table structure information; The text recognition model is a deep learning-based optical character recognition model. Specifically, it first locates the text region through a text detection sub-network, then reads the text content through a sequence recognition network, and outputs the text content string, the position of the text region in the image, and the recognition confidence for each recognized text segment. The object recognition model is a deep learning-based target detection model. Specifically, it identifies common objects in construction sites, such as steel bars, formwork, and safety protection facilities. The model outputs the object category, the rectangular position of the object in the image, and the detection confidence for each object. The table structure recognition model is used to identify the row boundaries, column boundaries, and cell regions of a table from preprocessed image data, and outputs the cell position, cell text content, and cell text recognition confidence score of each cell. S32. Based on the on-site identification information and the on-site auxiliary information, aggregate the text fragments, table cells, object labels and auxiliary information belonging to the same on-site image data to generate a candidate field set that corresponds one-to-one with each on-site image data. The candidate field set is used to represent multiple candidate field entries in each on-site image data that can be used to fill in the business form. Each candidate field entry in the candidate field set includes at least field name information. Field value information Spatial location information of the field in preprocessed image data and field identification confidence The field name information is generated based on text content and on-site ancillary information, and the field value information is generated based on text content and table structure information. The field identification confidence score is used to indicate the reliability of the field value information. Each candidate field entry is denoted as a quadruple. Then, the candidate field set consists of all candidate field entries corresponding to the same construction site image data:

[0042] in, The number of entries for the candidate field. Candidate field set. Together with the authenticity marker of this construction site image data, it constitutes a record in the sample set for subsequent model training; S33. Combine each candidate field set with its corresponding realism label to form a sample set for model training; S4. Train the authenticity scoring model, form template selection model, field mapping model, and interaction optimization model based on the sample set to obtain the corresponding trained models: Based on the sample set, a preset authenticity scoring model, form template selection model, field mapping model, and interaction optimization model are trained to obtain a post-trained authenticity scoring model for evaluating the authenticity of on-site image data, a post-trained form template selection model for determining target business form templates from a preset set of business form templates, a post-trained field mapping model for mapping candidate field entries to target business form template fields, and a post-trained interaction optimization model for determining the field presentation order and auto-fill strategy during user input. Specifically, the trained models include the post-trained authenticity scoring model, the post-trained form template selection model, the post-trained field mapping model, and the post-trained interaction optimization model. The authenticity scoring model undergoes offline training before going live (the step numbers in this embodiment may not necessarily be in chronological order). During the training phase, a true authenticity label is assigned to each sample. ,in: This indicates that the construction site image data is authentic and reliable after manual review or confirmation by an external system. This indicates that the construction site image data is obviously forged or seriously abnormal.

[0043] For the For each sample, its feature vector is denoted as... The authenticity score is recorded as The loss function uses the cross-entropy form:

[0044] in, For the sample size, For the first The true label of each sample is determined by minimizing the loss function. Learn parameters and .

[0045] The association between the candidate field set and the corresponding historical business form templates is used as supervision information. Features such as keywords, object categories, construction stage information, and inspection type information are extracted from the candidate field set. A classification model is trained based on a supervised learning algorithm, so that when the candidate field set is input, the selection probability of each business form template in the preset business form template set is output, thus obtaining the trained form template selection model. Specifically, the features such as keywords, object categories, construction stage information, and inspection type information are first combined into a feature vector. Set a set of preset business form templates ,in This represents the number of form templates. For each business form template... Define parameter vector Calculate its score:

[0046] in, This represents the dot product of the feature vector and the parameter vector. The selection probability is then obtained using the softmax function.

[0047] in, To select a business form template The probability of [the outcome]. During training, the supervision label is "which business form template a certain set of candidate fields ultimately corresponds to" from historical data. The parameter vectors are learned by minimizing the cross-entropy loss. When making predictions, the business form template with the highest probability is selected as the target business form template.

[0048] Using historical matching results between candidate field entries and fields in the target business form template as supervisory information, matching features are constructed using field name information, the spatial location of fields in preprocessed image data, and field recognition confidence. A matching model is trained based on a supervised learning algorithm, enabling the output of the matching relationship between each candidate field entry and the fields in the target business form template when the candidate field set is input, thus obtaining the trained field mapping model. Specifically, this model represents any candidate field entry. Define a matching relationship between a field in the target business form template and a specific field in the target business form template. Let the set of fields in the target business form template be... ,in Indicates the first The name of each field. For candidate field names and template field names Define text similarity and positional similarity : Text similarity It can be calculated based on edit distance. Let the edit distance function be... The string length function is ,but:

[0049] When the two names are exactly the same The larger the edit distance, the more The smaller.

[0050] Location similarity This reflects the proximity of the candidate field to the expected position in the template. The candidate field position vector is denoted as... The expected position vector of the template field is denoted as It can be defined as:

[0051] in, For Euclidean distance, This is the threshold for the maximum positional deviation.

[0052] The overall similarity is defined by combining textual similarity and positional similarity. :

[0053] in, This is a weighting coefficient, with a value between 0 and 1, used to control the influence of text information and location information on the matching results. For a given candidate field entry... Select overall similarity The template field that is the largest and greater than the preset threshold This serves as the final mapping result. During the training phase, historical manually labeled matching results can be used as supervisory information to adjust the mapping. The parameters used in similarity calculations are used to maximize the overall matching accuracy of the field mapping model.

[0054] The interaction optimization model is a policy model based on reinforcement learning. When training the interaction optimization model, the form filling process is abstracted as a Markov decision process: (1) State definition: At time step The form completion status is recorded as This includes: the set of confirmed fields; the set of fields to be confirmed; statistics on the number of times the user has modified each field; the current business form type (i.e., the target business form template identifier); the current authenticity marker and authenticity score; and summary information of the most recent steps in the current user operation sequence.

[0055] (2) Definition of action: At time step The set of actions that the interaction optimization model can take is denoted as Each action This can include: selecting the next field to be presented in the pending confirmation field; choosing whether to provide an autofill value for a certain field; choosing whether to display a prompt message for suspected abnormal fields; and choosing to skip low-priority fields in some scenarios.

[0056] (3) Reward function: At time step The reward obtained is recorded as This is used to measure the impact of the current action on the overall experience and quality. In this embodiment, it is defined as: ,in: This indicates the total number of steps or clicks the user has accumulated up to the current time step; This indicates the cumulative number of field modifications made by the user up to the current time step; This indicates a one-time approval flag. It is set to 1 when the current information is completed and approved by the business system in one go, and 0 otherwise. These are non-negative weighted parameters, controlling the impact of the number of operation steps, the number of modifications, and the one-time approval result on the reward. Through proper settings... A balance can be struck between "reducing the number of operations", "lowering the modification rate" and "increasing the first-time pass rate".

[0057] (4) Learning Updates: Interaction optimization model adopts The learning algorithm is trained. Let... function Indicates the state Take action below The value of is the state-action value function in reinforcement learning, with a learning rate of . The discount factor is The updated formula is:

[0058] in, Before the update value; For the updated value; In order to perform the action The new state that is reached later; The learning rate is between 0 and 1. This is the discount factor, with a value between 0 and 1.

[0059] By iteratively training on historical form entry logs, the interaction optimization model can output an optimized field presentation order and autofill strategy given the form entry status, candidate field set, and authenticity marker. This reduces the burden of filling out forms for on-site personnel while ensuring data quality.

[0060] S5. For the newly acquired on-site image data, using the trained model, based on the candidate field set and authenticity markers obtained from steps S1 to S3, generate and store electronic business forms, specifically including the following steps: S51. For the newly acquired on-site image data, execute steps S1 and S2 to obtain preprocessed image data with authenticity markers corresponding to the newly acquired on-site image data. S52. Input the preprocessed image data with authenticity markers into a text recognition model, an object recognition model, and a table structure recognition model to generate a candidate field set corresponding to the newly acquired on-site image data. The candidate field set is correlated with the authenticity score. Together, they serve as features of the new sample; S53. Call the trained authenticity scoring model (i.e. the authenticity scoring model used in S2) to score the authenticity of the newly acquired on-site image data, and use the obtained authenticity score as the authenticity mark of the newly acquired on-site image data. S54. Input the candidate field set into the trained form template selection model to determine the target business form template corresponding to the newly acquired on-site image data; input the candidate field set into the trained field mapping model to obtain the initial fill value of each field in the target business form template; S55. During the process of the user confirming and modifying the initial filled value, the current form filling status, the candidate field set, and the authenticity marker are input into the trained interaction optimization model to obtain the field presentation order and automatic filling strategy for the target business form template. The target business form template is dynamically adjusted to obtain the finally confirmed electronic business form. S56. The final confirmed electronic business form is associated with the corresponding newly collected on-site image data and authenticity mark for storage or output, and can be synchronized to the engineering management system or other business systems to realize the automatic conversion of construction site image data into standardized business forms.

[0061] Example 3 An automatic generation system for urban governance construction image authenticity forms includes: The data acquisition module includes a watermark camera application installed on a mobile terminal and a server connected to the mobile terminal. The watermark camera application is used to acquire construction time and geographical location information from time synchronization service and location service while acquiring construction site image data, and sends the construction site image data, site auxiliary information and intermediately generated anti-tampering marks to the server. The server is used to perform anti-tampering processing on the construction site image data and store the site image data with anti-tampering marks. The authenticity verification module takes tamper-proof marked on-site image data and corresponding on-site ancillary information as input, and performs digital signature verification, time consistency comparison, location consistency comparison, image content consistency detection, and sharpness and quality assessment to obtain time consistency features, location consistency features, digital signature verification features, image content consistency features, and sharpness and quality features. Based on the time consistency features, location consistency features, digital signature verification features, image content consistency features, and sharpness and quality features, the module performs multi-dimensional authenticity verification on the tamper-proof marked on-site image data to obtain authenticity verification results that indicate the authenticity level of the on-site image data. The module also performs image preprocessing on the on-site image data that passes the authenticity verification to generate preprocessed image data with authenticity marks. The recognition and candidate field generation module is used to input preprocessed image data with authenticity markers into text recognition models, object recognition models, and table structure recognition models to generate on-site recognition information, and generate a set of candidate fields based on the on-site recognition information and on-site ancillary information. The model training module is used to construct a sample set based on the candidate field set and the authenticity label, and to train the authenticity scoring model, form template selection model, field mapping model and interaction optimization model to obtain the trained model; The business form generation module is used to call the trained model to process the newly acquired on-site image data, automatically generate electronic business forms corresponding to the newly acquired on-site image data, and associate and store or output the electronic business forms with the newly acquired on-site image data and authenticity markers.

[0062] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features of the invention herein.

Claims

1. A method for automatically generating a form for verifying the authenticity of urban governance construction images, characterized in that, Includes the following steps: S1. Collect on-site image data and related information, generate on-site image data with anti-tampering marks and associate it with corresponding on-site related information; S2. Perform multi-dimensional authenticity verification and preprocessing on the on-site image data with anti-tampering marks and on-site ancillary information to generate preprocessed image data with authenticity marks and authenticity verification results. S3. Input the preprocessed image data with authenticity markers into each recognition model to obtain on-site recognition information, generate a candidate field set, and combine it with the authenticity markers to form a sample set; S4. Train the authenticity scoring model, form template selection model, field mapping model and interaction optimization model based on the sample set to obtain the corresponding trained models; S5. For the newly acquired on-site image data, use the trained model to generate an electronic business form and store it in association, based on the candidate field set and authenticity label obtained from steps S1 to S3.

2. The method for automatically generating a form for verifying the authenticity of urban governance construction images according to claim 1, characterized in that, S1 includes the following steps: S11. At the construction site, a watermark camera on a mobile terminal is used to collect construction site image data, and at the same time, the site-related information corresponding to the construction site image data is collected. The site-related information includes at least the project number, construction location, construction time, geographical location information, and the identification of the collection device. S12. Perform anti-tampering processing on the construction site image data based on the aforementioned site-related information, embedding anti-tampering tags that reflect the site-related information into the construction site image data, thereby obtaining site image data with anti-tampering tags and corresponding site-related information. The steps for performing anti-tampering processing on the construction site image data are as follows: The project number, construction time, and geographical location information in the site's supplementary information are hashed and encrypted to generate verification information for integrity verification. Visual and invisible digital watermark information carrying the site-related information and the verification information are embedded in multiple spatial areas of the construction site image data, so that cropping, smearing or recoding operations on the construction site image data can be detected by parsing the watermark information.

3. The method for automatically generating a form for verifying the authenticity of urban governance construction images according to claim 2, characterized in that, S2 includes the following steps: S21. Taking the tamper-proof marked on-site image data and corresponding on-site ancillary information as input, perform digital signature verification, time consistency comparison, position consistency comparison, image content consistency detection, and sharpness and quality assessment respectively to obtain time consistency features, position consistency features, digital signature verification features, image content consistency features, and sharpness and quality features. Based on the time consistency features, position consistency features, digital signature verification features, image content consistency features, and sharpness and quality features, perform multi-dimensional authenticity verification on the tamper-proof marked on-site image data to obtain authenticity verification results used to indicate the authenticity level of each on-site image data. S22. Based on the authenticity verification results, determine the on-site image data that has passed the authenticity verification, perform image preprocessing on the on-site image data that has passed the authenticity verification to obtain preprocessed image data, and associate the authenticity verification results corresponding to the preprocessed image data as authenticity markers to obtain preprocessed image data with authenticity markers.

4. The method for automatically generating a form for verifying the authenticity of urban governance construction images according to claim 3, characterized in that, The recognition models include a text recognition model, an object recognition model, and a table structure recognition model. Step S3 includes the following steps: S31. Input the preprocessed image data with authenticity markers into the text recognition model, object recognition model and table structure recognition model to obtain on-site recognition information including text content, object category, spatial location information and table structure information; The text recognition model is a deep learning-based optical character recognition model, the object recognition model is a deep learning-based target detection model, and the table structure recognition model is used to identify the row boundaries, column boundaries, and cell regions of a table from preprocessed image data, and output the cell position, cell text content, and cell text recognition confidence of each cell. S32. Based on the on-site identification information and the on-site auxiliary information, aggregate the text fragments, table cells, object labels and auxiliary information belonging to the same on-site image data to generate a candidate field set that corresponds one-to-one with each on-site image data. The candidate field set is used to represent multiple candidate field entries in each on-site image data that can be used to fill in the business form. Each candidate field entry in the candidate field set includes at least field name information, field value information, spatial location information of the field in the preprocessed image data, and field identification confidence level. The field name information is generated based on text content and on-site ancillary information, the field value information is generated based on text content and table structure information, and the field identification confidence level is used to indicate the reliability of the field value information. S33. Combine each candidate field set with its corresponding realism label to form a sample set for model training.

5. The method for automatically generating a form for verifying the authenticity of urban governance construction images according to claim 4, characterized in that, The post-trained model in step S4 includes a post-trained authenticity scoring model, a post-trained form template selection model, a post-trained field mapping model, and a post-trained interaction optimization model. The interaction optimization model is a policy model based on reinforcement learning, and its form filling status includes a set of confirmed fields, a set of fields to be confirmed, the number of times the user has modified each field, the user's operation sequence, the current business form type, the current authenticity marker and authenticity score, and the summary information of the most recent steps in the current user operation sequence. Actions used for reinforcement learning include selecting the next field to be presented among multiple fields to be confirmed, choosing whether to autofill a field, and choosing whether to issue a prompt for abnormal fields. The reward signals used for reinforcement learning include at least one or more of the following: total form filling time, number of user operation steps, and one-time approval of business forms; by training reinforcement learning on historical form filling logs, the interaction optimization model can output an optimized field presentation order and autofill strategy given the form filling status, candidate field set, and authenticity label.

6. The method for automatically generating a form for verifying the authenticity of urban governance construction images according to claim 5, characterized in that, S5 includes the following steps: S51. For the newly acquired on-site image data, execute steps S1 and S2 to obtain preprocessed image data with authenticity markers corresponding to the newly acquired on-site image data. S52. Input the preprocessed image data with authenticity markers into the text recognition model, object recognition model and table structure recognition model to generate a set of candidate fields corresponding to the newly acquired on-site image data; S53. Call the trained authenticity scoring model to score the authenticity of the newly acquired on-site image data, and use the obtained authenticity score as the authenticity mark of the newly acquired on-site image data. S54. Input the candidate field set into the trained form template selection model to determine the target business form template corresponding to the newly collected on-site image data; S55. Input the candidate field set into the trained field mapping model to obtain the initial fill values ​​of each field in the target business form template; S56. During the process of the user confirming and modifying the initial fill value, the current form filling status, the candidate field set, and the authenticity marker are input into the trained interaction optimization model to obtain the field presentation order and automatic filling strategy for the target business form template. The target business form template is dynamically adjusted to obtain the finally confirmed electronic business form. S57. The final confirmed electronic business form is associated with the corresponding newly collected on-site image data and authenticity marker, and stored or output.

7. An automatic generation system for urban governance construction image authenticity forms, characterized in that, include: The data acquisition module includes a watermark camera application installed on a mobile terminal and a server connected to the mobile terminal. The watermark camera application is used to acquire construction time and geographical location information from time synchronization service and location service while acquiring construction site image data, and sends the construction site image data, site auxiliary information and intermediately generated anti-tampering marks to the server. The server is used to perform anti-tampering processing on the construction site image data and store the site image data with anti-tampering marks. The authenticity verification module takes tamper-proof marked on-site image data and corresponding on-site ancillary information as input, and performs digital signature verification, time consistency comparison, location consistency comparison, image content consistency detection, and sharpness and quality assessment to obtain time consistency features, location consistency features, digital signature verification features, image content consistency features, and sharpness and quality features. Based on the time consistency features, location consistency features, digital signature verification features, image content consistency features, and sharpness and quality features, the module performs multi-dimensional authenticity verification on the tamper-proof marked on-site image data to obtain authenticity verification results that indicate the authenticity level of the on-site image data. The module also performs image preprocessing on the on-site image data that passes the authenticity verification to generate preprocessed image data with authenticity marks. The recognition and candidate field generation module is used to input preprocessed image data with authenticity markers into text recognition models, object recognition models, and table structure recognition models to generate on-site recognition information, and generate a set of candidate fields based on the on-site recognition information and on-site ancillary information. The model training module is used to construct a sample set based on the candidate field set and the authenticity label, and to train the authenticity scoring model, form template selection model, field mapping model and interaction optimization model to obtain the trained model; The business form generation module is used to call the trained model to process the newly acquired on-site image data, automatically generate electronic business forms corresponding to the newly acquired on-site image data, and associate and store or output the electronic business forms with the newly acquired on-site image data and authenticity markers.

Images