Design drawings and physical verification methods, devices, computer equipment and systems

Abstract

The present application relates to a method, device, computer equipment and system for verifying design drawings and physical objects. The method includes: obtaining a video image sequence, and identifying each physical object in the video image sequence to obtain multiple initial recognition results for each physical object; the video image sequence is an image sequence obtained by a mobile shooting secondary device, and the initial recognition result includes recognition confidence and initial recognition text; for each initial recognition result, judging whether the initial recognition result is a target recognition result according to the recognition confidence of the initial recognition result; obtaining the target recognition text of the physical object according to the initial recognition text of multiple target recognition results of the same physical object; generating on-site wiring structured data according to the target recognition text of each physical object and the multiple target recognition results of each physical object, and comparing the on-site wiring structured data with the preset drawing structured data to obtain the verification result. The use of this method can improve the verification efficiency and the accuracy of the verification.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, device, computer equipment and system for verifying design drawings and physical objects. Background Art

[0002] In the equipment expansion and renovation of converter stations, the consistency check between the design drawings of secondary equipment and the actual wiring is often an important link. At present, manual methods are often used to verify the consistency between the design drawings and the actual equipment of secondary equipment. However, this implementation method is labor-intensive, not only is the verification efficiency slow, but it is also difficult to ensure high accuracy. Summary of the invention

[0003] Based on this, it is necessary to provide a method, device, computer equipment and system for verifying design drawings and physical objects that can improve verification efficiency and accuracy in response to the above technical problems.

[0004] In a first aspect, the present application provides a method for verifying design drawings and physical objects. The method comprises:

[0005] Acquire a video image sequence, and identify each physical object in the video image sequence to obtain a plurality of initial recognition results for each physical object; the video image sequence is an image sequence obtained by a mobile secondary shooting device, and the initial recognition result includes a recognition confidence and an initial recognition text;

[0006] For each of the initial recognition results, judging whether the initial recognition result is a target recognition result according to the recognition confidence of the initial recognition result;

[0007] Obtaining target recognition text of the object according to the initial recognition text of the plurality of target recognition results of the same object;

[0008] According to the target recognition text of each of the physical objects and the multiple target recognition results of each of the physical objects, on-site wiring structured data is generated, and the on-site wiring structured data is compared with the preset drawing structured data to obtain a verification result.

[0009] In one embodiment, the recognition confidence includes object type confidence and text area confidence;

[0010] The step of judging whether the initial recognition result is a target recognition result according to the recognition confidence of the initial recognition result comprises:

[0011] When the object type confidence satisfies the type confidence requirement and the text region confidence satisfies the region confidence requirement, the corresponding initial recognition result is determined to be the target recognition result.

[0012] In one embodiment, the initial recognition result further includes a physical object movement ratio value and a text area movement ratio value.

[0013] When the object type confidence satisfies the type confidence requirement and the text region confidence satisfies the region confidence requirement, the step of determining the corresponding initial recognition result as the target recognition result includes:

[0014] When the physical object movement ratio value meets the first ratio value requirement, the text area movement ratio value meets the second ratio value requirement, the character length of the initial recognized text meets the character length requirement, the preset position text content of the initial recognized text meets the text content requirement, the physical object type confidence meets the type confidence requirement, and the text area confidence meets the area confidence requirement, the corresponding initial recognition result is determined to be the target recognition result.

[0015] In one embodiment, the step of obtaining the target recognition text of the physical object according to the initial recognition text of the plurality of target recognition results of the same physical object comprises:

[0016] For the same physical object, voting is performed one by one according to the initial recognition characters corresponding to the physical object, and the target recognition characters are determined based on the voting results.

[0017] In one embodiment, the initial recognition result includes the type of physical object, physical object coordinates, and text area coordinates;

[0018] The step of generating field wiring structured data according to the target recognition text of each of the physical objects and the multiple target recognition results of each of the physical objects comprises:

[0019] According to each physical object coordinate and each text area coordinate corresponding to each physical object, each physical object type and each target recognition text are sorted to obtain the field wiring structured data.

[0020] In one embodiment, the step of identifying each physical object in the video image sequence to obtain a plurality of initial identification results for each physical object includes:

[0021] Inputting the video image sequence into a pre-trained object tracking and type detection model;

[0022] A plurality of the initial recognition results of each of the physical objects output by the target tracking and type detection model are obtained.

[0023] In a second aspect, the present application also provides a device for verifying design drawings and physical objects. The device comprises:

[0024] An initial recognition result acquisition module is used to acquire a video image sequence and recognize each physical object in the video image sequence to obtain multiple initial recognition results for each physical object; the video image sequence is an image sequence obtained by a mobile secondary shooting device, and the initial recognition result includes recognition confidence and initial recognition text;

[0025] A target recognition result judgment module, used for judging, for each of the initial recognition results, whether the initial recognition result is a target recognition result according to the recognition confidence of the initial recognition result;

[0026] A target recognition text acquisition module, used for obtaining the target recognition text of the physical object according to the initial recognition text of the multiple target recognition results of the same physical object;

[0027] The comparison module is used to generate on-site wiring structured data according to the target recognition text of each of the physical objects and the multiple target recognition results of each of the physical objects, and compare the on-site wiring structured data with the preset drawing structured data to obtain a verification result.

[0028] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0029] Acquire a video image sequence, and identify each physical object in the video image sequence to obtain a plurality of initial recognition results for each physical object; the video image sequence is an image sequence obtained by a mobile secondary shooting device, and the initial recognition result includes a recognition confidence and an initial recognition text;

[0030] For each of the initial recognition results, judging whether the initial recognition result is a target recognition result according to the recognition confidence of the initial recognition result;

[0031] Obtaining target recognition text of the object according to the initial recognition text of the plurality of target recognition results of the same object;

[0032] According to the target recognition text of each of the physical objects and the multiple target recognition results of each of the physical objects, on-site wiring structured data is generated, and the on-site wiring structured data is compared with the preset drawing structured data to obtain a verification result.

[0033] In a fourth aspect, the present application also provides a system for verifying design drawings and physical objects, the system comprising:

[0034] Camera equipment, used to photograph secondary equipment;

[0035] A moving mechanism, mechanically connected to the camera device, and used to move the camera device in multiple directions so that the camera device can capture multiple target images;

[0036] The above-mentioned computer device is electrically connected or communicatively connected to the mobile mechanism, and is electrically connected to the camera device; the computer device is used to control the mobile mechanism to move the camera device and receive a video image sequence, and use the method of any of the above-mentioned embodiments to process the video image sequence to obtain a verification result.

[0037] In a fifth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0038] Acquire a video image sequence, and identify each physical object in the video image sequence to obtain a plurality of initial recognition results for each physical object; the video image sequence is an image sequence obtained by a mobile secondary shooting device, and the initial recognition result includes a recognition confidence and an initial recognition text;

[0039] For each of the initial recognition results, judging whether the initial recognition result is a target recognition result according to the recognition confidence of the initial recognition result;

[0040] Obtaining target recognition text of the object according to the initial recognition text of the plurality of target recognition results of the same object;

[0041] According to the target recognition text of each of the physical objects and the multiple target recognition results of each of the physical objects, on-site wiring structured data is generated, and the on-site wiring structured data is compared with the preset drawing structured data to obtain a verification result.

[0042] In a sixth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0043] Acquire a video image sequence, and identify each physical object in the video image sequence to obtain a plurality of initial recognition results for each physical object; the video image sequence is an image sequence obtained by a mobile secondary shooting device, and the initial recognition result includes a recognition confidence and an initial recognition text;

[0044] For each of the initial recognition results, judging whether the initial recognition result is a target recognition result according to the recognition confidence of the initial recognition result;

[0045] Obtaining target recognition text of the object according to the initial recognition text of the plurality of target recognition results of the same object;

[0046] According to the target recognition text of each of the physical objects and the multiple target recognition results of each of the physical objects, on-site wiring structured data is generated, and the on-site wiring structured data is compared with the preset drawing structured data to obtain a verification result.

[0047] The above-mentioned method, device, computer equipment and system for design drawings and physical objects obtain a video image sequence, and identify each physical object in the video image sequence to obtain multiple initial recognition results for each physical object. Among them, the video image sequence is an image sequence obtained by mobile shooting of secondary equipment, and the initial recognition result includes recognition confidence and initial recognition text. For each initial recognition result, it is determined whether the initial recognition result is a target recognition result according to the recognition confidence of the initial recognition result. According to the initial recognition text of multiple target recognition results of the same physical object, the target recognition text of the physical object is obtained. According to the target recognition text of each physical object and the multiple target recognition results of each physical object, the field wiring structured data is generated, and the field wiring structured data is compared with the preset drawing structured data to obtain the verification result. In this way, the internal wiring and construction design drawings of the secondary equipment can be automatically checked by a computer program without manual verification. On the one hand, it can greatly reduce the workload of on-site staff, thereby improving the verification efficiency. On the other hand, it can also avoid various problems caused by manual detection errors, reduce the occurrence of false detection and omissions, and thus improve the accuracy of verification. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 One of the flow charts of a method for verifying a design drawing and a physical object in one embodiment;

[0049] Figure 2 The second flowchart of the method for verifying a design drawing and an actual object in one embodiment;

[0050] Figure 3 The third flowchart of a method for verifying a design drawing and an actual object in one embodiment;

[0051] Figure 4 It is a structural block diagram of a method and device for verifying a design drawing and a real object in one embodiment;

[0052] Figure 5 is an internal structure diagram of a computer device in one embodiment;

[0053] Figure 6 A schematic diagram of the structure of a system for verifying design drawings and physical objects in one embodiment;

[0054] Figure 7 The figure is a schematic diagram of the verification process of a system for verifying design drawings and actual objects in one embodiment. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0056] In the following embodiments, when used herein, the singular forms "a", "an" and "said / the" may also include plural forms, unless the context clearly indicates otherwise. It should also be understood that the terms "include / comprise" or "have" etc. specify the presence of stated features, wholes, steps, operations, components, parts or combinations thereof, but do not exclude the possibility of the presence or addition of one or more other features, wholes, steps, operations, components, parts or combinations thereof. At the same time, the term "and / or" used in this specification includes any and all combinations of the relevant listed items. At the same time, "plurality" refers to two or more cases, such as 2, 3, 5, 8, etc.

[0057] As described in the background technology, traditional technologies have problems of low efficiency and low accuracy when conducting drawing verification (i.e. verifying the consistency between the design drawings and the actual equipment of secondary equipment). In recent years, with the continuous deepening of the standardization of station construction work, the basic data conditions for the verification of the consistency between design drawings and actual equipment have been gradually met. At the same time, convolutional neural networks in artificial intelligence technology, as the most mature image feature extraction technology at present, can provide a technical basis for fields such as automatic image classification and automatic detection / tracking. The continuous improvement of the automation capabilities of artificial intelligence chips (such as GPU, NPU, etc.) also provides mature technical conditions for realizing drawing verification through artificial intelligence.

[0058] On this basis, the present application provides a method, device, computer equipment and system for verifying design drawings and actual objects. By intelligently and automatically verifying the internal wiring of secondary equipment and the construction design drawings, on the one hand, it can greatly reduce the workload of on-site staff, and on the other hand, it can avoid various problems caused by manual inspection errors, reduce the occurrence of false detections and omissions, and thus improve the accuracy of verification.

[0059] In one embodiment, Figure 1As shown, a method for verifying design drawings and physical objects is provided. This embodiment uses the method applied to a computer device as an example. It can be understood that the method can also be applied to a server, and can also be applied to a system including a computer device and a server, and is implemented through the interaction between the computer device and the server. In one embodiment, the method for verifying design drawings and physical objects of the present application can be applied to the maintenance of secondary equipment and engineering acceptance work of a converter station. In this embodiment, the method for verifying design drawings and physical objects includes the following steps:

[0060] Step 202, obtain a video image sequence, and identify each physical object in the video image sequence to obtain multiple initial recognition results for each physical object; the video image sequence is an image sequence obtained by a mobile secondary shooting device, and the initial recognition result includes recognition confidence and initial recognition text.

[0061] The video image sequence is an image sequence obtained by moving and shooting a secondary device, which may include multiple images. For example, the video image sequence may be a video, or may be multiple images obtained by continuous shooting during movement. In one embodiment, the images in the video image sequence may be arranged in the order of shooting. The physical object may refer to information that can distinguish the wiring object, including but not limited to equipment labels, equipment numbers, and access pipeline numbers.

[0062] Specifically, the computer device can obtain a video image sequence. Since the video image sequence is an image sequence obtained by moving and shooting the secondary device, each image in the video image sequence can include various physical objects and can show the actual wiring conditions of the secondary device. At the same time, the same physical object can be included in multiple images.

[0063] The computer device can process each image in the video image sequence to identify the physical object in each image, and output the initial recognition result of each physical object in the image. After processing multiple images, the computer device can output multiple initial recognition results for each physical object in the video image sequence. For example, the video image sequence includes image A and image B, image A includes physical object A and physical object B, and image B includes physical object B and physical object C. After processing the video image sequence, the computer can obtain an initial recognition result of physical object A (obtained from image A), two initial recognition results of physical object B (obtained from image A and image B respectively), and an initial recognition result of image C (obtained from image C).

[0064] Step 204: for each of the initial recognition results, determine whether the initial recognition result is a target recognition result according to the recognition confidence of the initial recognition result.

[0065] Among them, the target recognition result can be an initial recognition result with higher accuracy.

[0066] Specifically, since the accuracy of some initial recognition results is low, if the initial recognition results are used for subsequent processing, the accuracy of verification is likely to be reduced. Therefore, in order to ensure the accuracy of verification, for each initial recognition result, the accuracy of the initial recognition result can be judged according to the recognition confidence of the initial recognition result. If the recognition confidence of the initial recognition result meets the corresponding confidence requirements, the initial recognition result can be determined to be the target recognition result, and subsequent processing can be performed based on the initial recognition result. If the recognition confidence of the initial recognition result does not meet the corresponding confidence requirements, it can be considered that the accuracy of the initial recognition result is relatively large, and the probability that the initial recognition result does not conform to the actual situation is relatively large, so the initial recognition result is discarded, that is, the initial recognition result is not used in the subsequent processing process.

[0067] Step 206, obtaining the target recognition text of the physical object based on the initial recognition text of the multiple target recognition results of the same physical object.

[0068] The target recognition text may be the best text recognition result of the physical object.

[0069] Specifically, for each physical object, when a target recognition result of the physical object is obtained, the computer device can comprehensively determine the best recognition result of the physical object based on each target recognition result of the physical object. Specifically, each target recognition result includes an initial recognition text. For each physical object, the computer device can determine the target recognition result corresponding to the physical object, and obtain the best text recognition result of the physical object based on the initial recognition text in the determined multiple target recognition results, that is, obtain the target recognition text of the physical object.

[0070] Step 208, generating on-site wiring structured data according to the target recognition text of each of the physical objects and the multiple target recognition results of each of the physical objects, and comparing the on-site wiring structured data with the preset drawing structured data to obtain a verification result.

[0071] Among them, the field wiring structured data can be data that reflects the field wiring situation in a specific data form, so as to facilitate comparison with the design drawings. It can be understood that the data form used can be determined according to the actual situation, and this application does not impose specific restrictions on this. For example, the field wiring structured data can be a drawing that reflects the actual wiring situation, or it can be a data set obtained by arranging according to the field wiring situation. The drawing structured data can be data that reflects the design drawings in a specific data form, and its data form matches the data form of the field wiring structured data for easy comparison.

[0072] Specifically, after obtaining the target recognition text of each physical object and determining the target recognition result of each physical object, the field wiring structured data can be generated accordingly to reflect the field wiring situation of the secondary equipment through the field wiring structured data. The computer device can compare the field wiring structured data with the preset drawing structured data to verify the field wiring situation of the secondary equipment with the design drawing, and obtain the verification result according to the comparison result.

[0073] In the above-mentioned method of design drawings and physical objects, a video image sequence is obtained, and each physical object in the video image sequence is identified to obtain multiple initial identification results for each physical object. Among them, the video image sequence is an image sequence obtained by mobile shooting of secondary equipment, and the initial identification result includes identification confidence and initial identification text. For each initial identification result, it is determined whether the initial identification result is a target identification result according to the identification confidence of the initial identification result. According to the initial identification text of multiple target identification results of the same physical object, the target identification text of the physical object is obtained. According to the target identification text of each physical object and the multiple target identification results of each physical object, the field wiring structured data is generated, and the field wiring structured data is compared with the preset drawing structured data to obtain the verification result. In this way, the internal wiring and construction design drawings of the secondary equipment can be automatically checked by a computer program without manual verification. On the one hand, it can greatly reduce the workload of on-site staff, thereby improving the verification efficiency. On the other hand, it can also avoid various problems caused by manual detection errors, reduce the occurrence of false detection and omissions, and thus improve the accuracy of verification.

[0074] In one embodiment, the recognition confidence includes object type confidence and text area confidence.

[0075] The step of judging whether the initial recognition result is a target recognition result according to the recognition confidence of the initial recognition result comprises: when the object type confidence meets the type confidence requirement and the text area confidence meets the area confidence requirement, determining that the corresponding initial recognition result is the target recognition result.

[0076] The computer device may identify each physical object in the video image sequence to determine the physical type of each physical object. The physical object type confidence is the confidence that the physical object type obtained by identification is consistent with the real physical object type. After the physical object type is identified, the computer device may identify the text area, that is, the area where the text to be identified is located, so as to facilitate subsequent text recognition. The text area confidence is the confidence that the text area obtained by identification is the area where the text to be identified is located.

[0077] Specifically, the computer device can determine whether the initial recognition result is a target recognition result based on the object type confidence and text area confidence of each initial recognition result. For a certain initial recognition result, if the object type confidence of the initial recognition result meets the type confidence requirement, and the text area confidence of the initial recognition result meets the area confidence requirement, then it is determined that the accuracy of the initial recognition result meets the requirements of subsequent processing, and the initial recognition result is a target recognition result.

[0078] It can be understood that both the "type confidence requirement" and the "text area confidence requirement" can be determined based on actual conditions (such as verification accuracy requirements, verification efficiency requirements, etc.), and this application does not impose specific restrictions on this. The specific judgment condition of "satisfying" can also be determined based on actual conditions, for example, it can be greater than (or greater than or equal to) a threshold, fall within a preset value range, etc., and this application does not impose specific restrictions on this. In one embodiment, the judgment condition of "satisfying" can be greater than, that is, when the recognition confidence is greater than the corresponding confidence threshold, it is determined that the recognition confidence meets the corresponding confidence requirement.

[0079] In this embodiment, the object type confidence and text area confidence of each initial recognition result are used to determine whether the initial recognition result is a target recognition result, so that the initial recognition result with higher accuracy can be determined as the target recognition result and then processed. In this way, the accuracy of verification can be further improved.

[0080] In one embodiment, the initial recognition result further includes a physical object movement ratio value and a text area movement ratio value.

[0081] When the object type confidence satisfies the type confidence requirement and the text region confidence satisfies the region confidence requirement, the step of determining the corresponding initial recognition result as the target recognition result includes:

[0082] When the physical object movement ratio value meets the first ratio value requirement, the text area movement ratio value meets the second ratio value requirement, the character length of the initial recognized text meets the character length requirement, the preset position text content of the initial recognized text meets the text content requirement, the physical object type confidence meets the type confidence requirement, and the text area confidence meets the area confidence requirement, the corresponding initial recognition result is determined to be the target recognition result.

[0083] Among them, the object movement ratio value can reflect the position change of the same object in the two adjacent images, and further reflect the spatial change of the object in the video image sequence. The computer device can further determine the object corresponding to each initial recognition result through the object movement ratio value. Similarly, the text area movement ratio value can reflect the position change of the same text area in the two adjacent images, and further reflect the spatial change of the text area in the video image sequence. The preset position of the initially recognized text can be the position of the iconic text. In one embodiment, the preset position can be determined according to the prefix and / or suffix of the number.

[0084] Specifically, the computer device can also determine whether the initial recognition result is a target recognition result by combining the object movement ratio value, the text area movement ratio value and the initial recognition text, that is, combining the object movement ratio value, the text area movement ratio value, the object type confidence, the text area confidence, the text length and text content of the initial recognition text, and making a comprehensive judgment. For each initial recognition result, the initial recognition result is determined as a target recognition result only when the object movement ratio value, the text area movement ratio value, the text length of the initial recognition text, the text content of the initial recognition text, the object type confidence and the text area confidence of the initial recognition result all meet the corresponding requirements.

[0085] It can be understood that the "first ratio value requirement", "second ratio value requirement", "text length requirement" and "preset text content" can all be determined based on actual conditions (such as verification accuracy requirements, verification efficiency requirements, etc.), and this application does not impose specific restrictions on this. In one embodiment, the text length requirement can be the minimum numbering length of the type of physical object, and whether the preset text content meets the text content requirement can be determined based on whether the prefix and / or subsequent numbering rules of the initially recognized text are met.

[0086] In this embodiment, the spatial variation of the physical object in the video image sequence and the initial recognition text are combined to comprehensively analyze whether the initial recognition structure is the target recognition result, thereby further improving the accuracy of the verification.

[0087] In one embodiment, the step of obtaining the target recognition text of the physical object based on the initial recognition text of multiple target recognition results of the same physical object includes: for the same physical object, voting one by one according to each of the initial recognition texts corresponding to the physical object, and determining the target recognition text based on the voting results.

[0088] Specifically, for a certain physical object, after determining the target recognition result of the physical object, the best text recognition result of the physical object can be comprehensively judged based on the initial recognition text of each target recognition result. For each initial recognition text corresponding to the same physical object, the text can be compared and voted for bit by bit according to the text position, iterate the best, and generate the target recognition text of the physical object. For each physical object, its corresponding target recognition text is determined according to the above process. In one embodiment, when voting bit by bit, the text with the most votes can be confirmed as the target recognition text of that bit.

[0089] For example, a physical object includes three target recognition results, and the initial recognition characters of each target recognition result are ABCDE, ABZDE, and XYCDE. Then the first characters of each initial recognition character are A, A, and X, and voting is performed accordingly, and the character with the most votes is confirmed as the first target recognition character, that is, the first character of the target recognition character is A. Similarly, the second characters of each initial recognition character are B, B, and Y, and the target recognition character is the second character B. According to this understanding, the third character, fourth character, and fifth character of the target recognition character are determined respectively, and finally the target recognition character of the physical object is determined to be ABCDE.

[0090] In this embodiment, voting is performed one by one according to each initial recognition text corresponding to the same physical object, and the target recognition text is determined based on the voting result, thereby improving the accuracy of the target recognition text and further improving the verification accuracy.

[0091] In one embodiment, the initial recognition result includes the object type, the object coordinates, and the text area coordinates.

[0092] The step of generating field wiring structured data according to the target recognition text of each of the physical objects and the multiple target recognition results of each of the physical objects comprises:

[0093] According to each physical object coordinate and each text area coordinate corresponding to each physical object, each physical object type and each target recognition text are sorted to obtain the field wiring structured data.

[0094] Specifically, the physical type of all physical objects and the text numbers on the physical objects (i.e., the target identification text of the physical objects) can be arranged according to the coordinate relationship of each physical object in the video image sequence (such as the up-down, left-right relationship) and the coordinate relationship of the text area in the video image sequence, and the field wiring structured data can be generated accordingly. In one embodiment, when performing the comparison, the computer device can check whether the physical object is consistent with the design drawing one by one according to the physical object type, position relationship and target identification text, and obtain the verification result accordingly.

[0095] In this embodiment, the types of objects and target identification characters are sorted according to the coordinates of the objects and the coordinates of the text regions to obtain the field wiring structured data. In this way, the field wiring situation can be reflected through the arrangement of the data, and subsequent comparison is facilitated.

[0096] In one embodiment, the step of identifying each physical object in the video image sequence to obtain a plurality of initial identification results for each physical object includes:

[0097] Inputting the video image sequence into a pre-trained object tracking and type detection model;

[0098] A plurality of the initial recognition results of each of the physical objects output by the target tracking and type detection model are obtained.

[0099] Specifically, a pre-trained target tracking and type detection model can be obtained through sample autonomous training, so that the pre-trained target tracking and type detection model can realize target tracking and physical object recognition. When the video image sequence is input into the pre-trained target tracking and type detection model, the recognition result output by the model can be obtained, and subsequent processing is performed in the manner of the above embodiment.

[0100] In this embodiment, a pre-trained target tracking and type detection model is used to obtain an initial recognition result for each object in a video image sequence, thereby further improving verification efficiency and verification accuracy.

[0101] To facilitate understanding of the solution of this application, a specific example is provided below for illustration. Figure 2-3 As shown, the verification method of the design drawings and the actual objects of this application can be carried out according to Figure 2 The process steps shown are implemented. After acquiring the video image sequence obtained by multi-view shooting, the video image sequence can be input into a pre-trained target tracking and type detection model, which is an AI model, and can realize the tracking and detection of physical objects. The target tracking and type detection model is an AI model that can output the initial recognition results of each physical object. Each initial recognition result includes the physical type of the physical object, the image position coordinates of the physical object in the image (i.e., the physical object coordinates), the physical object movement ratio value, the text area coordinates, the text area movement ratio value and the initial recognition text.

[0102] After obtaining multiple initial recognition results for each physical object, the computer device can summarize and structure the multiple initial recognition results of a physical object to comprehensively judge the best recognition result of each physical object. Specifically, for each initial recognition result of a physical object, the computer device can judge whether the physical coordinates, physical movement ratio value, type confidence, text area confidence and text area movement ratio value of the initial recognition result meet the corresponding requirements, judge whether the text length of the initial recognition text meets the numbering rules of the physical object of the type, and judge whether the prefix / suffix of the initial recognition text meets the numbering rules, and determine the accuracy of the initial recognition result according to each judgment result. When the physical coordinates, physical movement ratio value, type confidence, text area confidence, text area movement ratio value, text length of the initial recognition text and prefix / suffix of the initial recognition text of the initial recognition result all meet the corresponding requirements, the initial recognition result is determined to be the target recognition result. The computer device can use each target recognition result of a physical object to determine the best text recognition result corresponding to the physical object, that is, the target recognition text of the physical object.

[0103] When determining the target recognition text of a certain object, the computer device can compare and vote the initial recognition text of each target recognition result according to the text position, iterate the best, and obtain the best text recognition result. The computer device can arrange the object type and the target recognition text corresponding to each object according to the up-down-left-right relationship of the coordinates of each object in the video image sequence, and generate field wiring structured data accordingly.

[0104] Computer equipment can obtain drawing structured data, which includes the types of physical objects, relative position relationships and text numbers in the design drawings, and compare the on-site wiring structured data with the drawing structured data one by one according to the physical object types, position relationships and text contents to verify whether the physical objects are consistent with the design drawings, and obtain verification results accordingly.

[0105] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0106] Based on the same inventive concept, the embodiment of the present application also provides a design drawing and physical object verification device for implementing the design drawing and physical object verification method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more design drawing and physical object verification device embodiments provided below can refer to the limitations of the design drawing and physical object verification method above, and will not be repeated here.

[0107] In one embodiment, Figure 4 As shown, a device for verifying design drawings and physical objects is provided, including: an initial recognition result acquisition module, a target recognition result judgment module, a target recognition text acquisition module and a comparison module, wherein:

[0108] An initial recognition result acquisition module is used to acquire a video image sequence and recognize each physical object in the video image sequence to obtain multiple initial recognition results for each physical object; the video image sequence is an image sequence obtained by a mobile secondary shooting device, and the initial recognition result includes recognition confidence and initial recognition text;

[0109] A target recognition result judgment module, used for judging, for each of the initial recognition results, whether the initial recognition result is a target recognition result according to the recognition confidence of the initial recognition result;

[0110] A target recognition text acquisition module, used for obtaining the target recognition text of the physical object according to the initial recognition text of the multiple target recognition results of the same physical object;

[0111] The comparison module is used to generate on-site wiring structured data according to the target recognition text of each of the physical objects and the multiple target recognition results of each of the physical objects, and compare the on-site wiring structured data with the preset drawing structured data to obtain a verification result.

[0112] In one embodiment, the recognition confidence includes object type confidence and text area confidence. The target recognition result judgment module includes a confidence judgment unit, which is used to determine that the corresponding initial recognition result is the target recognition result when the object type confidence meets the type confidence requirement and the text area confidence meets the area confidence requirement.

[0113] In one embodiment, the initial recognition result further includes a physical object movement ratio value and a text area movement ratio value. The confidence judgment unit is further configured to determine that the corresponding initial recognition result is the target recognition result when the physical object movement ratio value meets the first ratio value requirement, the text area movement ratio value meets the second ratio value requirement, the text length of the initial recognition text meets the text length requirement, the preset text content of the initial recognition text meets the text content requirement, the physical object type confidence meets the type confidence requirement, and the text area confidence meets the area confidence requirement.

[0114] In one embodiment, the target recognition text acquisition module includes a voting unit, which is used to vote for the same physical object one by one according to the initial recognition texts corresponding to the physical object, and determine the target recognition text based on the voting results.

[0115] In one embodiment, the initial recognition result includes the object type, the object coordinates and the text area coordinates. The comparison module includes a data generation unit, which is used to sort the object type and the target recognition text according to the object coordinates and the text area coordinates corresponding to each object, so as to obtain the field wiring structured data.

[0116] In one embodiment, the initial recognition result acquisition module includes a recognition unit, which is used to input the video image sequence into a pre-trained target tracking and type detection model, and obtain multiple initial recognition results of each of the physical objects output by the target tracking and type detection model.

[0117] Each module in the above-mentioned design drawing and physical object verification device can be fully or partially implemented by software, hardware and their combination. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the corresponding operations of the above modules.

[0118] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 5As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a method for verifying a design drawing and a physical object is implemented.

[0119] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0120] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0121] Acquire a video image sequence, and identify each physical object in the video image sequence to obtain a plurality of initial recognition results for each physical object; the video image sequence is an image sequence obtained by a mobile secondary shooting device, and the initial recognition result includes a recognition confidence and an initial recognition text;

[0122] For each of the initial recognition results, judging whether the initial recognition result is a target recognition result according to the recognition confidence of the initial recognition result;

[0123] Obtaining target recognition text of the object according to the initial recognition text of the plurality of target recognition results of the same object;

[0124] According to the target recognition text of each of the physical objects and the multiple target recognition results of each of the physical objects, on-site wiring structured data is generated, and the on-site wiring structured data is compared with the preset drawing structured data to obtain a verification result.

[0125] In one embodiment, the recognition confidence includes object type confidence and text area confidence. When the processor executes the computer program, the following steps are also implemented: when the object type confidence meets the type confidence requirement and the text area confidence meets the area confidence requirement, determining the corresponding initial recognition result as the target recognition result.

[0126] In one embodiment, the initial recognition result further includes a physical object movement ratio value and a text area movement ratio value. When the processor executes the computer program, the following steps are also implemented: when the physical object movement ratio value meets the first ratio value requirement, the text area movement ratio value meets the second ratio value requirement, the text length of the initial recognition text meets the text length requirement, the preset text content of the initial recognition text meets the text content requirement, the physical object type confidence meets the type confidence requirement, and the text area confidence meets the area confidence requirement, the corresponding initial recognition result is determined to be the target recognition result.

[0127] In one embodiment, when the processor executes the computer program, the following steps are also implemented: for the same physical object, voting is performed one by one according to each of the initial recognition characters corresponding to the physical object, and the target recognition characters are determined based on the voting results.

[0128] In one embodiment, the initial recognition result includes the object type, the object coordinates and the text area coordinates. When the processor executes the computer program, the following steps are also implemented: according to the object coordinates and the text area coordinates corresponding to each of the objects, the object type and the target recognition text are sorted to obtain the field wiring structured data.

[0129] In one embodiment, when the processor executes the computer program, the following steps are also implemented: inputting the video image sequence into a pre-trained target tracking and type detection model; obtaining multiple initial recognition results for each of the physical objects output by the target tracking and type detection model.

[0130] In one embodiment, Figure 6 As shown, a system for verifying design drawings and actual objects is provided.

[0131] The system includes:

[0132] Camera equipment, used to photograph secondary equipment;

[0133] A moving mechanism, mechanically connected to the camera device, and used to move the camera device in multiple directions so that the camera device can capture multiple target images;

[0134] The above-mentioned computer device is electrically connected or communicatively connected to the mobile mechanism, and is electrically connected to the camera device; the computer device is used to control the mobile mechanism to move the camera device and receive a video image sequence, and use the method described in any of the above embodiments to process the video image sequence to obtain a verification result.

[0135] In one embodiment, the computer device can control the opening / closing, up / down movement, left / right translation of the mobile mechanism, and receive the video image sequence captured by the camera device in real time. In one example, the computer device can be a laptop or an MCU control device.

[0136] In one embodiment, if the mobile mechanism and the computer device are connected by communication, a WIFI connection can be used for connection.

[0137] In this embodiment, by combining artificial intelligence technology with automatic control technology, a camera device is used to automatically shoot the internal wiring of the equipment from multiple angles, transmit video images in real time, and transmit the images to a computer device in real time to check whether they are consistent with the drawings. In this way, human resource costs can be greatly reduced, human errors can be avoided, and the accuracy and work efficiency of operation and maintenance inspection and project acceptance can be improved.

[0138] In one embodiment, the moving mechanism may be an automatic telescopic moving mechanism. In one embodiment, the moving mechanism may include a motor that is mechanically connected to the camera device and electrically or communicatively connected to the computer device.

[0139] In one embodiment, the moving mechanism can move the camera device at a constant speed to shoot the connection status of the internal device of the secondary device at a constant speed. In one example, the moving mechanism can drive the camera device to move vertically up / down at a constant speed and control the on / off and left / right translation of the camera device to obtain the best shooting angle.

[0140] In one example, the process of using the design drawing and physical object verification system to verify the actual object can be as follows: Figure 7 Specifically including:

[0141] Step 702, placing the mobile mechanism and the camera device in front of the secondary device;

[0142] Step 704, the computer device controls the moving mechanism to move the camera device, and controls the camera device to start shooting;

[0143] Step 706, the computer device receives the video image sequence in real time, tracks and recognizes the type of the physical object and the text on it;

[0144] Step 708: The computer device collects the physical object categories and the initial recognition text thereon and makes a judgment according to the above method, for example, whether the initial recognition text best conforms to the numbering rule of the corresponding part of the drawing.

[0145] Step 710: If not, the computer device may control the moving mechanism to move the camera device at multiple angles to perform multi-angle shooting.

[0146] In step 712, the computer device checks the obtained field wiring structured data with the drawing structured data, and records whether the two are consistent and the differences between the two.

[0147] Step 714, the computer device checks the wiring conditions of the secondary devices one by one until the checking is completed.

[0148] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0149] Acquire a video image sequence, and identify each physical object in the video image sequence to obtain a plurality of initial recognition results for each physical object; the video image sequence is an image sequence obtained by a mobile secondary shooting device, and the initial recognition result includes a recognition confidence and an initial recognition text;

[0150] For each of the initial recognition results, judging whether the initial recognition result is a target recognition result according to the recognition confidence of the initial recognition result;

[0151] Obtaining target recognition text of the object according to the initial recognition text of the plurality of target recognition results of the same object;

[0152] According to the target recognition text of each of the physical objects and the multiple target recognition results of each of the physical objects, on-site wiring structured data is generated, and the on-site wiring structured data is compared with the preset drawing structured data to obtain a verification result.

[0153] In one embodiment, the recognition confidence includes object type confidence and text area confidence. When the computer program is executed by the processor, the following steps are also implemented: when the object type confidence meets the type confidence requirement and the text area confidence meets the area confidence requirement, determining the corresponding initial recognition result as the target recognition result.

[0154] In one embodiment, the initial recognition result further includes a physical object movement ratio value and a text area movement ratio value. When the computer program is executed by the processor, the following steps are also implemented: when the physical object movement ratio value meets the first ratio value requirement, the text area movement ratio value meets the second ratio value requirement, the text length of the initial recognition text meets the text length requirement, the preset text content of the initial recognition text meets the text content requirement, the physical object type confidence meets the type confidence requirement, and the text area confidence meets the area confidence requirement, the corresponding initial recognition result is determined to be the target recognition result.

[0155] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: for the same physical object, voting is performed one by one according to each of the initial recognition characters corresponding to the physical object, and the target recognition characters are determined based on the voting results.

[0156] In one embodiment, the initial recognition result includes the object type, the object coordinates and the text area coordinates. When the computer program is executed by the processor, the following steps are also implemented: according to the object coordinates and the text area coordinates corresponding to each of the objects, the object type and the target recognition text are sorted to obtain the field wiring structured data.

[0157] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented: inputting the video image sequence into a pre-trained target tracking and type detection model; obtaining multiple initial recognition results for each of the physical objects output by the target tracking and type detection model.

[0158] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0159] Acquire a video image sequence, and identify each physical object in the video image sequence to obtain a plurality of initial recognition results for each physical object; the video image sequence is an image sequence obtained by a mobile secondary shooting device, and the initial recognition result includes a recognition confidence and an initial recognition text;

[0160] For each of the initial recognition results, judging whether the initial recognition result is a target recognition result according to the recognition confidence of the initial recognition result;

[0161] Obtaining target recognition text of the object according to the initial recognition text of the plurality of target recognition results of the same object;

[0162] According to the target recognition text of each of the physical objects and the multiple target recognition results of each of the physical objects, on-site wiring structured data is generated, and the on-site wiring structured data is compared with the preset drawing structured data to obtain a verification result.

[0163] In one embodiment, the recognition confidence includes object type confidence and text area confidence. When the computer program is executed by the processor, the following steps are also implemented: when the object type confidence meets the type confidence requirement and the text area confidence meets the area confidence requirement, determining the corresponding initial recognition result as the target recognition result.

[0164] In one embodiment, the initial recognition result further includes a physical object movement ratio value and a text area movement ratio value. When the computer program is executed by the processor, the following steps are also implemented: when the physical object movement ratio value meets the first ratio value requirement, the text area movement ratio value meets the second ratio value requirement, the text length of the initial recognition text meets the text length requirement, the preset text content of the initial recognition text meets the text content requirement, the physical object type confidence meets the type confidence requirement, and the text area confidence meets the area confidence requirement, the corresponding initial recognition result is determined to be the target recognition result.

[0165] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: for the same physical object, voting is performed one by one according to each of the initial recognition characters corresponding to the physical object, and the target recognition characters are determined based on the voting results.

[0166] In one embodiment, the initial recognition result includes the object type, the object coordinates and the text area coordinates. When the computer program is executed by the processor, the following steps are also implemented: according to the object coordinates and the text area coordinates corresponding to each of the objects, the object type and the target recognition text are sorted to obtain the field wiring structured data.

[0167] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented: inputting the video image sequence into a pre-trained target tracking and type detection model; obtaining multiple initial recognition results for each of the physical objects output by the target tracking and type detection model.

[0168] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0169] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0170] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for verifying design drawings and objects, characterized in that: The method comprises: Acquire a video image sequence, and identify each physical object in the video image sequence to obtain a plurality of initial recognition results for each physical object; the video image sequence is an image sequence obtained by a mobile secondary shooting device, and the initial recognition result includes recognition confidence, initial recognition text, physical object movement ratio value and text area movement ratio value; the recognition confidence includes physical object type confidence and text area confidence; For each of the initial recognition results, judging whether the initial recognition result is a target recognition result according to the recognition confidence of the initial recognition result, specifically comprising: when the physical object movement ratio value meets the first ratio value requirement, the text area movement ratio value meets the second ratio value requirement, the text length of the initial recognition text meets the text length requirement, the pre-positioned text content of the initial recognition text meets the text content requirement, the physical object type confidence meets the type confidence requirement, and the text area confidence meets the area confidence requirement, determining that the corresponding initial recognition result is the target recognition result; Obtaining target recognition text of the object according to the initial recognition text of the plurality of target recognition results of the same object; According to the target recognition text of each of the physical objects and the multiple target recognition results of each of the physical objects, on-site wiring structured data is generated, and the on-site wiring structured data is compared with the preset drawing structured data to obtain a verification result.

2. The method according to claim 1, characterized in that The step of obtaining the target recognition text of the physical object according to the initial recognition text of the plurality of target recognition results of the same physical object comprises: For the same physical object, voting is performed one by one according to the initial recognition characters corresponding to the physical object, and the target recognition characters are determined based on the voting results.

3. The method according to claim 1, characterized in that: The initial recognition result includes the object type, object coordinates and text area coordinates; The step of generating field wiring structured data according to the target recognition text of each of the physical objects and the multiple target recognition results of each of the physical objects comprises: According to each physical object coordinate and each text area coordinate corresponding to each physical object, each physical object type and each target recognition text are sorted to obtain the field wiring structured data.

4. The method according to any one of claims 1 to 3, characterized in that: The step of identifying each physical object in the video image sequence to obtain a plurality of initial identification results for each physical object comprises: Inputting the video image sequence into a pre-trained object tracking and type detection model; A plurality of the initial recognition results of each of the physical objects output by the target tracking and type detection model are obtained.

5. A device for verifying design drawings and objects, characterized in that: The device comprises: An initial recognition result acquisition module is used to acquire a video image sequence and recognize each physical object in the video image sequence to obtain multiple initial recognition results for each physical object; the video image sequence is an image sequence obtained by a mobile secondary shooting device, and the initial recognition result includes recognition confidence, initial recognition text, physical object movement ratio value and text area movement ratio value; the recognition confidence includes physical object type confidence and text area confidence; A target recognition result judgment module is used to judge whether each of the initial recognition results is a target recognition result according to the recognition confidence of the initial recognition result, specifically comprising: when the physical object movement ratio value meets the first ratio value requirement, the text area movement ratio value meets the second ratio value requirement, the text length of the initial recognition text meets the text length requirement, the pre-positioned text content of the initial recognition text meets the text content requirement, the physical object type confidence meets the type confidence requirement, and the text area confidence meets the area confidence requirement, determining that the corresponding initial recognition result is the target recognition result; A target recognition text acquisition module, used for obtaining the target recognition text of the physical object according to the initial recognition text of the multiple target recognition results of the same physical object; The comparison module is used to generate on-site wiring structured data according to the target recognition text of each of the physical objects and the multiple target recognition results of each of the physical objects, and compare the on-site wiring structured data with the preset drawing structured data to obtain a verification result.

6. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

7. A system for verifying design drawings and objects, characterized in that: The system comprises: Camera equipment, used to photograph secondary equipment; A moving mechanism, mechanically connected to the camera device, and used to move the camera device in multiple directions so that the camera device can capture multiple target images; The computer device as described in claim 6 is electrically connected or communicatively connected to the mobile mechanism, and is electrically connected to the camera device; the computer device is used to control the mobile mechanism to move the camera device and receive a video image sequence, and process the video image sequence using the method described in any one of claims 1 to 4 to obtain a verification result.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images