Inspection method, marking method, system, device, terminal, equipment and medium

Abstract

This disclosure provides an inspection method, inspection and annotation method, inspection and annotation system, inspection device, inspection and annotation terminal, electronic device, and computer-readable storage medium, relating to the field of computer technology. The method includes acquiring a machine-readable scanned image of an item to be inspected; performing recognition on the machine-readable scanned image according to an image analysis model to obtain the location information and category information corresponding to the target item in the machine-readable scanned image; if the category information corresponding to the target item matches a predetermined category, sending the location information corresponding to the target item and the machine-readable scanned image to a target terminal for performing an inspection and annotation operation on the machine-readable scanned image by comparing the item and location information; and receiving the inspection and annotation result obtained by the target terminal based on the inspection and annotation operation. This method simplifies the collection of annotation data, reduces the cost of manual annotation, and increases the accuracy of annotation.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to an inspection method, an inspection labeling method, an inspection labeling system, an inspection device, an inspection labeling terminal, an electronic device, and a computer-readable storage medium. Background Technology

[0002] Currently, the inspection system utilizes artificial intelligence to perform deep learning on machine-inspected images and the corresponding item information, enabling machines to automatically determine whether there are contraband or foreign objects in the images. The implementation and widespread adoption of AI-powered image review technology requires image annotation and algorithm training on massive amounts of images.

[0003] Existing image annotation systems mainly complete the annotation work after the machine inspection process is completed by analyzing historical data of machine inspection, such as scanned images and inspection conclusions. However, these annotation processes are cumbersome, which not only consumes a lot of manpower but also affects the accuracy of the annotation data.

[0004] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] The purpose of this disclosure is to provide an inspection method, inspection labeling method, inspection labeling system, inspection device, inspection labeling terminal, electronic device, and computer-readable storage medium, which at least to some extent overcomes the problems of cumbersome labeling workflow for inspection data in related technologies, resulting in a large amount of manpower consumption and inaccurate labeling data.

[0006] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0007] According to one aspect of this disclosure, an inspection method is provided, applied to a server, comprising: acquiring a machine inspection scan image of an item to be inspected; performing recognition on the machine inspection scan image according to an image analysis model to obtain location information and category information corresponding to the target item in the machine inspection scan image; if the category information corresponding to the target item matches a predetermined category, sending the location information corresponding to the target item and the machine inspection scan image to a target terminal so as to perform an inspection annotation operation on the machine inspection scan image by comparing the item to be inspected and the location information; and receiving the inspection annotation result obtained by the target terminal according to the inspection annotation operation.

[0008] In some embodiments of this disclosure, the verification method further includes: sending the verification annotation results to a model optimization device to optimize the image analysis model based on the verification annotation results.

[0009] According to another aspect of this disclosure, a verification and annotation method is provided, applied to a terminal, comprising: receiving a machine inspection scan image sent by a server and location information corresponding to a target item obtained by identifying the machine inspection scan image according to an image analysis model, wherein the category information of the target item matches a predetermined category; in response to receiving the location information corresponding to the target item and the machine inspection scan image, starting a terminal annotation program to display the location information corresponding to the target item and the machine inspection scan image; in response to generating a verification and annotation result by performing a verification and annotation operation on the machine inspection scan image by comparing the inspection item and the location information; and sending the verification and annotation result to the server.

[0010] In some embodiments of this disclosure, the location information includes a first graphic area of ​​the target item. Therefore, launching the terminal annotation program to display the location information of the target item and the machine inspection scan image includes: launching the terminal annotation program to display the machine inspection scan image and the first graphic area.

[0011] In some embodiments of this disclosure, the inspection labeling result includes a first inspection labeling result. Generating the inspection labeling result in response to the inspection labeling operation performed on the machine inspection scan image by comparing the inspection item and location information includes: in response to a selection operation on a first graphic area, displaying a first text labeling area corresponding to the first graphic area; in response to a first input operation on the first text labeling area, determining a first text labeling corresponding to the first input operation; and generating the first inspection labeling result based on the first text labeling.

[0012] In some embodiments of this disclosure, the inspection annotation result further includes a second inspection annotation result. Generating the inspection annotation result in response to the inspection annotation operation performed on the machine inspection scan image by comparing the inspection item and location information further includes: generating a second graphic area in response to a graphic annotation operation on the machine inspection scan image; displaying a second text annotation area corresponding to the second graphic area in response to a selection operation on the second graphic area; determining a second text annotation corresponding to the second input operation in response to a second input operation on the second text annotation area; and generating a second inspection annotation result based on the second graphic area and the second text annotation.

[0013] In some embodiments of this disclosure, the graphic annotation operation of the machine inspection scan image includes: a correction operation of a first graphic region of the machine inspection scan image; and / or closed trajectory information input on the machine inspection scan image.

[0014] In some embodiments of this disclosure, the verification and annotation method further includes: displaying the boundary of the first graphic region in a first line style; and displaying the boundary of the second graphic region in a second line style.

[0015] In some embodiments of this disclosure, the terminal annotation program is a client application based on Three.js and UniAPP.

[0016] According to another aspect of this disclosure, an inspection and labeling system is provided, including a scanning device, an inspection device, and a terminal; wherein the scanning device scans the items to be inspected, generating machine-readable inspection scan images; the inspection device is connected to the scanning device and is used to acquire the machine-readable inspection scan images of the items to be inspected; the machine-readable inspection scan images are identified according to an image analysis model to obtain the location information and category information corresponding to the target items in the machine-readable inspection scan images; if the category information corresponding to the target items matches a predetermined category, the location information corresponding to the target items and the machine-readable inspection scan images are sent to the terminal so that the machine-readable inspection scan images can be inspected and labeled by comparing the items to be inspected and the location information. The terminal is connected to the inspection device. The terminal receives machine-scanned images of the target item and location information of the target item identified by an image analysis model, wherein the category information of the target item matches a predetermined category. In response to receiving the location information of the target item and the machine-scanned images, the terminal initiates a labeling program to display the location information of the target item and the machine-scanned images. In response to performing a labeling operation on the machine-scanned images by comparing the target item and the location information, the terminal generates a labeling result. The terminal then sends the labeling result to the inspection device. The inspection device also receives the labeling result obtained by the target terminal based on the labeling operation.

[0017] In some embodiments of this disclosure, a model optimization device is also included, wherein the model optimization device is connected to the verification device, and the model optimization device is used to obtain the verification annotation results from the verification device; and optimize the image analysis model based on the verification annotation results.

[0018] According to another aspect of this disclosure, an inspection apparatus is provided, comprising: an image acquisition unit for acquiring a machine inspection scan image of an item to be inspected; an inspection unit for performing recognition on the machine inspection scan image according to an image analysis model to obtain location information and category information corresponding to the target item in the machine inspection scan image; a first sending unit for sending the location information corresponding to the target item and the machine inspection scan image to a target terminal if the category information corresponding to the target item matches a predetermined category, so as to perform an inspection annotation operation on the machine inspection scan image by comparing the item to be inspected and the location information; and an annotation result acquisition unit for receiving the inspection annotation result obtained by the target terminal according to the inspection annotation operation.

[0019] According to another aspect of this disclosure, an inspection and labeling terminal is provided, comprising: a receiving unit for receiving a machine inspection scan image sent by a server and location information corresponding to a target item identified according to an image analysis model, wherein the category information of the target item matches a predetermined category; a display unit for displaying the location information corresponding to the target item and the machine inspection scan image by initiating a terminal labeling program in response to receiving the location information corresponding to the target item and the machine inspection scan image; a labeling unit for generating an inspection and labeling result in response to an inspection and labeling operation performed on the machine inspection scan image by comparing the inspection item and the location information; and a labeling result sending unit for sending the inspection and labeling result to the server.

[0020] According to another aspect of this disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform the above-described verification method or verification labeling method by executing the executable instructions.

[0021] According to another aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the above-described verification method or verification annotation method.

[0022] The verification method provided by the embodiments of this disclosure has at least the following beneficial effects:

[0023] By adding an annotation step to the inspection task, the collection of annotation data is simplified and the cost of manual annotation is reduced without affecting the progress of normal inspection work. At the same time, the accuracy of annotation is increased, and the model accuracy and training efficiency of the image recognition model using the annotation data as training samples are improved.

[0024] Furthermore, the method described in this application performs the inspection and labeling task via a terminal, making it convenient for on-site personnel to perform practical operations, and offering excellent portability and feasibility. Moreover, by comparing the inspection items and location information, the false negative rate in the inspection process is reduced.

[0025] Furthermore, the method of checking and labeling by matching predetermined categories in this application can improve the flexibility of the inspection work, further improve the efficiency of the inspection work, and enhance the inspection experience of the inspected object.

[0026] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0027] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0028] Figure 1 A flowchart illustrating a verification method performed by a server in some embodiments of this disclosure is shown.

[0029] Figure 2 A flowchart illustrating a terminal-executed verification annotation method is shown in some embodiments of this disclosure.

[0030] Figure 3A The flowchart illustrates a method for generating verification labeling results by a terminal performing a verification labeling operation in some embodiments of this disclosure.

[0031] Figure 3B This diagram illustrates an interface diagram showing the generation of verification labeling results by a terminal performing a verification labeling operation in some embodiments of this disclosure.

[0032] Figure 4A This document illustrates a flowchart of a method for generating verification labeling results by performing a verification labeling operation on a terminal in some embodiments of the present disclosure.

[0033] Figure 4B This diagram illustrates an interface schematic of another terminal performing a verification annotation operation to generate verification annotation results, as shown in some embodiments of this disclosure.

[0034] Figure 5 The diagram illustrates a method for displaying the location information of a target item based on the Uniapp framework and Three.js, as shown in some embodiments of this disclosure.

[0035] Figure 6 A schematic diagram of an inspection and labeling system is shown in some embodiments of this disclosure.

[0036] Figure 7 A schematic diagram of an inspection device is shown in some embodiments of this disclosure.

[0037] Figure 8 A schematic diagram of a verification labeling terminal is shown in some embodiments of this disclosure.

[0038] Figure 9 A structural block diagram of a verification labeling computer device is shown in some embodiments of this disclosure. Detailed Implementation

[0039] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0040] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0041] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this disclosure, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0042] In view of the technical problems existing in the above-mentioned related technologies, the present disclosure provides a verification and labeling method to solve at least one or all of the above-mentioned technical problems.

[0043] The terminals involved in the embodiments of this application can be mobile terminals such as mobile phones, game consoles, tablet computers, e-book readers, smart glasses, MP4 (Moving Picture Experts Group Audio Layer IV) players, smart home devices, AR (Augmented Reality) devices, VR (Virtual Reality) devices, etc. Alternatively, the terminal can also be a personal computer (PC), such as a laptop computer, etc.

[0044] The terminal may contain a terminal annotation program that provides verification and annotation methods.

[0045] The terminal and the server are connected via a communication network. Optionally, the communication network can be a wired network or a wireless network.

[0046] The server can be a single server, a combination of several servers, a virtualization platform, or a cloud computing service center. The server provides backend services for the verification and annotation methods. In some embodiments of this application, the server undertakes the main computational work, while the terminal undertakes the annotation computation work.

[0047] Those skilled in the art will understand that the number of terminals described above can be more or less. For example, there may be only one terminal, or there may be dozens or hundreds of terminals, or even more. This application does not limit the number of terminals or the type of device.

[0048] Optionally, the aforementioned wireless or wired networks use standard communication technologies and / or protocols. The network is typically the Internet, but can also be any network, including but not limited to Local Area Networks (LANs), Metropolitan Area Networks (MANs), Wide Area Networks (WANs), mobile, wired or wireless networks, private networks, or any combination of virtual private networks. In some embodiments, technologies and / or formats including Hyper Text Markup Language (HTML), Extensible Markup Language (XML), etc., are used to represent data exchanged over the network. Furthermore, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Networks (VPNs), and Internet Protocol Security (IPsec) can be used to encrypt all or some links. In other embodiments, custom and / or dedicated data communication technologies can be used to replace or supplement the aforementioned data communication technologies.

[0049] The following will describe in more detail the steps of the inspection method and inspection labeling method in this example embodiment, with reference to the accompanying drawings and embodiments.

[0050] Figure 1 The flowchart illustrates a verification method executed by a server in some embodiments of this disclosure. The methods provided in the embodiments of this disclosure can be executed by any electronic device with computing power, such as a server. In the following illustrative examples, a server is used as the execution subject.

[0051] like Figure 1As shown, the verification method 100 provided in some embodiments of this disclosure may include the following steps:

[0052] In step S110, an inspection scan image of the item to be inspected is acquired.

[0053] The machine inspection scan image is the image generated by the scanning device scanning the item to be inspected. After the scan is completed, the scanning device sends the machine inspection scan image to the server, which then receives and stores the image.

[0054] In step S120, the machine inspection scan image is identified according to the image analysis model to obtain the location information and category information of the target item in the machine inspection scan image.

[0055] Image analysis models for recognizing scanned images from machine inspections refer to the technology of using machines to process, analyze, and understand images in order to identify various different items.

[0056] The target item refers to one or more identified items. The target item in the machine inspection scan image can correspond to certain location and category information within the machine inspection scan image.

[0057] In some embodiments of this disclosure, the location information may be a set of pixel coordinates of the contour lines of a polygonal region identified by an image analysis model; the location information may also be the vertex coordinates and side length information of a rectangular region identified by an image analysis model; the location information may also be the center coordinates and radius information of a circular region identified by an image analysis model.

[0058] Among them, category information refers to the item category of the target item as identified by the image analysis model.

[0059] In this example, the image recognition algorithm incorporated in the image analysis model is not specifically limited. Those skilled in the art can refer to the records in related technologies when implementing the technical solution of this application. For example, in one implementation, the above-mentioned image analysis model can be a deep learning model trained based on a neural network combined with a large number of image recognition samples.

[0060] In some embodiments of this disclosure, the image analysis model may also be deployed on another server to perform recognition of machine-checked scanned images independently and return the recognition results to the server that performs the verification annotation.

[0061] In step S130, if the category information corresponding to the target item matches the predetermined category, the location information corresponding to the target item and the machine inspection scan image are sent to the target terminal so that the machine inspection scan image can be checked and labeled by comparing the inspection item and the location information.

[0062] The pre-defined category refers to one or more item categories set by the user before the scanned image is acquired. For example, the pre-defined category can be set as a prohibited item category, including firearms, knives, lighters, controlled implements, tools, flammable and explosive materials, liquids, electronic devices, power banks, etc.

[0063] In some embodiments of this disclosure, the server determines whether one or more identified target items include target items of the same category as those in a predetermined category, such as the prohibited items categories listed above. If so, the server sends the location information and scanned image of the target item to the target terminal; if not, the server will not be triggered to send the identification result to the target terminal.

[0064] The target terminal can be understood as a terminal device used to perform inspection and annotation on machine-inspected scanned images.

[0065] In step S140, the verification labeling result obtained by the target terminal based on the verification labeling operation is received.

[0066] The inspection and annotation operation refers to the user marking the location and attributes of the target object using graphic annotations such as polygons, rectangles, and circles, or text annotations, based on the on-site inspection results. After the annotation operation is completed, the location, attributes, and other annotation information of the annotated item are used as the inspection and annotation results.

[0067] The method disclosed in this application simplifies the collection of labeled data and reduces the cost of manual labeling by adding a labeling step to the inspection task without affecting the progress of normal inspection work; at the same time, it increases the accuracy of labeling and improves the model accuracy and model training efficiency of the image recognition model using labeled data as training samples.

[0068] Furthermore, the method described in this application performs the inspection and labeling task via a terminal, making it convenient for on-site personnel to perform practical operations, and offering excellent portability and feasibility. Moreover, by comparing the inspection items and location information, the false negative rate in the inspection process is reduced.

[0069] Furthermore, the method of checking and labeling by matching predetermined categories in this application helps to improve the flexibility of the inspection, further improve the efficiency of the inspection, and enhance the inspection experience of the inspected object.

[0070] In some embodiments of this disclosure, after step S140, the verification and annotation results are sent to a model optimization device to optimize the image analysis model based on the verification and annotation results.

[0071] The model optimization device is used to optimize the image recognition algorithm in the image analysis model.

[0072] In some embodiments of this disclosure, the image analysis model can be optimized in real time based on the verification and annotation results. For example, incremental training can be performed on the image recognition algorithm embedded in the image analysis model based on the location information corresponding to the target item obtained from the image analysis model and the verification and annotation results returned by the terminal, thereby achieving the effect of real-time optimization for each verification result. Specifically, the training process may involve calculating the loss based on new samples on the basis of an existing image recognition model, and then optimizing the algorithm through gradient descent.

[0073] By using the verification and annotation results as feedback data to optimize the model in real time, the model iteration is accelerated, thus improving the real-time image judgment accuracy of the system. On the one hand, this reduces the false alarm rate in the intelligent verification process; on the other hand, it also reduces the false alarm rate of the image recognition algorithm, thereby reducing the waste of human and material resources. In summary, the verification and annotation method disclosed in this application not only improves the accuracy of verification but also accelerates the system optimization process, providing a new type of verification system with high iterativeness and high reliability for verification work.

[0074] In some embodiments of this disclosure, the above method is applicable to general inspection scenarios, that is, each item is scanned by machine and labeled by the terminal, which improves the inspection efficiency in general inspection scenarios and effectively prevents the omission of illegal and prohibited items.

[0075] In some embodiments of this disclosure, the above method is applicable to sampling inspection scenarios, improving the accuracy of item detection and making the item detection results more convincing, thereby improving the quality and efficiency of sampling inspection work.

[0076] Figure 2 The flowchart illustrates a verification and annotation method executed by a terminal in some embodiments of this disclosure. In the following illustrative examples, the terminal is used as the execution subject. Figure 2 As shown, method 200 may include the following steps:

[0077] In step S210, the machine inspection scan image sent by the server and the location information of the target item obtained by identifying the machine inspection scan image according to the image analysis model are received, wherein the category information of the target item matches the predetermined category.

[0078] The machine inspection scan image is the image generated by the scanning device scanning the item to be inspected. After the scan is completed, the scanning device sends the machine inspection scan image to the server, which receives the machine inspection scan image and uses an image analysis model to perform image recognition on the machine inspection scan image.

[0079] Step S210 corresponds to steps S110 to S130, and will not be described again here.

[0080] In step S220, in response to receiving the location information and machine inspection scan image corresponding to the target item, the terminal annotation program is started to display the location information and machine inspection scan image corresponding to the target item.

[0081] The location information can be the coordinates of the target item, so that the terminal annotation program can display the location of the target item in the form of lines, rectangles, polygons, etc., based on the coordinates.

[0082] The location information can be one or more of the above forms, and the quantity can be one or more.

[0083] The terminal annotation program is an annotation program installed on the terminal to provide verification annotation methods. Depending on the terminal platform, the specific form of the terminal annotation program client can also vary; for example, the application client can be a mobile client, a PC client, or a World Wide Web (3W) client, etc. This disclosure does not impose any limitations on this.

[0084] In step S230, an inspection labeling result is generated in response to the inspection labeling operation performed on the machine inspection scan image by comparing the inspection item and location information.

[0085] In some embodiments of this disclosure, an inspection annotation operation can be performed on the machine inspection scan image based on the result of comparing it with the actual inspection item to generate an inspection annotation result.

[0086] In step S230, the verification and labeling results are sent to the server.

[0087] This application implements the verification and annotation method on the terminal side, thereby simplifying the collection of annotation data and reducing the cost of manual annotation without affecting the progress of the verification work; at the same time, it increases the accuracy of annotation and improves the model accuracy and model training efficiency of related models using annotation data as training samples.

[0088] Furthermore, the method described in this application performs the verification and labeling task via a terminal, making it convenient for on-site personnel to perform practical operations, and offering excellent portability and feasibility. Moreover, by comparing the verification items and location information, it reduces the false negative rate of actual verifications.

[0089] Furthermore, by performing targeted verification and annotation work on the identification results of predetermined categories, this application helps to improve the flexibility of the verification work, further improve the efficiency of the verification work, and enhance the inspection experience of the inspected objects.

[0090] In some embodiments of this disclosure, the location information includes a first graphic area of ​​the target item. In step S220, starting the terminal annotation program to display the location information of the target item and the machine inspection scan image includes: starting the terminal annotation program to display the machine inspection scan image and the first graphic area.

[0091] The first graphic region can be a rectangle, polygon, circle, or irregular shape. If the first graphic region is a rectangle, the position information can include the vertex coordinates and the lengths of the long and short sides; if the first graphic region is a polygon, the position information can include the set of pixel coordinates of the polygon's outline; if the first graphic region is a circle, the position information can include the center coordinates and radius.

[0092] Taking a rectangle as the first graphic region as an example, this rectangle is the smallest rectangle that completely covers the object. The smallest rectangle of the target object can be determined as the first graphic region by processing the set of pixel coordinates using the moment estimation algorithm. Alternatively, any fitting method can be used to fit the set of pixel coordinates of the polygonal outline into the shape of a rectangle, circle, or other regular shape. This disclosure does not limit the fitting method.

[0093] Figure 3A The flowchart illustrates a method for generating verification labeling results by a terminal performing a verification labeling operation in some embodiments of this disclosure. Figure 3B This diagram illustrates an interface diagram showing the generation of verification labeling results by a terminal performing a verification labeling operation in some embodiments of this disclosure.

[0094] like Figure 3A As shown, the method 300 includes the following steps:

[0095] In step S310, in response to the selection operation of the first graphic area, the first text annotation area corresponding to the first graphic area is displayed.

[0096] In some embodiments of this disclosure, a user performs a text annotation operation on a first graphic region in a machine inspection scan image by comparing the inspection item and location information. The text annotation operation includes first selecting the first graphic region and then displaying the first text annotation region corresponding to the first graphic region.

[0097] For example, Figure 3BThe interface shown displays the first graphic regions - rectangular region 1 and rectangular region 2 - corresponding to two target items that match the predetermined category obtained by the image analysis model performing image recognition on the scanned image 350 of vehicle model 340.

[0098] Users can select rectangular area 1 or rectangular area 2 based on the on-site inspection results to input text annotation information in the first text annotation area 3601 and the first text annotation area 3602 within the corresponding annotation information input area 360. The text annotation information is an evaluation result of the machine-recognized first graphic area, and the terminal annotation program can recommend labels to the user, such as "accurate recognition" or "inaccurate recognition."

[0099] In some embodiments of this disclosure, the terminal also displays a manual annotation input field. When the user is not satisfied with the displayed selected labels, they can add manual annotations to the first text annotation area 3601 and the first text annotation area 3602 in the manual annotation input field to generate the first text annotation.

[0100] In step S320, in response to a first input operation on the first text annotation area, a first text annotation corresponding to the first input operation is determined.

[0101] For example Figure 3B As shown, in response to the user's input operation on the first text annotation area 3601, the first text annotation "accurately recognized" is generated.

[0102] In step S330, a first verification annotation result is generated based on the first text annotation.

[0103] For example Figure 3B As shown, the text annotation results 3601 and 3602 input by the user for the first graphic area - rectangular area 1 and rectangular area 2 are used as the first verification annotation results.

[0104] By performing text annotation on the first graphic area, the recognition results of the image analysis model can be manually verified while the actual inspection is being conducted, which simplifies the annotation work and ensures the quality of the annotation.

[0105] Furthermore, this method embodiment can accurately identify the presence and type of prohibited items during inspection, thereby correcting false alarms from the machine and improving the accuracy of the inspection.

[0106] Figure 4A This document illustrates a flowchart of a method for generating verification labeling results by performing a verification labeling operation on a terminal in some embodiments of the present disclosure. Figure 4BThis diagram illustrates an interface schematic of another terminal performing a verification annotation operation to generate verification annotation results, as shown in some embodiments of this disclosure.

[0107] like Figure 4A As shown, the method may include the following steps:

[0108] In step S410, a second graphic region is generated in response to the graphic annotation operation on the machine inspection scan image.

[0109] In some embodiments of this disclosure, the second graphic region is a graphic annotation result generated by the user through comparing and inspecting the item and location information on the machine inspection scan image, using any graphic region such as a polygon, rectangle, or circle. The second graphic region can be one or more.

[0110] For example Figure 4B The interface shown not only displays the first graphic regions—rectangular region 1 and rectangular region 2—that match the predetermined category of two target items obtained by the image analysis model performing image recognition on the scanned image 460 of car model 450, but also displays the two second graphic regions—rectangular region 3 and rectangular region 4—that the user obtains by performing graphic inspection and annotation operations on the machine inspection scanned image by comparing and inspecting the item and location information.

[0111] Before step S410, the method also includes a first verification annotation result generated by the user based on the first graphic region. For example, in Figure 4B On the interface shown, the user can select rectangular area 1 or rectangular area 2 according to the on-site inspection results to execute the above method 300, thereby realizing the input of the text annotation information "Accurate Recognition" in the first text annotation area 4701 and "Inaccurate Recognition" in the first text annotation area 4702 in the corresponding annotation information input area 470.

[0112] In step S420, in response to the selection operation of the second graphic area, the second text annotation area corresponding to the second graphic area is displayed.

[0113] In some embodiments of this disclosure, a user performs a text annotation operation on a second graphic area. The text annotation operation includes first selecting the second graphic area, and then displaying a second text annotation area corresponding to the second graphic area.

[0114] In step S430, in response to a second input operation on the second text annotation area, a second text annotation corresponding to the second input operation is determined.

[0115] For example, Figure 4BAs shown in the interface, users can select rectangular area 3 or rectangular area 4 based on the on-site inspection results to input text annotation information in the second text annotation area 4703 and the second text annotation area 4704 within the corresponding annotation information input area 470. The text annotation information is an evaluation result of the second graphic area input by the user, and the terminal annotation program can recommend labels to the user, such as categories of items that the machine missed reporting.

[0116] In some embodiments of this disclosure, the terminal also displays a manual annotation input field. When the user is not satisfied with the displayed selected labels, they can add manual annotations to the second text annotation areas 4703 and 4704 in the manual annotation input field to generate the second text annotation.

[0117] In step S440, a second verification annotation result is generated based on the second graphic area and the second text annotation.

[0118] For example Figure 4B As shown, the text annotation results “missed report - knife” and “missed report - wine bottle” entered by the user for the second graphic area - rectangular area 3 and rectangular area 4 are used as the second verification annotation results.

[0119] The above operations can supplement the labeling of items missed by the machine, thereby increasing the accuracy of the inspection. Furthermore, using the missed data as labeled data to form feedback information not only makes the labeling task more accurate, but also helps to optimize the image analysis model, further improving the accuracy of subsequent inspection work.

[0120] In some embodiments of this disclosure, the graphic annotation operation of the machine inspection scan image in step S410 may include a correction operation of a first graphic region of the machine inspection scan image; and / or closed trajectory information input on the machine inspection scan image.

[0121] The correction operation for the first graphic region of the machine inspection scan image includes: scaling the first graphic region to make it fit the object better, thereby generating a second graphic region.

[0122] The closed trajectory information input on the scanned image during machine inspection can be any regular or irregular shape such as a polygon, rectangle, or circle.

[0123] This method embodiment can correct annotation errors and / or missing annotation information in the annotation results, thereby ensuring the accuracy of the annotation results. Furthermore, by scaling the first graphic area, it is possible to quickly locate the prohibited item area, especially the accuracy of quickly locating the extremely small area of ​​the prohibited item.

[0124] In some embodiments of this disclosure, the method may further include displaying the boundary of the first graphic region with a first line style and displaying the boundary of the second graphic region with a second line style. For example, the first line style and the second line style may be solid lines and dashed lines, respectively, or relatively thick and thin straight lines, respectively, or they may be distinguished by different colors, so that the image analysis model can determine machine annotation and manual annotation by recognizing different line styles.

[0125] This method embodiment can intuitively show users the situation of model labeled data and manually labeled data, and can also improve the effect of model optimization.

[0126] In some embodiments of this disclosure, the first line style and the second line style can also be the same line, and the image analysis model distinguishes between machine annotation and manual annotation through text annotation.

[0127] In some embodiments of this disclosure, the terminal annotation program is a client application based on Three.js and UniAPP. Figure 5 For example, Figure 5 The diagram illustrates a method for displaying the location information of a target item based on the Uniapp framework and Three.js, according to some embodiments of this disclosure. Figure 5 As shown, the method may include the following steps:

[0128] In step S510, communication is established between the Render.js component of the view layer 500a and the js component of the logic layer 500b in the Uni-app framework to provide high-performance view interaction capabilities.

[0129] In step S512, after the page is loaded, the JavaScript of the logic layer 500b calls the server interface to obtain the machine inspection scan image information.

[0130] In step S514, the logic layer 500b sends an image rendering command to the view layer 500a.

[0131] In step S516, the view layer 500a receives the image information, calls the API method of Three.js to initialize the scene, and after the scene initialization is completed, calls the camera, geometry and canvas of Three.js to complete the loading of the canvas.

[0132] In step S518, after the Canvas of view layer 500a is loaded, the track controller, drawing events, and mobile gesture events are initialized through the API methods of Three.js.

[0133] In step S520, the view layer 500a calculates the aspect ratio of the image based on the image information, calls Three.js to create a two-dimensional planar geometry, and renders the image using methods such as textures and shaders.

[0134] In step S522, the view layer 500a sends the instruction after the image rendering is completed to the logic layer.

[0135] In step S524, logic layer 500b receives a message indicating that the rendering of the machine inspection image is complete.

[0136] In step S526, logic layer 500b calls the server interface to obtain the location information of the target item.

[0137] In step S528, the logic layer 500b sends the drawing position information instruction and position information to the view layer 500a.

[0138] In step S530, the view layer 500a receives the location information of the target item, traverses all location information in sequence, and obtains the location information of a single location area.

[0139] In step S532, the view layer 500a calculates the position coordinates of the position information in the image using a relevant algorithm based on the single position information.

[0140] In step S534, the view layer 500a uses the Three.js API methods such as materials and geometry to draw the location information of the target item based on the location coordinates of the location information, and creates a text sprite as a unique identifier for the location information.

[0141] In step S536, the view layer 500a completes the drawing and display of the location information on the page.

[0142] The use of Three.js and uniapp client applications makes this method compatible and scalable, meets the complex and ever-changing scenarios of verification, and greatly reduces the cost of the verification and labeling system.

[0143] Figure 6 A schematic diagram of an inspection and labeling system is shown in some embodiments of this disclosure. For example... Figure 6 As shown, the inspection and labeling system 600 includes: a scanning device 600a, an inspection device 600b, and a terminal 600c.

[0144] In this process, the scanning device 600a performs step S602, scanning the items to be inspected and generating a machine inspection scan image.

[0145] The inspection device 600b is connected to the scanning device and is used to perform step S604 and acquire the machine inspection scan image of the inspection item.

[0146] The inspection device 600b is used to perform step S606, which involves identifying the machine inspection scan image according to the image analysis model to obtain the location information and category information of the target item in the machine inspection scan image.

[0147] The inspection device 600b is used to perform step S608: if the category information corresponding to the target item matches the predetermined category, the location information corresponding to the target item and the machine inspection scan image are sent to the terminal so that the machine inspection scan image can be inspected and labeled by comparing the inspection item and the location information.

[0148] Terminal 600c is connected to the inspection device. Terminal 600c is used to perform step S610, receive the machine inspection scan image sent by the inspection device and the location information corresponding to the target item identified according to the image analysis model, wherein the category information of the target item matches the predetermined category.

[0149] Terminal 600c is used to execute step S612, and in response to receiving the location information and machine inspection scan image corresponding to the target item, start the terminal annotation program to display the location information and machine inspection scan image corresponding to the target item.

[0150] Terminal 600c is used to execute step S614 and generate verification labeling results in response to the executed verification labeling operation.

[0151] Terminal 600c is used to execute step S616 and send the inspection labeling results to the inspection device.

[0152] The inspection device 600b is also used to perform step S618 and receive the inspection labeling result obtained by the terminal 600c based on the inspection labeling operation.

[0153] This application adds a terminal image annotation device to the existing inspection system to increase the terminal image annotation step, thereby simplifying the collection of annotation data and reducing the cost of manual annotation without affecting the normal inspection work, thus shortening the training time of the image analysis model; at the same time, it increases the accuracy of annotation and improves the model accuracy of related models using the annotation data as training samples.

[0154] In some embodiments of this disclosure, the verification and annotation system 600 further includes a model optimization device. The model optimization device is connected to the verification device and is used to obtain verification and annotation results from the verification device and optimize the image analysis model based on the verification and annotation results.

[0155] Furthermore, by using real-time annotation as feedback data to optimize the model in real time, the real-time accuracy of the system's map interpretation can be improved. On the one hand, this reduces the false negative rate during the inspection process, especially in inspection scenarios where annotation tasks are triggered based on intelligent map review results. On the other hand, improving accuracy can also reduce the false alarm rate of intelligent map review devices, thereby reducing the waste of human and material resources.

[0156] Furthermore, the verification and labeling system disclosed in this application not only improves the accuracy of verification but also accelerates the system optimization process, providing a new type of verification system with high reliability and high iterative capability for verification work.

[0157] Figure 7 A schematic diagram of an inspection device is shown in some embodiments of this disclosure. For example... Figure 7 As shown, the device 700 includes:

[0158] The image acquisition unit 710 is used to acquire machine inspection scan images of the items to be inspected; the inspection unit 720 is used to perform recognition on the machine inspection scan images according to the image analysis model to obtain the location information and category information corresponding to the target items in the machine inspection scan images; the first sending unit 730 is used to send the location information corresponding to the target items and the machine inspection scan images to the target terminal if the category information corresponding to the target items matches a predetermined category so as to perform inspection annotation operations on the machine inspection scan images by comparing the items to be inspected and the location information; and the annotation result acquisition unit 740 is used to receive the inspection annotation results obtained by the target terminal according to the inspection annotation operations.

[0159] In some embodiments of this disclosure, the apparatus 700 may further include a second sending unit for sending the verification and annotation results to the model optimization apparatus to optimize the image analysis model based on the verification and annotation results.

[0160] Figure 8 A schematic diagram of a verification labeling terminal is shown in some embodiments of this disclosure. For example... Figure 8 As shown, terminal 800 includes:

[0161] The receiving unit 810 is used to receive the machine inspection scan image sent by the server and the location information corresponding to the target item identified by the image analysis model, wherein the category information of the target item matches a predetermined category; the display unit 820 is used to start the terminal annotation program to display the location information corresponding to the target item and the machine inspection scan image in response to receiving the location information corresponding to the target item and the machine inspection scan image; the annotation unit 830 is used to generate the verification annotation result in response to the verification annotation operation performed on the machine inspection scan image; and the annotation result sending unit 840 is used to send the verification annotation result to the server.

[0162] In some embodiments of this disclosure, if the location information includes a first graphic area of ​​the target item, the display unit 820 is also used to initiate a terminal annotation program to display the machine's inspection scan image and the first graphic area.

[0163] In some embodiments of this disclosure, if the verification annotation result includes a first verification annotation result, then the annotation unit 830 may further include a first text annotation area display unit, used to display a first text annotation area corresponding to the first graphic area in response to a selection operation on the first graphic area; a first text annotation input unit, used to determine a first text annotation corresponding to the first input operation in response to a first input operation on the first text annotation area; and a first verification annotation result generation unit, used to generate a first verification annotation result based on the first text annotation.

[0164] In some embodiments of this disclosure, the verification annotation result further includes a second verification annotation result. Then, a second graphic region generation unit is used to generate a second graphic region in response to a graphic annotation operation on the machine-inspected scanned image; a second text annotation region display unit is used to display a second text annotation region corresponding to the second graphic region in response to a selection operation on the second graphic region; a second text annotation input unit is used to determine a second text annotation corresponding to the second input operation in response to a second input operation on the second text annotation region; and a second verification annotation result generation unit is used to generate a second verification annotation result based on the second graphic region and the second text annotation.

[0165] In some embodiments of this disclosure, the graphic annotation operation of the machine inspection scan image includes: a correction operation of a first graphic region of the machine inspection scan image; and / or closed trajectory information input on the machine inspection scan image.

[0166] In some embodiments of this disclosure, the device further includes a first line display unit for displaying the region boundary of the first graphic region in a first line style; and a second line display unit for displaying the region boundary of the second graphic region in a second line style.

[0167] In some embodiments of this disclosure, the terminal annotation program is a client application based on Three.js and UniAPP.

[0168] Regarding the inspection device and inspection labeling terminal in the above embodiments, the specific manner in which each unit performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated here.

[0169] It should be noted that the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of the present invention, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may, for example, be executed synchronously or asynchronously in multiple modules.

[0170] Those skilled in the art will understand that various aspects of the present invention can be implemented as systems, methods, or program products. Therefore, various aspects of the present invention can be specifically implemented in the following forms: entirely hardware implementations, entirely software implementations (including firmware, microcode, etc.), or implementations combining hardware and software aspects, collectively referred to herein as “circuits,” “modules,” or “systems.”

[0171] The following reference Figure 9 To describe an electronic device 900 according to this embodiment of the present invention. Figure 9 The electronic device 900 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0172] like Figure 9 As shown, the electronic device 900 is manifested in the form of a general-purpose computing device. The components of the electronic device 900 may include, but are not limited to: at least one processing unit 910, at least one storage unit 920, and a bus 930 connecting different system components (including the storage unit 920 and the processing unit 910).

[0173] The storage unit stores program code that can be executed by the processing unit 910, causing the processing unit 910 to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of the present invention.

[0174] For example, the processing unit 910 can perform actions such as Figure 1 As shown in the diagram, S110 acquires the machine inspection scan image of the item to be inspected; S120 performs recognition on the machine inspection scan image according to the image analysis model to obtain the location information and category information corresponding to the target item in the machine inspection scan image; S130, if the category information corresponding to the target item matches a predetermined category, the location information corresponding to the target item and the machine inspection scan image are sent to the target terminal so that the machine inspection scan image can be marked by comparing the item to be inspected and the location information; and S140 receives the inspection marking result obtained by the target terminal according to the inspection marking operation.

[0175] For example, the processing unit 910 can also perform actions such as Figure 2S210, as shown, receives the machine inspection scan image sent by the server and the location information corresponding to the target item obtained by identifying the machine inspection scan image according to the image analysis model, wherein the category information of the target item matches a predetermined category; S220, in response to receiving the location information corresponding to the target item and the machine inspection scan image, starts the terminal annotation program to display the location information of the target item and the machine inspection scan image; S230, in response to the inspection annotation operation performed on the machine inspection scan image by comparing the inspection item and the location information, generates an inspection annotation result; and S240, sends the inspection annotation result to the server.

[0176] Storage unit 920 may include readable media in the form of volatile storage units, such as random access memory (RAM) 9201 and / or cache memory 9202, and may further include read-only memory (ROM) 9203.

[0177] Storage unit 920 may also include a program / utility 9204 having a set (at least one) program module 9205, such program module 9205 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0178] Bus 930 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0179] Electronic device 900 can also communicate with one or more external devices (e.g., keyboard, pointing device, Bluetooth device, etc.), one or more devices that enable a user to interact with electronic device 900, and / or any device that enables electronic device 900 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 950. Furthermore, electronic device 900 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 960. As shown, network adapter 960 communicates with other modules of electronic device 900 via bus 930. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 900, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0180] In exemplary embodiments of this disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the methods described above is stored. In some possible embodiments, various aspects of the invention may also be implemented as a program product comprising program code that, when the program product is run on a terminal device, causes the terminal device to perform the steps of the various exemplary embodiments of the invention described in the "Exemplary Methods" section above.

[0181] According to embodiments of the present invention, a program product for implementing the above-described method may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, a readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0182] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0183] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0184] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0185] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0186] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0187] Furthermore, although the steps of the method in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.

[0188] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, mobile terminal, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0189] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.

Claims

1. An inspection method, characterized in that, The method, applied to a server performing verification and annotation, adds an annotation step to the verification task to complete the annotation work within the machine verification process; wherein, the method includes: Acquire machine inspection scan images generated by the scanning device scanning the items to be inspected during the machine inspection process; Based on the image analysis model, the machine inspection scan image is identified to obtain the location information and category information of the target item in the machine inspection scan image. The location information includes the first graphic region of the target item. If the category information corresponding to the target item matches a predetermined category, the location information corresponding to the target item and the machine inspection scan image are sent to the target terminal. The target terminal is equipped with a terminal annotation program for providing an inspection annotation method, so that the target terminal can activate the terminal annotation program to display the machine inspection scan image and the first graphic area. On-site personnel perform an inspection annotation operation on the machine inspection scan image by comparing the on-site inspection results of the item with the location information. This includes: the user determining a first inspection annotation result based on the on-site inspection results of the item, indicating whether the first graphic area is accurately or inaccurately identified; scaling the first graphic area to generate a second graphic area that better fits the item; the user inputting the second graphic area onto the machine inspection scan image based on the on-site inspection results of the item to determine a second inspection annotation result including item types missed by the machine; and... The system receives the verification labeling result obtained by the target terminal based on the verification labeling operation, wherein the verification labeling result includes the first verification labeling result and the second verification labeling result.

2. The inspection method according to claim 1, characterized in that, The method further includes: The verification and annotation results are sent to the model optimization device to optimize the image analysis model based on the verification and annotation results.

3. A method for verifying labeling, characterized in that, The method is applied to a terminal equipped with a terminal annotation program that provides an inspection annotation method. The method adds an annotation step to the inspection task to complete the annotation work within the machine inspection process; wherein the method includes: The system receives machine inspection scan images sent by a server for performing inspection and labeling, as well as location information corresponding to the target item identified by the machine inspection scan images based on an image analysis model. The category information of the target item matches a predetermined category, and the location information includes a first graphic area of ​​the target item. The machine inspection scan image is generated by the scanning device in the machine inspection process and sent to the server. In response to receiving the location information and machine inspection scan image corresponding to the target item, the terminal annotation program is started to display the first graphic area of ​​the target item and the machine inspection scan image; In response to on-site personnel performing an inspection annotation operation on the machine inspection scan image by comparing the on-site inspection results of the inspected items with the location information, an inspection annotation result is generated, which includes: a user determining a first inspection annotation result based on the on-site inspection results of the inspected items, indicating whether the first graphic region is accurately or inaccurately identified; scaling the first graphic region to generate a second graphic region that better fits the item; the user inputting the second graphic region onto the machine inspection scan image based on the on-site inspection results of the inspected items to determine a second inspection annotation result including item types missed by the machine; and The verification and labeling results are sent to the server, and the verification and labeling results include the first verification and labeling results and the second verification and labeling results.

4. The inspection and labeling method according to claim 3, characterized in that, The generation of inspection annotation results in response to the on-site personnel's inspection annotation operation on the machine inspection scan image by comparing the on-site inspection results of the inspected items with the location information includes: In response to the selection operation of the first graphic area, the first text annotation area corresponding to the first graphic area is displayed; In response to a first input operation on the first text annotation area, determine the first text annotation corresponding to the first input operation; as well as The first verification annotation result is generated based on the first text annotation.

5. The inspection and labeling method according to claim 4, characterized in that, The process of generating inspection annotation results in response to on-site personnel performing an inspection annotation operation on the machine inspection scan image by comparing the on-site inspection results of the inspected items with the location information further includes: In response to the graphic annotation operation on the machine inspection scan image, a second graphic region is generated; In response to the selection of the second graphic area, the second text annotation area corresponding to the second graphic area is displayed; In response to a second input operation on the second text annotation area, determine the second text annotation corresponding to the second input operation; as well as The second verification annotation result is generated based on the second graphic area and the second text annotation.

6. The inspection and labeling method according to claim 5, characterized in that, The graphic annotation operation for the machine inspection scan image includes: Correction operations on the first graphic region of the machine inspection scan image; and / or The closed trajectory information input on the scanned image of the machine inspection.

7. The verification and labeling method according to claim 6, characterized in that, The method includes: The boundary of the first graphic region is displayed using the first line style; The boundary of the second graphic area is displayed using the second line style.

8. The inspection and labeling method according to claim 3, characterized in that, The terminal annotation program is a client application based on Three.js and UniAPP.

9. An inspection and labeling system, characterized in that, The system is used to add an annotation step to the inspection task, so as to complete the annotation work in the machine inspection process; wherein, the system includes a scanning device, an inspection device, and a terminal, and the terminal is installed with a terminal annotation program for providing inspection annotation methods; wherein, The scanning device scans the items to be inspected and generates machine inspection scan images; The inspection device is connected to the scanning device. The inspection device is used to acquire the machine inspection scan image of the item to be inspected in the machine inspection process; to perform recognition on the machine inspection scan image according to the image analysis model to obtain the location information and category information corresponding to the target item in the machine inspection scan image, wherein the location information includes a first graphic area of ​​the target item; if the category information corresponding to the target item matches a predetermined category, the location information corresponding to the target item and the machine inspection scan image are sent to the terminal so that the machine inspection scan image can be marked by comparing the item to be inspected and the location information. The terminal is connected to the inspection device. The terminal receives the machine inspection scan image sent by the inspection device and the location information corresponding to the target item identified according to the image analysis model. In response to receiving the location information corresponding to the target item and the machine inspection scan image, the terminal starts a labeling program to display the first graphic area of ​​the target item and the machine inspection scan image. In response to on-site personnel performing an inspection labeling operation on the machine inspection scan image by comparing the on-site inspection results of the inspected item with the location information, an inspection labeling result is generated, including: a user determining a first inspection labeling result based on the on-site inspection results of the inspected item whether the first graphic area is accurately identified or inaccurately identified; scaling the first graphic area to generate a second graphic area that better fits the item; the user inputting the second graphic area onto the machine inspection scan image based on the on-site inspection results of the inspected item to determine a second inspection labeling result including item types missed by the machine; and sending the inspection labeling result to the inspection device. The verification device is further configured to receive the verification labeling result obtained by the terminal based on the verification labeling operation, wherein the verification labeling result includes the first verification labeling result and the second verification labeling result.

10. The inspection and labeling system according to claim 9, characterized in that, It also includes a model optimization device, wherein, The model optimization device is connected to the verification device, and the model optimization device is used to obtain the verification annotation results from the verification device; and optimize the image analysis model based on the verification annotation results.

11. An inspection device, characterized in that, The device is used to add a labeling step to the inspection task, so as to complete the labeling work in the machine inspection process; wherein, the device includes: The image acquisition unit is used to acquire the machine inspection scan image generated by the scanning device scanning the inspection item during the machine inspection process. The inspection unit is used to perform recognition on the machine inspection scan image according to the image analysis model to obtain the location information and category information of the target item in the machine inspection scan image, wherein the location information includes a first graphic area of ​​the target item; The first sending unit is configured to send the location information of the target item and the machine inspection scan image to a target terminal if the category information corresponding to the target item matches a predetermined category. The target terminal is equipped with a terminal annotation program that provides an inspection annotation method. The target terminal then activates the terminal annotation program to display the machine inspection scan image and the first graphic area. On-site personnel perform an inspection annotation operation on the machine inspection scan image by comparing the on-site inspection results of the item with the location information. This includes: a user determining a first inspection annotation result based on the on-site inspection results of the item, indicating whether the first graphic area is accurately or inaccurately identified; scaling the first graphic area to generate a second graphic area that better fits the item; and the user inputting the second graphic area onto the machine inspection scan image based on the on-site inspection results of the item to determine a second inspection annotation result including item types missed by the machine. The annotation result acquisition unit is used to receive the verification annotation result obtained by the target terminal according to the verification annotation operation, wherein the verification annotation result includes the first verification annotation result and the second verification annotation result.

12. A verification and labeling terminal, characterized in that, The terminal is equipped with a terminal annotation program for providing verification annotation methods. The terminal is used to add an annotation step to the verification task, thereby completing the annotation work during the machine inspection process; wherein, the terminal includes: The receiving unit is used to receive machine inspection scan images sent by the server that performs the inspection and labeling, as well as the location information of the target item identified by the image analysis model. The category information of the target item matches a predetermined category, and the location information includes a first graphic area of ​​the target item. The machine inspection scan image is generated by the scanning device in the machine inspection process and sent to the server. The display unit is used to respond to receiving the location information and machine inspection scan image corresponding to the target item, and to start the terminal annotation program to display the first graphic area and machine inspection scan image corresponding to the target item. A labeling unit is configured to generate an inspection labeling result in response to an inspection labeling operation performed by on-site personnel on the machine inspection scan image by comparing the on-site inspection results of the inspected items with the location information. The labeling unit includes: a user determining a first inspection labeling result based on the on-site inspection results of the inspected items, indicating whether the first graphic region is accurately or inaccurately identified; scaling the first graphic region to generate a second graphic region that better fits the item; and the user inputting the second graphic region onto the machine inspection scan image based on the on-site inspection results of the inspected items to determine a second inspection labeling result including item types missed by the machine. The annotation result sending unit is used to send the verification annotation result to the server, wherein the verification annotation result includes the first verification annotation result and the second verification annotation result.

13. An electronic device, characterized in that, include: processor; as well as Memory for storing the executable instructions of the processor; The processor is configured to execute the verification method of any one of claims 1 to 2 or the verification labeling method of any one of claims 3 to 8 by executing the executable instructions.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the verification method according to any one of claims 1 to 2 or the verification labeling method according to any one of claims 3 to 8.

Images