Visual verification device based on artificial intelligence object identification layout and steps

Abstract

The invention discloses a visual verification device based on artificial intelligence object identification layout and steps, and the device combines a visual camera verification module, a scene drilling module, a multi-angle image collection module, an object identification module, a multi-angle object identification result combination module, and a four-in-one visual verification module. A combination of shooting position value points, quantity, angles and camera functions is constructed, data of various entities in the sand table drilling process is effectively obtained, a basic environment is constructed for drilling, a sand table drilling recognition model is created, pictures shot by the same space-time entity through cameras at multiple angles are trained, and the training effect is improved. According to the method, entity identification data vectors at multiple angles are obtained, an entity identification model is generated, and meanwhile, a four-in-one visual verification mechanism is created, so that not only can the actual condition of a user during drilling be shown, but also the user can read more clearly in a schematic diagram mode.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a visual verification device based on artificial intelligence object recognition layout and steps. Background Art

[0002] In complex scenes, the mutual occlusion of objects makes it difficult to identify and locate multiple objects. Since the similarity between objects is inconsistent, identifying objects from different angles will affect the recognition effect. Using a single-angle image acquisition device and object recognition model cannot accurately identify and locate multiple similar targets in complex scenes. At present, the relative position of the object and the camera can be determined after the object is identified by a binocular camera, but it is impossible to visually locate the object when it is blocked. Summary of the invention

[0003] The purpose of the present invention is to provide a visual verification device based on artificial intelligence object recognition layout and steps to solve the problems raised in the above background technology.

[0004] To achieve the above-mentioned purpose, the present invention provides the following technical solutions: a visual verification device based on artificial intelligence object recognition layout and steps, including a visual camera verification module, a scene rehearsal module, a multi-angle image acquisition module, an object recognition module, a multi-angle object recognition result combination module, and a four-in-one visual verification module; The visual camera verification module is used to connect and assemble the set workbench, touch screen, and camera together; The scene rehearsal module prefabricates a variety of object recognition scenes for switching between different scene base maps. After selecting a scene, the verification rules of the corresponding scene model are automatically bound, and the objects to be recognized in the scene are placed in the scene according to the steps. The multi-angle image acquisition module is used to set the shooting angle of the object to be identified in the current scene layout, and perform shooting and acquisition in the same time period; The object recognition module is used to create a corresponding data set according to the image at each shooting angle, and use the data set to train a recognition model for the photographed object; The multi-angle object recognition result combination module is used to combine the results of the recognition training in the object recognition module to determine the consistency of the number and type of recognized objects; The four-in-one visual verification module is used to combine and verify the on-site answer photo, answer diagram, standard answer diagram, and scoring analysis table on one interface;.

[0005] Preferably, in the visual camera verification module, a touch screen is placed on a workbench, and the touch screen is used to display different scenes as an electronic sand table, which is set as a stationary object. There are several cameras, and they are placed on the front, back, left, right and top of the workbench as needed; when the system is enabled, the sand table displays any scene, the camera is turned on and aimed at the sand table, and the system displays the sand table effects captured by all cameras. The position of the camera is manually or electrically adjusted until it is clear, and the verification is completed.

[0006] Preferably, when the scene rehearsal module prefabricates the object recognition scene, based on the constructed touch screen, workbench, and camera, the different scenes displayed on the touch screen are used as the base map, and the simulated objects are placed on the display screen according to certain rules according to the scenes shown in the base map.

[0007] Preferably, the shooting angles of the multi-angle image acquisition module include top view, left side, and right side; and the images of the placed simulation objects are acquired at the same time.

[0008] Preferably, the object recognition module creates corresponding data sets for the object categories that can be judged by the images at each shooting angle, and then trains the object recognition model according to the object types in the data sets. The trained object model can recognize the categories and location information of all objects contained in the current angle.

[0009] Preferably, the multi-angle object recognition result combination module determines the consistency of the number and types of recognized objects based on the object recognition results of the multi-angle collected images.

[0010] Preferably, the steps of model training in the object recognition module are as follows: creating entity category labels appearing in the data sets for different shooting angles, using an entity recognition annotation tool to annotate the positions of the entities in the images and the entity names in the data sets, and after the annotation is completed, performing entity recognition model training on different data sets respectively. After the model training is completed, verifying the model recognition effect, analyzing the reasons if the model recognition effect is not good, and optimizing the model by increasing the data set, adjusting the model training parameters, and other means.

[0011] Preferably, in the four-in-one visual verification module, the on-site answering photos are taken by the user in the order of starting to place, completing the placing, starting to evacuate, and completing the evacuation according to the system prompts, and the user places and evacuates the objects respectively; the photos to be displayed are selected as needed, and the default selection is to use the photo of the completed placing as the default display photo, or to display the photos of the starting placing and the starting evacuation, and all the objects in the photos are marked with identifiable labels, and the first placed object and the first evacuated object are recorded at the same time; the answer schematic diagram is used to convert the content of the on-site answering photo according to the schematic diagram style of the standard answer, and the on-site answering photo is consistent with the content of the converted answer schematic diagram; the standard answer schematic diagram is used to convert the content of the on-site answering photo according to the application scenario set by the user The correct answer is presented in the form of a schematic diagram, each object and various scenes are drawn in detail and the sequence information is configured for comparison with the answer schematic diagram. The standard answer may be one or more groups; the scoring analysis table is used to analyze and display the reasons for the placement errors according to the rules, and the analysis results can be interacted with the answer schematic diagram in various styles; the four-in-one visual verification module uses the bird's-eye view shooting angle as the base map of the current scene, and combines the results in the multi-angle object recognition result combination module to confirm all object categories in the bird's-eye view, and then corrects the position recognition results of all objects according to the correction parameters of the bird's-eye view camera, and restores them to the base map according to the actual position.

[0012] Preferably, in the four-in-one visual verification module, when performing visual display, corresponding electronic tags are created according to the types of objects in the current scene, including categories of different orientations that need to be distinguished for the same object; the electronic tags are displayed on the base map of the corresponding scene to achieve visual display.

[0013] Preferably, in the four-in-one visual verification module, the specific steps of using rule verification are: according to the pre-set rules and scoring criteria, click rule verification to verify the placement results of the object recognition in the current scene. The verification rules include the placement direction of the objects, the number of objects, the spacing between the objects, the order of placement, etc., and the points where points will be deducted will be marked and displayed at the corresponding positions on the scene background map, and a one-to-one comparison will be performed to obtain the scores, and the reasons for the placement errors will be analyzed; the way to interact with the analysis results and the response schematic diagram is to click anywhere in the scoring analysis table, and the corresponding placement point will be displayed on the response schematic diagram.

[0014] Compared with the prior art, the present invention has the following beneficial effects: (1) The present invention constructs a combination of camera position value points, quantity, angle, and camera function (model) through experimental testing, effectively obtaining data of various entities during the sandbox exercise, thereby constructing a basic environment for the exercise; (2) The present invention creates a sandbox training recognition model through entity training, trains photos taken by cameras at multiple angles of the same space-time entity, obtains multi-angle entity recognition data vectors, and generates an entity recognition model; based on the spatial position of the entity in the model and the recognition results of the multi-angle photos, the position and angle of each entity are restored one by one on the response diagram, thereby improving the effect of entity recognition; (3) The present invention creates a four-in-one visual verification mechanism, which can effectively generate a response diagram automatically from a response picture. The two elements correspond one to one, which can not only show the actual situation of the user's rehearsal, but also make the user read more clearly in the form of a diagram; the response picture and the response diagram are then displayed together with the standard diagram to more clearly see the difference between the response result and the standard diagram; when the above three pictures are displayed at the same time, a scoring analysis table is added. Through the function of the scoring analysis table interacting with the above pictures, the user can know at a glance the score compliance status of this exercise and the incorrect analysis, so as to further improve the actual combat level. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a schematic diagram of the structure of the present invention. DETAILED DESCRIPTION

[0016] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Example

[0017] The present invention provides a visual verification device based on artificial intelligence object recognition layout and steps, including a visual camera verification module, a scene rehearsal module, a multi-angle image acquisition module, an object recognition module, a multi-angle object recognition result combination module, a four-in-one visual verification module, and a comprehensive evaluation and analysis module; like Figure 1As shown, the visual camera verification module assembles the workbench, touch screen, and camera together, wherein the touch screen can be a capacitive touch screen. During assembly, the capacitive touch screen is installed on a standard workbench. The touch screen can display different scenes as an electronic sand table. Then, multiple cameras are installed on the front, back, left, right, and top of the workbench as needed. The installation angle and position of the camera can be placed as needed, and when the system is enabled, the sand table can show any scene. Turn on the camera and aim it at the sand table. The system can display the sand table effects taken by all cameras. The position of the camera is manually or electrically adjusted until it is clear, and the verification is completed. This is also applicable to other display devices.

[0018] The scenario rehearsal module prefabricates a variety of object recognition scenarios through the built-in system, which can be suitable for switching between different scenario base maps. After selecting the scene, it automatically binds the verification rules of the corresponding scenario model. In actual use, the touch screen can be used to switch between different usage scenarios. Different usage scenarios use different verification rules. Therefore, according to different scenario requirements on the touch screen, the simulated objects to be identified can be placed in the scene, that is, on the touch screen according to certain rules. For example, if the scene of "signal-controlled intersection (exit section of flat intersection)" is selected on the touch screen, the road section represented by the scene will appear on the touch screen, and the staff can place the simulated objects according to the road section displayed in the scene; or the staff can switch scenes on the base map at will to perform operations.

[0019] The multi-angle image acquisition module can set the shooting angle of the object to be identified in the current scene layout; by setting the shooting angles of multiple cameras, shooting and acquisition are performed in the same time period, so as to ensure that the simulated object can be clearly seen from different angles and the image of the simulated object can be clearly acquired; when the simulated object is placed on the industrial touch screen, the position or angle of the camera can be adjusted according to the position of the simulated object, so as to ensure that the corresponding scene section on the touch screen is clearly photographed, which is convenient for later image recognition; for example, if the staff has selected the scene of "signal-controlled intersection (exit section of flat intersection)", the staff will place the simulated object on it according to the rules. In order to ensure that the image of the simulated object and the touch screen is clearly acquired, the staff can adjust the angle of the camera or install an additional camera or turn off a camera, so as to ensure that the multi-angle image acquisition of the simulated object, such as up, down, left, right, front and back, is clear.

[0020] The object recognition module creates a corresponding data set based on the images from each shooting angle, and uses the data set to train the recognition model for the photographed objects. The steps of model training are as follows: for data sets with different shooting angles, create entity category labels that appear in the data sets respectively, use entity recognition annotation tools to annotate the positions of the entities in the data sets and the entity names in the images, and after the annotation is completed, perform entity recognition model training on different data sets respectively. After the model training is completed, verify the model recognition effect. If the model recognition effect is not good, analyze the cause, and optimize the model by increasing the data set, adjusting the model training parameters, etc. The trained object model can recognize the categories and location information of all objects contained in the current angle.

[0021] The multi-angle object recognition result combination module combines the results of the recognition training in the object recognition module to determine the consistency of the number and type of recognized objects.

[0022] When the simulated object recognition is completed, the four-in-one visual verification module is used to combine the on-site response photos, response diagrams, standard answer diagrams, and scoring analysis tables on one interface for verification; the on-site response photos are taken by the user in the order of starting placement, completing placement, starting evacuation, and completing evacuation as prompted by the system; the photos to be displayed are selected as needed, usually the photos of completed placement are selected as the default display photos, or the photos of starting placement and starting evacuation are displayed. All objects in the photos are marked with recognizable labels, and the first objects placed and the first objects evacuated are recorded at the same time. For example, in the scenario of "signal-controlled intersection (exit section of flat intersection)", the staff will place the simulated objects on the touch screen according to the prompts of the system, and then the camera will take pictures of the placed simulated objects during the entire placement process. Usually, the photos of completed placement are used as on-site response photos; the response diagram refers to the conversion of the content of the on-site response photos according to the diagram style of the standard answer, and the on-site response photos and the converted The content of the subsequent answer diagram is consistent. For example, if the photo of the completed placement in the scene of "signal-controlled intersection (exit section of flat intersection)" is the on-site answer photo, the photo will be converted into an answer diagram, which is used to compare with the following standard answer diagram; the standard answer diagram refers to the correct answer presented in the form of a diagram according to the application scenario set by the user, with each object and various scenes drawn in detail and configured with sequence information, which is used to compare with the answer diagram. The standard answer may be one or more groups; when the comparison between the standard answer diagram and the answer diagram is completed, the scoring analysis table will analyze and display the reasons for the placement errors according to the items deducted according to the rules, and the analysis results can interact with the answer diagram in various styles. For example, you can use the mouse to click on a corresponding deduction item on the scoring analysis table, and the answer diagram will jump or flash or display the wrong placement in other ways, or click on the placed simulation on the answer diagram with the mouse, and the corresponding content on the scoring analysis table will be displayed.

[0023] In the process of verification using the four-in-one visual verification module, the bird's-eye view shooting angle is used as the base map of the current scene, and the results in the multi-angle object recognition result combination module are combined to confirm the categories of all objects in the bird's-eye view, and then the position recognition results of all objects are corrected according to the correction parameters of the bird's-eye view camera, and restored to the base map according to the actual position; then a visual display is performed, and corresponding electronic tags are created according to the types of objects in the current scene, including the categories of different orientations that need to be distinguished for the same object; the electronic tags are displayed on the base map of the corresponding scene to achieve visual display; the specific steps of rule verification are: according to the pre-set rules and scoring criteria, click rule verification to verify the placement results of the object recognition in the current scene. The verification rules include rules such as object placement direction, object quantity, object placement interval, and placement order, and the deduction points will be marked and displayed at the corresponding positions of the scene base map, and a one-to-one comparison is performed to obtain a score, and the cause of the placement error is analyzed, and different scenes correspond to different verification rules.

[0024] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments are to be regarded as exemplary and non-limiting in all respects, and the scope of the invention is to be defined by the appended claims rather than by the foregoing description, and it is intended that all variations falling within the meaning and range of equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be regarded as limiting the claim to which it relates.

Claims

1. A visual verification device based on artificial intelligence object recognition layout and steps, characterized by: It includes visual camera calibration module, scene rehearsal module, multi-angle image acquisition module, object recognition module, multi-angle object recognition result combination module, and four-in-one visual verification module; The visual camera verification module is used to connect and assemble the set workbench, touch screen, and camera together; The scene rehearsal module prefabricates a variety of object recognition scenes for switching between different scene base maps. After selecting a scene, the verification rules of the corresponding scene model are automatically bound, and the objects to be recognized in the scene are placed in the scene according to the steps. The multi-angle image acquisition module is used to set the shooting angle of the object to be identified in the current scene layout, and perform shooting and acquisition in the same time period; The object recognition module is used to create a corresponding data set according to the image at each shooting angle, and use the data set to train a recognition model for the photographed object; The multi-angle object recognition result combination module is used to combine the results of the recognition training in the object recognition module to determine the consistency of the number and type of recognized objects; The four-in-one visual verification module is used to combine and verify the on-site response photo, response diagram, standard answer diagram, and scoring analysis table on one interface.

2. The visual verification device based on artificial intelligence object recognition layout and steps according to claim 1 is characterized in that: In the visual camera verification module, a touch screen is placed on a workbench. The touch screen is used to display different scenes as an electronic sand table, which is set as a stationary object. There are a number of cameras, which are placed on the front, back, left, right and top of the workbench as needed. When the system is enabled, the sand table displays any scene. The camera is turned on and aimed at the sand table. The system displays the sand table effects captured by all cameras. The position of the camera is adjusted manually or electrically until it is clear, and the verification is completed.

3. The visual verification device based on artificial intelligence object recognition layout and steps according to claim 1 is characterized in that: When the scene rehearsal module prefabricates the object recognition scene, based on the built touch screen, workbench, and camera, the different scenes displayed on the touch screen are used as the base map, and the simulated objects are placed on the display screen according to certain rules according to the scenes shown in the base map.

4. The visual verification device based on artificial intelligence object recognition layout and steps according to claim 1 is characterized in that: The shooting angles of the multi-angle image acquisition module include top view, left side, and right side; and the image of the placed simulation object is acquired at the same time.

5. The visual verification device based on artificial intelligence object recognition layout and steps according to claim 1 is characterized in that: The object recognition module creates corresponding data sets for the object categories that can be judged by the images of each shooting angle, and then trains the object recognition model according to the object types in the data sets. The trained object model can recognize the categories and location information of all objects contained in the current angle.

6. The visual verification device based on artificial intelligence object recognition layout and steps according to claim 1 is characterized in that: The multi-angle object recognition result combination module determines the consistency of the number and types of recognized objects based on the object recognition results of the multi-angle collected images.

7. The visual verification device based on artificial intelligence object recognition layout and steps according to claim 1 is characterized in that: The steps of model training in the object recognition module are as follows: create entity category labels appearing in the data sets for different shooting angles, use entity recognition annotation tools to annotate the positions of the entities in the images and the entity names in the data sets, and after the annotation is completed, perform entity recognition model training on different data sets respectively. After the model training is completed, verify the model recognition effect. If the model recognition effect is not good, analyze the cause, and optimize the model by increasing the data set, adjusting the model training parameters, etc.

8. The visual verification device based on artificial intelligence object recognition layout and steps according to claim 1 is characterized in that: In the four-in-one visual verification module, the on-site response photos are taken by the user in the order of starting to place, completing the placement, starting to evacuate, and completing the evacuation according to the system prompts, and the order of placing and evacuating objects is taken respectively; the photos to be displayed are selected as needed, and the default selection is to use the photos of the completed placement as the default display photos, or to display the photos of the beginning of placement and the beginning of evacuation. All objects in the photos are marked with identifiable labels, and the first objects placed and the first objects evacuated are recorded at the same time; the response schematic diagram is used to convert the content of the on-site response photo according to the schematic diagram style of the standard answer, and the on-site response photo is consistent with the content of the converted response schematic diagram; the standard answer schematic diagram is used according to the application scenario set by the user The correct answer is presented in the form of a schematic diagram, with each object and various scenes drawn in detail and configured with sequence information, which is used to compare with the answer schematic diagram, and the standard answer is one or more groups; the scoring analysis table is used to analyze and display the reasons for the placement errors according to the rules, and the analysis results can interact with the answer schematic diagram in various styles. The four-in-one visual verification module uses the bird's-eye view shooting angle as the base map of the current scene, and combines the results in the multi-angle object recognition result combination module to confirm all object categories in the bird's-eye view, and then corrects the position recognition results of all objects according to the correction parameters of the bird's-eye view camera, and restores them to the base map according to the actual position.

9. The visual verification device based on artificial intelligence object recognition layout and steps according to claim 7 is characterized in that: In the four-in-one visual verification module, when performing visual display, corresponding electronic tags are created according to the types of objects in the current scene, including categories of different orientations that need to be distinguished for the same object; the electronic tags are displayed on the base map of the corresponding scene to achieve visual display.

10. The visual verification device based on artificial intelligence object recognition layout and steps according to claim 1, characterized in that: In the four-in-one visual verification module, the specific steps of rule verification are as follows: according to the pre-set rules and scoring criteria, click rule verification to verify the placement results of the object recognition in the current scene. The verification rules include the placement direction of objects, the number of objects, the interval distance between objects, the order of placement, etc., and mark the points where deductions will appear at the corresponding positions of the scene base map, compare them one by one and get the scores, and analyze the reasons for the placement errors; The way to interact with the analysis results and the answer diagram is to click anywhere in the score analysis table, and the corresponding placement point will be displayed on the answer diagram.

Images