Visual Verification Device for Artificial Intelligence Object Recognition Layout and Steps

Abstract

The present invention discloses a visualization verification device based on artificial intelligence object recognition layout and steps. By combining a visualization camera calibration module, a scenario rehearsal module, a multi-angle image acquisition module, an object recognition module, a multi-angle object recognition result combination module, and a four-in-one visualization verification module, a combination of camera position value points, quantity, angles, and camera functions is constructed, effectively obtaining data of various entities during the sand table rehearsal process, thereby constructing a basic environment for the rehearsal and creating a sand table rehearsal recognition model. Photos of the same spatio-temporal entity taken by cameras from multiple angles are trained to obtain multi-angle entity recognition data vectors and generate an entity recognition model. At the same time, a four-in-one visualization verification mechanism is also created, which can not only display the actual situation during the user's rehearsal but also make it clearer for the user to read in the form of schematic diagrams.

Description

Technical Field

[0001] The present invention relates to the technical field of artificial intelligence, and specifically to a visualization verification device based on artificial intelligence object recognition layout and steps. Background Art

[0002] In complex scenarios, due to the mutual occlusion of objects, it is difficult to identify and locate multiple objects; due to the inconsistent similarity between objects, identifying objects from different angles will affect the recognition effect. Using a single-angle image acquisition device and object recognition model cannot accurately identify and layout multiple similar targets in complex scenarios. And currently, the relative position of an object to a camera can be determined after object recognition by a binocular camera, but the object cannot be visually located when it is occluded. Summary of the Invention

[0003] The purpose of the present invention is to provide a visualization verification device based on artificial intelligence object recognition layout and steps to solve the problems raised in the above background art.

[0004] To achieve the above purpose, the present invention provides the following technical solution: A visualization verification device based on artificial intelligence object recognition layout and steps, including a visualization camera calibration module, a scenario rehearsal module, a multi-angle image acquisition module, an object recognition module, a multi-angle object recognition result combination module, and a four-in-one visualization verification module;

[0005] The visualization camera calibration module is used to connect and assemble a set workbench, touch screen, and camera together;

[0006] The scenario rehearsal module prefabricates various object recognition scenarios, is used for switching different scenario backgrounds, automatically binds the verification rules of the corresponding scenario model after selecting a scenario, and arranges the objects to be recognized in the scenario step by step according to the scenario;

[0007] The multi-angle image acquisition module is used to set the shooting angles of the objects to be recognized in the current scenario layout and perform shooting and acquisition in the same time period;

[0008] The object recognition module is used to create a corresponding data set according to the images of each shooting angle and use the data set to train the recognition model of the shot objects;

[0009] The multi-angle object recognition result combination module is used to combine the recognition and training results in the object recognition module to determine the consistency of the number and types of recognized objects;

[0010] The four-in-one visualization verification module is used to combine and verify on-site response photos, response schematic diagrams, standard answer schematic diagrams, and scoring analysis tables on one interface;

[0011] Preferably, in the visual camera calibration module, the touch screen is placed on the workbench. The touch screen is used to display different scenes as an electronic sand table and is set as a static object. The number of cameras is several and they are arranged around the workbench as needed, including in the front, back, left, right, and top. When the system is enabled, the sand table displays any scene. The cameras are turned on and aimed at the sand table. The system displays the effects of the sand table captured by all cameras. Manually or electrically adjust the positions of the cameras until they are clear, and then the verification is completed.

[0012] Preferably, when the scenario rehearsal module identifies the prefabricated object scenarios, based on the touch screen, workbench, and cameras that have been set up, using the different scenes displayed on the touch screen as the base map, the simulated objects are placed and rehearsed on the display screen according to the rules shown in the base map.

[0013] Preferably, the shooting angles of the multi-angle image acquisition module include top view, left side, and right side; and images of the placed simulated objects are acquired at the same time.

[0014] Preferably, the object recognition module creates corresponding data sets for the object categories that can be judged from the images of each shooting angle respectively, and then trains the object recognition models according to the object types in the data sets. The trained object models can identify the categories and position information of all objects included in the current angle.

[0015] Preferably, the multi-angle object recognition result combination module determines the consistency of the recognized object quantities and types according to the object recognition results of the multi-angle acquired images.

[0016] Preferably, the steps of the model training in the object recognition module are as follows: Create entity category labels that appear in the data sets for the data sets of different shooting angles respectively. Use the entity recognition annotation tool to annotate the positions and entity names of the entities in the images of the data sets. After the annotation is completed, train the entity recognition models for different data sets respectively. After the model training is completed, verify the model recognition effect. If the model recognition effect is not good, analyze the reasons and optimize the model by means such as increasing the data set and adjusting the model training parameters.

[0017] Preferably, in the four-in-one visual verification module, the on-site response photos are taken by the user according to the order of starting to place, finishing placing, starting to evacuate, and finishing evacuating as prompted by the system. After placing and evacuating the objects respectively, the photos are taken. The photos to be displayed are selected as needed. By default, the photo of finishing placing is selected as the default display photo, or the photos of starting to place and starting to evacuate are displayed. All the objects in the photos are marked with recognizable labels. At the same time, the first placed object and the first evacuated object are recorded. The response schematic diagram is used to convert the content of the on-site response photo according to the schematic diagram style of the standard answer. The content of the on-site response photo is consistent with that of the converted response schematic diagram. The standard answer schematic diagram is used to display the correct answer in the form of a schematic diagram according to the application scenario set by the user. Each object and various scenarios are drawn in detail and the sequence information is configured to be compared with the response schematic diagram. There may be one or more groups of standard answers. The scoring analysis table is used to analyze and display the reasons for incorrect placement according to the items for deducting points according to the rules, and can interact with the response schematic diagram in various styles with the analysis results. In the four-in-one visual verification module, the top-down shooting angle is used as the base map of the current scenario. Combining the results in the multi-angle object recognition result combination module, all object categories in the top-down view are confirmed. Then, according to the correction parameters of the top-down camera, the position recognition results of all objects are corrected and restored to the base map according to the actual positions.

[0018] Preferably, in the four-in-one visual verification module, when performing visual display, corresponding electronic tags are created according to the types of objects in the current scenario, including different orientation categories that need to be distinguished for the same object. The electronic tags are displayed on the base map of the corresponding scenario to achieve visual display.

[0019] Preferably, in the four-in-one visual verification module, the specific steps for rule verification are as follows: According to the preset rules and scoring rules, click rule verification to verify the placement results of object recognition in the current scenario. The verification rules include rules such as object placement direction, object quantity, interval distance of object placement, and front-back order of placement. The points for deduction are marked and displayed at the corresponding positions on the scenario base map, compared one by one to obtain a score, and the reasons for incorrect placement are analyzed. The way of interacting the analysis result with the response schematic diagram is to click anywhere on the scoring analysis table, and the corresponding placement points will be displayed on the response schematic diagram.

[0020] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0021] (1) The present invention constructs a combination of camera position value points, quantity, angle, and camera function (model) through experimental tests, effectively obtains various entity data during the sand table drill, and thus constructs a basic environment for the drill;

[0022] (2) The present invention creates a sand table drill recognition model through entity training. Photos of the same spatio-temporal entity taken by cameras from multiple angles are trained to obtain entity recognition data vectors from multiple angles, and an entity recognition model is generated. According to the spatial position of the entity in the model and the recognition results of the multi-angle photos, the position and angle of each entity are restored one by one on the response schematic diagram, thereby improving the effect of entity recognition.

[0023] (3) The present invention creates a four-in-one visual verification mechanism, which can effectively automatically generate a response schematic diagram from the response picture, with the elements of the two corresponding one by one. It can not only show the actual situation during the user's drill, but also make it clearer for the user to read in the form of a schematic diagram. When the response picture and the response schematic diagram are displayed simultaneously with the standard schematic diagram, the gap between the response result and the standard schematic diagram can be seen more clearly. When the above three pictures are displayed simultaneously, together with the scoring analysis table, through the function of the interaction between the scoring analysis table and the above pictures, the user can clearly know whether the score of this drill meets the standard and the analysis of the errors, so as to further improve the actual combat level. Description of the Drawings

[0024] Figure 1 It is a schematic structural diagram of the present invention. Detailed Embodiments

[0025] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of the present invention. Embodiment

[0026] The present invention provides a visual verification device based on artificial intelligence object recognition layout and steps, including a visual camera calibration module, a scenario drill module, a multi-angle image acquisition module, an object recognition module, a multi-angle object recognition result combination module, a four-in-one visual verification module, and a comprehensive evaluation and analysis module.

[0027] Such as Figure 1As shown, the visualization camera calibration module assembles the set workbench, touch screen, and camera together. The touch screen can be a capacitive touch screen. During assembly, the capacitive touch screen is installed on the standard workbench. The touch screen can display different scenarios as an electronic sand table. Then, multiple cameras are installed on the front, back, left, right, and top of the workbench as needed. The installation angles and positions of the cameras can be arranged as required. When the system is enabled, the sand table can display any scenario. Turn on the camera and aim it at the sand table. The system can display the sand table effects captured by all cameras. Manually or electrically adjust the position of the camera until it is clear, and the verification is completed. This also applies to other display devices.

[0028] The scenario rehearsal module prefabricates various object recognition scenarios through the built-in system, which can be suitable for the switching of different scenario backgrounds. After selecting a scenario, the verification rules corresponding to the scenario model are automatically bound. In actual use, the touch screen can be used to switch different usage scenarios, and different verification rules are used for different usage scenarios. Thus, according to different scenario requirements on the touch screen, the simulated objects to be recognized can be placed on the touch screen, i.e., in the scenario, according to certain rules as needed. For example, when the scenario of "signal-controlled intersection (exit section of plane intersection)" is selected on the touch screen, the road section represented by this scenario will appear on the touch screen, and the staff can place the simulated objects according to the road section shown in this scenario; or the staff can also arbitrarily switch scenarios on the background map for operation.

[0029] The multi-angle image acquisition module can set the shooting angles of the objects to be recognized in the current scenario layout; by setting the shooting angles of multiple cameras and performing shooting and acquisition in the same time period, it is ensured that the simulated objects can be clearly seen from different angles and the images of the simulated objects are clearly acquired; when the simulated objects are placed on the work touch screen, the position or angle of the camera can be adjusted according to the position of the simulated objects, so as to ensure that the shooting on the corresponding scenario road section on the touch screen is clear and facilitate subsequent image recognition; for example, when the staff has selected the scenario of "signal-controlled intersection (exit section of plane intersection)", the staff places the simulated objects on it according to the rules. To ensure the clear image acquisition of the simulated objects and the touch screen, the staff can adjust the angle of the camera, add a camera, or turn off a certain camera, so as to ensure the clear multi-angle image acquisition of the simulated objects from top, bottom, left, right, front, and back, etc.

[0030] The object recognition module creates corresponding datasets based on the images taken from each shooting angle, and uses the datasets to train the recognition model for the photographed objects. The steps of model training are as follows: entity category labels that appear in the datasets are created respectively for the datasets with different shooting angles, and the entity recognition annotation tool is used to annotate the positions and entity names of the entities in the images of the datasets. After the annotation is completed, the entity recognition models are trained for different datasets respectively. After the model training is completed, the recognition effect of the model is verified. If the recognition effect of the model is not good, the reasons are analyzed, and the model is optimized by means of increasing the datasets, adjusting the model training parameters, etc. The trained object model can recognize the category and position information of all the objects included in the current angle.

[0031] The multi-angle object recognition result combination module combines the recognition and training results in the object recognition module to determine the consistency of the number and types of recognized objects.

[0032] After the simulation of object recognition is completed, the four-in-one visualization verification module is used to combine and verify the on-site response photo, response schematic diagram, standard answer schematic diagram, and scoring analysis form on one interface; among them, the on-site response photo is taken by the user according to the order of starting to place, finishing placing, starting to evacuate, and finishing evacuating as prompted by the system; the photos to be displayed are selected as needed. Usually, the photo of finishing placing is selected as the default display photo, or the photos of starting to place and starting to evacuate are displayed. All the objects in the photos are marked with recognizable labels, and at the same time, the first object to be placed and the first object to be evacuated are recorded. For example, in the scenario of "signal-controlled intersection (exit section of plane intersection)", the staff places the simulated objects on the touch screen according to the system's prompts, and then the camera will take pictures of the placed simulated objects during the whole placing process. Usually, the photo of finishing placing is used as the on-site response photo; the response schematic diagram refers to converting the content of the on-site response photo according to the style of the standard answer schematic diagram, and the content of the on-site response photo is consistent with that of the converted response schematic diagram. For example, if the photo of finishing placing in the scenario of "signal-controlled intersection (exit section of plane intersection)" is the on-site response photo, then this photo is converted into a response schematic diagram, which is used to compare with the following standard answer schematic diagram; the standard answer schematic diagram refers to presenting the correct answer in the form of a schematic diagram according to the application scenario set by the user, drawing each object and various scenarios in detail and configuring the sequence information, which is used to compare with the response schematic diagram. There may be one or more groups of standard answers; after the comparison between the standard answer schematic diagram and the response schematic diagram is completed, the scoring analysis form analyzes and displays the reasons for the incorrect placement according to the items of deducting points in the rules, and can interact with the response schematic diagram in various styles. For example, if a corresponding item of deducting points on the scoring analysis form is clicked with the mouse, the incorrect placement will be displayed by jumping, flashing, or other means on the response schematic diagram, or if the placed simulated object is clicked with the mouse on the response schematic diagram, the corresponding content on the scoring analysis form will be displayed.

[0033] In the process of verification using the four-in-one visualization verification module, the top-down shooting angle is used as the base map of the current scene. Combining the results in the multi-angle object recognition result combination module, all object categories in the top-down view are confirmed. Then, according to the calibration parameters of the top-down camera, the position recognition results of all objects are corrected and restored to the base map according to the actual positions. Then, for visualization display, according to the object types in the current scene, corresponding electronic tags are created for different orientations of the same object that need to be distinguished. The electronic tags are displayed on the base map of the corresponding scene to achieve visualization display. The specific steps for rule verification are as follows: According to the pre-set rules and scoring criteria, click on rule verification to verify the placement results of the object recognition in the current scene. The verification rules include rules such as the object placement direction, the number of objects, the spacing distance between object placements, and the front-back order of placements. And the points deducted are marked and displayed at the corresponding positions on the scene base map, compared one by one to obtain a score, analyze the reasons for the placement errors, and different scenes correspond to different verification rules.

[0034] For those skilled in the art, it is obvious that the present invention is not limited to the details of the above exemplary embodiments, and without departing from the spirit or basic characteristics of the present invention, the present invention can be implemented in other specific forms. Therefore, from any point of view, the embodiments are regarded as exemplary and non-limiting. The scope of the present invention is defined by the appended claims rather than the above description. Therefore, all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be embraced within the present invention. Any reference signs in the claims should not be regarded as limiting the claimed rights.

Claims

1. A visual verification device based on artificial intelligence object recognition layout and steps, characterized by: It includes visual camera calibration module, scene rehearsal module, multi-angle image acquisition module, object recognition module, multi-angle object recognition result combination module, and four-in-one visual verification module; The visual camera verification module is used to connect and assemble the set workbench, touch screen, and camera together; The scene rehearsal module prefabricates a variety of object recognition scenes for switching between different scene base maps. After selecting a scene, the verification rules of the corresponding scene model are automatically bound, and the objects to be recognized in the scene are placed in the scene according to the steps. The multi-angle image acquisition module is used to set the shooting angle of the object to be identified in the current scene layout, and perform shooting and acquisition in the same time period; The object recognition module is used to create a corresponding data set according to the image at each shooting angle, and use the data set to train a recognition model for the photographed object; The multi-angle object recognition result combination module is used to combine the results of the recognition training in the object recognition module to determine the consistency of the number and type of recognized objects; The four-in-one visual verification module is used to combine and verify the on-site response photos, response schematics, standard answer schematics, and scoring analysis tables on one interface; in the four-in-one visual verification module, the on-site response photos are taken by the user in the order of starting to place, completing placement, starting to evacuate, and completing evacuation according to the system prompts; the photos to be displayed are selected as needed, and the default selection is to use the photos of completed placement as the default display photos, or to display the photos of starting to place and starting to evacuate. All objects in the photos are marked with identifiable labels, and the first objects placed and the first objects evacuated are recorded at the same time; the response schematic is used to convert the content of the on-site response photos according to the schematic style of the standard answer, and the on-site response photos and the converted response schematics are compared. The content is consistent; the standard answer schematic diagram is used to display the correct answer in the form of a schematic diagram according to the application scenario set by the user, draw each object and various scenes in detail and configure the sequence information for comparison with the answer schematic diagram, and the standard answer is one or more groups; the scoring analysis table is used to analyze and display the reasons for the placement errors according to the items that are deducted according to the rules, and can interact with the analysis results and the answer schematic diagram in various styles; the four-in-one visual verification module uses the bird's-eye view shooting angle as the base map of the current scene, and combines the results in the multi-angle object recognition result combination module to confirm all object categories in the bird's-eye view, and then corrects the position recognition results of all objects according to the correction parameters of the bird's-eye view camera, and restores them to the base map according to the actual position.

2. The visual verification device based on artificial intelligence object recognition layout and steps according to claim 1 is characterized in that: In the visual camera verification module, a touch screen is placed on a workbench. The touch screen is used to display different scenes as an electronic sand table, which is set as a stationary object. There are a number of cameras, which are placed on the front, back, left, right and top of the workbench as needed. When the system is enabled, the sand table displays any scene. The camera is turned on and aimed at the sand table. The system displays the sand table effects captured by all cameras. The position of the camera is adjusted manually or electrically until it is clear, and the verification is completed.

3. The visual verification device based on artificial intelligence object recognition layout and steps according to claim 1 is characterized in that: When the scene rehearsal module prefabricates the object recognition scene, based on the built touch screen, workbench, and camera, the different scenes displayed on the touch screen are used as the base map, and the simulated objects are placed on the display screen according to certain rules according to the scenes shown in the base map.

4. The visual verification device based on artificial intelligence object recognition layout and steps according to claim 1 is characterized in that: The shooting angles of the multi-angle image acquisition module include top view, left side, and right side; and the image of the placed simulation object is acquired at the same time.

5. The visual verification device based on artificial intelligence object recognition layout and steps according to claim 1 is characterized in that: The object recognition module creates corresponding data sets for the object categories that can be judged by the images of each shooting angle, and then trains the object recognition model according to the object types in the data sets. The trained object model can recognize the categories and location information of all objects contained in the current angle.

6. The visual verification device based on artificial intelligence object recognition layout and steps according to claim 1 is characterized in that: The multi-angle object recognition result combination module determines the consistency of the number and types of recognized objects based on the object recognition results of the multi-angle collected images.

7. The visual verification device based on artificial intelligence object recognition layout and steps according to claim 1 is characterized in that: The steps of model training in the object recognition module are as follows: create entity category labels appearing in the data sets for different shooting angles, use entity recognition annotation tools to annotate the positions of the entities in the images and the entity names in the data sets, and after the annotation is completed, perform entity recognition model training on different data sets respectively. After the model training is completed, verify the model recognition effect. If the model recognition effect is not good, analyze the cause, and optimize the model by increasing the data set and adjusting the model training parameters.

8. The visual verification device based on artificial intelligence object recognition layout and steps according to claim 1 is characterized in that: In the four-in-one visual verification module, when performing visual display, corresponding electronic tags are created according to the types of objects in the current scene, including the categories of different orientations that need to be distinguished for the same object; the electronic tags are displayed on the base map of the corresponding scene to achieve visual display.

9. The visual verification device based on artificial intelligence object recognition layout and steps according to claim 1 is characterized in that: In the four-in-one visual verification module, the specific steps of rule verification are as follows: according to the pre-set rules and scoring criteria, click rule verification to verify the placement results of the object recognition in the current scene. The verification rules include the placement direction of the objects, the number of objects, the interval distance between the objects, and the order of placement. The points where points will be deducted will be marked and displayed at the corresponding positions of the scene base map, and a one-to-one comparison will be made to obtain the scores and analyze the reasons for the placement errors. The way to interact with the analysis results and the answer diagram is to click anywhere in the score analysis table, and the corresponding placement point will be displayed on the answer diagram.

Images