Hazardous chemical transportation pictogram classification method and device based on AR and AI

Abstract

The invention discloses a dangerous chemical transportation pictogram classification method and device based on AR and AI, and relates to the field of AR. The method comprises the following steps: starting an acquisition program of AR glasses, and judging whether the AR glasses recognize a dangerous chemical transportation pictographic label or not; if it is determined that the AR glasses recognize the dangerous chemical transportation pictographic label, whether the shooting posture of the AR glasses is a preset standard posture or not is judged; if it is determined that the shooting posture of the AR glasses is a preset standard posture, shooting a target pictographic image of the dangerous chemical transportation pictographic label through the AR glasses; inputting the target pictographic image into a preset target detection model for processing to obtain bounding box coordinates and category information corresponding to the target pictographic image; and controlling AR glasses to generate a virtual label of the dangerous chemical transportation pictographic label according to the boundary frame coordinates and the category information, and covering the virtual label on the dangerous chemical transportation pictographic label. By implementing the technical scheme provided by the invention, the identification accuracy of the pictographic label for dangerous chemical transportation is improved.

Description

Technical Field

[0001] The present application relates to the AR field, and specifically to a method and device for classifying hazardous chemicals transportation pictograms based on AR and AI. Background Art

[0002] Dangerous chemicals, referred to as hazardous chemicals, include chemical substances with explosive, corrosive, toxic, flammable or combustible properties, which may be harmful or extremely dangerous to humans, equipment and the environment. In view of this, in order to protect the environment and human safety, it is particularly important to accurately classify these chemicals. To this end, the United Nations has launched the "Globally Harmonized System of Classification and Labelling of Chemicals" to unify standards. In the production, operation, and import and export links, companies must comply with relevant domestic and foreign laws and regulations to ensure the safety of hazardous chemicals during transportation and storage. This includes the use of standardized packaging and clear labeling, and the attachment of hazardous chemicals transportation pictogram labels to hazardous chemicals. Customs inspection agencies will inspect whether they comply with regulations based on the declaration information submitted by the company and the hazardous chemicals transportation pictogram labels on the hazardous chemicals.

[0003] At present, the identification of hazardous chemicals transport pictographic labels is currently in the manual review stage at the customs and enterprise ends. The manual identification results need to rely on the customs' professional knowledge, which will bring about high identification costs and identification errors. Therefore, the identification of hazardous chemicals transport pictographic labels in related technologies has the problem of low accuracy.

[0004] Therefore, there is an urgent need for a hazardous chemicals transportation pictogram classification method and device based on AR and AI. Summary of the invention

[0005] The present application provides a method and device for classifying hazardous chemicals transportation pictograms based on AR and AI, which improves the accuracy of identifying hazardous chemicals transportation pictogram labels.

[0006] In a first aspect of the present application, a hazardous chemicals transportation pictogram classification method based on AR and AI is provided, the method comprising: starting an acquisition program of AR glasses to determine whether the AR glasses recognize the hazardous chemicals transportation pictogram label; if it is determined that the AR glasses recognize the hazardous chemicals transportation pictogram label, determining whether the shooting posture of the AR glasses is a preset standard posture; if it is determined that the shooting posture of the AR glasses is the preset standard posture, shooting a target pictogram image of the hazardous chemicals transportation pictogram label through the AR glasses; inputting the target pictogram image into a preset target detection model for processing to obtain bounding box coordinates and category information corresponding to the target pictogram image; controlling the AR glasses to generate a virtual label of the hazardous chemicals transportation pictogram label according to the bounding box coordinates and category information, and covering the virtual label on the hazardous chemicals transportation pictogram label.

[0007] By adopting the above technical solution, the target pictographic images containing the pictographic labels for the transportation of hazardous chemicals are collected through AR glasses, and the preset target detection model is used for identification and classification, and finally a virtual label is generated on the AR glasses, realizing the real-time, accurate and intuitive presentation of hazardous chemical information. At the same time, the method ensures the quality and consistency of the collected images and improves the accuracy of subsequent recognition by judging whether the shooting posture of the AR glasses meets the preset standard posture. The method makes full use of the interactivity of AR technology and the intelligence of AI technology, enabling on-site operators to quickly and accurately identify the types of hazardous chemicals, and improving the accuracy, safety and efficiency of the identification of pictographic labels for the transportation of hazardous chemicals.

[0008] Optionally, before starting the acquisition program of the AR glasses and determining whether the AR glasses recognize the hazardous chemicals transportation pictogram label, the method also includes: obtaining a plurality of hazardous chemicals transportation pictogram samples and sample classifications corresponding to each of the hazardous chemicals transportation pictogram samples; extracting key image areas of the hazardous chemicals transportation pictogram samples, and adding corresponding hazardous chemicals category labels to each of the key image areas according to the sample classification to generate a labeling file; randomly dividing the labeling file into a training set and a test set according to a preset ratio; inputting the training set into a preset YOLOv5 network model for training, and testing the preset YOLOv5 network model according to the test set to obtain model accuracy; determining whether the model accuracy is greater than or equal to a preset accuracy threshold; if it is determined that the model accuracy is greater than or equal to the preset accuracy threshold, stopping training to obtain the preset target detection model.

[0009] By adopting the above technical solution, by obtaining a large number of hazardous chemicals transportation pictogram samples, annotating and preprocessing them, this method establishes a high-quality training data set, which provides rich and accurate prior knowledge for the learning of the preset target detection model. At the same time, this method adopts the preset YOLOv5 network structure, which can effectively extract and fuse the multi-scale features of the image, and has strong target detection and classification capabilities. Through reasonable data set division and cross-validation, this method can objectively evaluate the accuracy and generalization performance of the preset target detection model, avoiding the problems of overfitting and underfitting. By setting the accuracy threshold, this method can dynamically control the convergence degree of the preset target detection model training, while meeting actual needs, and minimizing the waste of computing resources.

[0010] Optionally, determining whether the shooting posture of the AR glasses is a preset standard posture specifically includes: using a gyroscope sensor built into the AR glasses to detect the shooting posture of the AR glasses; calculating a deviation value between the shooting posture and the preset standard posture; determining whether the deviation value is greater than or equal to a preset deviation threshold; if it is determined that the deviation value is greater than or equal to the preset deviation threshold, generating a virtual auxiliary line through the display screen of the AR glasses to guide the user to adjust the shooting posture of the AR glasses to the preset standard posture.

[0011] By adopting the above technical solution, by utilizing the built-in gyroscope sensor of AR glasses, the method can detect the deviation value of the shooting posture in real time and accurately. By setting a preset deviation threshold, the method can determine whether the current shooting posture meets the preset standard posture requirements. When it is detected that the deviation value is too large, the method can generate virtual auxiliary lines through the display screen of AR glasses to intuitively guide the user to adjust the shooting posture, thereby improving the friendliness of human-computer interaction and the convenience of operation. Through posture correction and standardization, the method can effectively eliminate image distortion and differences introduced by factors such as shooting angle and distance, and provide high-quality and consistent input data for subsequent target detection and classification.

[0012] Optionally, the step of inputting the target pictographic image into a preset target detection model for processing to obtain bounding box coordinates and category information corresponding to the target pictographic image specifically includes: inputting the target pictographic image into the preset target detection model; extracting multi-scale features from the target pictographic image to obtain feature maps of multiple scales; fusing the feature maps of multiple scales to obtain a fused feature map; and predicting the category information and position information of the target pictographic image based on the fused feature map to obtain bounding box coordinates and category information corresponding to the target pictographic image.

[0013] By adopting the above technical solutions, the method can effectively extract and utilize multi-scale and multi-level feature information in the image. Among them, through multi-scale convolution and pooling operations, the local and global features of hazardous chemical targets can be adaptively captured, with strong scale invariance and position invariance. Through cross-scale and cross-level feature combination and weighting, semantic information at different abstract levels can be fully utilized. Through the joint optimization of regression and classification tasks, the category information and location information of the target pictographic image can be output simultaneously, realizing end-to-end target detection and recognition. Through this modular and systematic design, the method can significantly improve the recognition accuracy and efficiency of the target pictographic image, and provide reliable and accurate data support for subsequent virtual label generation and AR presentation.

[0014] Optionally, controlling the AR glasses to generate a virtual label for the hazardous chemicals transportation pictographic label according to the bounding box coordinates and category information, and overlaying the virtual label on the hazardous chemicals transportation pictographic label, specifically includes: selecting a label style of the corresponding category from a preset virtual label template library according to the category information; utilizing the spatial mapping function of the AR glasses to obtain the plane area where the hazardous chemicals transportation pictographic label is located; and overlaying the virtual label on the plane area corresponding to the hazardous chemicals transportation pictographic label.

[0015] By adopting the above technical solutions, the generation and presentation of virtual labels are further optimized, and the intuitiveness and aesthetics of AR interaction are improved. By presetting the virtual label template library, the method can quickly and automatically match and select the corresponding label style according to the category information of the target pictographic image. By utilizing the spatial mapping function of AR glasses, the method can accurately obtain the plane area where the hazardous chemicals transportation pictographic label is located, providing a reliable spatial reference for the positioning and coverage of the virtual label. By seamlessly integrating the hazardous chemicals transportation pictographic label with the real environment, the method can create a realistic and natural AR experience, enhancing the user's immersion and interactivity. Through this intelligent and visual label presentation method, the method can significantly improve the recognition efficiency and accuracy of hazardous chemicals information, and provide intuitive and convenient auxiliary decision support for on-site operators.

[0016] Optionally, the determining whether the AR glasses recognize the hazardous chemicals transportation pictographic label specifically includes: obtaining a video stream captured in real time by a camera of the AR glasses, the video stream including multiple frames of images; processing a target image to identify whether the hazardous chemicals transportation pictographic label exists in the target image; if it is determined that the hazardous chemicals transportation pictographic label exists in the target image, determining whether the target image meets preset image quality requirements, the preset image quality requirements including image clarity requirements, image integrity requirements, and image angle requirements; if it is determined that the target image meets the preset image quality requirements, determining that the AR glasses recognize the hazardous chemicals transportation pictographic label; if it is determined that the hazardous chemicals transportation pictographic label does not exist in the target image or the target image does not meet the preset image quality requirements, determining that the AR glasses do not recognize the hazardous chemicals transportation pictographic label.

[0017] By adopting the above technical solution, the video stream collected by the AR glasses camera is acquired in real time, and each frame of the image is continuously and dynamically analyzed and processed, realizing the real-time tracking and positioning of the pictographic label for the transportation of hazardous chemicals. By presetting image quality requirements, such as clarity, completeness and angle, it is possible to automatically evaluate whether the current image meets the requirements of subsequent recognition and labeling, avoiding misjudgment and missed judgment due to poor image quality.

[0018] Optionally, before inputting the target pictographic image into a preset target detection model for processing to obtain the bounding box coordinates and category information corresponding to the target pictographic image, the method also includes: performing a preprocessing operation on the target pictographic image, the preprocessing operation including image size normalization, image enhancement, and image noise removal.

[0019] By adopting the above technical solution and performing a series of preprocessing operations on the original acquired images, such as size normalization, image enhancement and image noise removal, this method can effectively eliminate image distortion and interference introduced by factors such as shooting conditions and equipment differences, and provide standardized and high-quality input data for subsequent feature extraction and target detection.

[0020] In a second aspect of the present application, a hazardous chemicals transportation pictogram classification device based on AR and AI is provided, the device comprising an AR glasses starting module, a judgment module, a determination module and a processing module, wherein: the AR glasses starting module starts the acquisition program of the AR glasses to determine whether the AR glasses recognize the hazardous chemicals transportation pictogram label; the judgment module is used to determine whether the shooting posture of the AR glasses is a preset standard posture if it is determined that the AR glasses recognize the hazardous chemicals transportation pictogram label; the determination module is used to shoot a target pictogram image of the hazardous chemicals transportation pictogram label through the AR glasses if it is determined that the shooting posture of the AR glasses is the preset standard posture; the processing module is used to input the target pictogram image into a preset target detection model for processing to obtain the bounding box coordinates and category information corresponding to the target pictogram image; the processing module is also used to control the AR glasses to generate a virtual label of the hazardous chemicals transportation pictogram label according to the bounding box coordinates and category information, and cover the virtual label on the hazardous chemicals transportation pictogram label.

[0021] In the third aspect of the present application, an electronic device is provided, including a processor, a memory, a user interface and a network interface, the memory is used to store instructions, the user interface and the network interface are both used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes any one of the methods described above.

[0022] In a fourth aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores instructions, and when the instructions are executed, any of the methods described above is executed.

[0023] In summary, one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. The target pictographic images containing the pictographic labels for the transportation of hazardous chemicals are collected through AR glasses, and the preset target detection model is used for identification and classification, and finally a virtual label is generated on the AR glasses, realizing the real-time, accurate and intuitive presentation of hazardous chemical information. At the same time, this method ensures the quality and consistency of the collected images and improves the accuracy of subsequent recognition by judging whether the shooting posture of the AR glasses meets the preset standard posture. This method makes full use of the interactivity of AR technology and the intelligence of AI technology, enabling on-site operators to quickly and accurately identify the types of hazardous chemicals, and improving the accuracy, safety and efficiency of the identification of pictographic labels for the transportation of hazardous chemicals.

[0024] 2. By acquiring a large number of hazardous chemicals transportation pictogram samples, annotating and preprocessing them, this method establishes a high-quality training data set, which provides rich and accurate prior knowledge for the learning of the preset target detection model. At the same time, this method adopts the preset YOLOv5 network structure, which can effectively extract and fuse the multi-scale features of the image and has strong target detection and classification capabilities. Through reasonable data set division and cross-validation, this method can objectively evaluate the accuracy and generalization performance of the preset target detection model, avoiding the problems of overfitting and underfitting. By setting the accuracy threshold, this method can dynamically control the convergence degree of the preset target detection model training, while meeting actual needs, and minimizing the waste of computing resources.

[0025] 3. By utilizing the built-in gyroscope sensor of AR glasses, this method can detect the deviation value of the shooting posture in real time and accurately. By setting a preset deviation threshold, the method can determine whether the current shooting posture meets the preset standard posture requirements. When it is detected that the deviation value is too large, the method can generate virtual auxiliary lines through the display screen of AR glasses to intuitively guide the user to adjust the shooting posture, thereby improving the friendliness of human-computer interaction and the convenience of operation. Through posture correction and standardization, the method can effectively eliminate image distortion and differences introduced by factors such as shooting angle and distance, and provide high-quality and consistent input data for subsequent target detection and classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a flow chart of a hazardous chemicals transportation pictogram classification method based on AR and AI disclosed in an embodiment of the present application; Figure 2 It is an example schematic diagram of a hazardous chemicals transportation pictogram classification method based on AR and AI disclosed in an embodiment of the present application; Figure 3 It is a module schematic diagram of a hazardous chemicals transportation pictogram classification device based on AR and AI disclosed in an embodiment of the present application; Figure 4It is a structural schematic diagram of an electronic device disclosed in an embodiment of the present application.

[0027] Explanation of the accompanying drawings: 301, AR glasses starting module; 302, judgment module; 303, determination module; 304, processing module; 400, electronic device; 401, processor; 402, communication bus; 403, user interface; 404, network interface; 405, memory. DETAILED DESCRIPTION

[0028] In order to enable technicians in this field to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments.

[0029] In the description of the embodiments of the present application, words such as "for example" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "for example" or "for example" is intended to present related concepts in a specific way.

[0030] In the description of the embodiments of the present application, the meaning of the term "multiple" refers to two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "include", "comprise", "have" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0031] Before introducing the embodiments of the present application, some terms involved in the embodiments of the present application are first defined and explained.

[0032] The full name of hazardous chemicals is dangerous chemicals, which refers to chemicals with explosive, corrosive, poisonous, flammable, and combustion-supporting properties. They are harmful or highly toxic to humans, equipment, and the environment. Therefore, in order to protect humans and the environment, hazardous chemicals need to be correctly classified. The United Nations has formulated the "Globally Harmonized System of Classification and Labeling of Chemicals". Enterprises need to comply with national and international regulations when producing, operating, and importing and exporting hazardous chemicals to ensure that hazardous chemicals meet safety requirements during transportation and storage. They need to use standard packaging and labeling, and affix hazardous chemicals transportation pictographic labels to hazardous chemicals. Import and export customs check the compliance of hazardous chemicals through the company's declaration information and the hazardous chemicals transportation pictographic labels on the hazardous chemicals. This technical solution is to detect and identify hazardous chemicals transportation pictographic patterns, and use deep learning technology to efficiently identify and classify the types of hazardous chemicals.

[0033] This application provides a pictogram classification method for hazardous chemicals transportation based on AR and AI. Figure 1 , Figure 1 It is a flowchart of a hazardous chemicals transport pictogram classification method based on AR and AI provided in an embodiment of the present application. The method is applied to a server, which is a server that executes a hazardous chemicals transport pictogram classification program. The server can be a single server, or a server cluster composed of multiple servers, or a cloud computing service center. The server can communicate with the AR glasses via a wired or wireless network. The method includes steps S101 to S105, and the above steps are as follows: Step S101: Start the collection program of the AR glasses to determine whether the AR glasses recognize the hazardous chemicals transportation pictographic label.

[0034] In a possible implementation, before step S101, the method further includes: obtaining a plurality of hazardous chemicals transport pictogram samples and a sample classification corresponding to each of the hazardous chemicals transport pictogram samples; extracting key image areas of the hazardous chemicals transport pictogram samples, and adding corresponding hazardous chemicals category labels to each of the key image areas according to the sample classification to generate a labeling file; randomly dividing the labeling file into a training set and a test set according to a preset ratio; inputting the training set into a preset YOLOv5 network model for training, and testing the preset YOLOv5 network model according to the test set to obtain a model accuracy; determining whether the model accuracy is greater than or equal to a preset accuracy threshold; if it is determined that the model accuracy is greater than or equal to the preset accuracy threshold, stopping the training to obtain the preset target detection model.

[0035] Specifically, the server downloads 26 samples of hazardous chemicals transportation pictograms from a specified website, such as Figure 2These samples cover seven categories: explosive transport pictograms, flammable gas transport pictograms, flammable solid transport pictograms, corrosive goods transport pictograms, toxic goods transport pictograms, compressed goods transport pictograms and other items transport pictograms.

[0036] After obtaining the original sample, the server annotates and preprocesses it to convert it into a standard format suitable for model training. Specifically, the user pre-attaches the corresponding hazardous chemical category label to each hazardous chemical transport pictogram sample according to the sample classification standard. For example, for a flammable solid transport pictogram sample, the server annotates it as a "flammable solid transport pictogram" category according to the annotation entered by the user.

[0037] Next, the server uses an image segmentation algorithm to extract the key image areas of the hazardous chemicals transport pictogram from each hazardous chemicals transport pictogram sample, removing background interference and irrelevant information. For example, for a flammable solid transport pictogram, the server segments the area where key elements such as flames are located. Next, the server attaches the corresponding hazardous chemicals category label to each extracted key image area based on the pre-defined sample classification and generates a complete annotation file.

[0038] Next, the server divides the labeled standard file sample set into a training set and a test set, and distributes them according to a certain ratio (such as 8:2). After the data preparation is completed, the server uses the labeled training set to train the preset YOLOv5 network model. In an embodiment of the present application, the preset YOLOv5 network model is a target detection algorithm that implements end-to-end feature extraction and target prediction through a convolutional neural network. During the training process, the server inputs the training samples into the YOLOv5 model, and adjusts and optimizes the parameters of the preset YOLOv5 network model through the forward propagation and back propagation mechanisms, so that it can automatically learn and summarize the key features and discrimination rules of the hazardous chemicals transportation pictogram samples from the hazardous chemicals transportation pictogram samples. For example, by learning the color and size features of different types of hazardous chemicals transportation pictogram samples, the preset YOLOv5 network model establishes a robust feature representation and classification decision boundary, thereby realizing the recognition and classification of unknown hazardous chemicals transportation pictogram samples.

[0039] After each round of training iteration, the server uses the test set samples to evaluate the performance of the current preset YOLOv5 network model. By comparing the prediction results of the preset YOLOv5 network model on the test set with the true label, the server calculates the accuracy of the preset YOLOv5 network model as the model accuracy. At the same time, the server also sets a preset accuracy threshold as the stop condition for the preset YOLOv5 network model training. When the training accuracy of the preset YOLOv5 network model on the test set is greater than or equal to the preset accuracy threshold, the server automatically stops training and obtains the preset target detection model.

[0040] In step S101, it is determined whether the AR glasses recognize the hazardous chemicals transportation pictographic label, specifically including: obtaining a video stream collected in real time by the camera of the AR glasses, the video stream including multiple frames of images; processing the target image to identify whether the hazardous chemicals transportation pictographic label exists in the target image; if it is determined that the hazardous chemicals transportation pictographic label exists in the target image, determining whether the target image meets preset image quality requirements, the preset image quality requirements including image clarity requirements, image integrity requirements and image angle requirements; if it is determined that the target image meets the preset image quality requirements, it is determined that the AR glasses recognize the hazardous chemicals transportation pictographic label; if it is determined that the hazardous chemicals transportation pictographic label does not exist in the target image or the target image does not meet the preset image quality requirements, it is determined that the AR glasses do not recognize the hazardous chemicals transportation pictographic label.

[0041] Specifically, the server starts the camera acquisition program of the AR glasses and obtains the video stream captured by the front camera of the AR glasses in real time. The video stream consists of a series of continuous image frames, which records the real-time scene in front of the AR glasses lens. Then, the server processes and analyzes each frame of the video stream in turn. The server determines that any frame of the multiple frames is the target image; for the target image, the server uses the preset target detection model to determine whether there is a hazardous chemicals transportation pictographic label in it.

[0042] The preset target detection model is trained on a large number of hazardous chemicals transportation pictogram samples through the aforementioned steps, and can detect and locate the positions of various hazardous chemicals transportation pictograms from the input target image.

[0043] If the preset target detection model successfully detects the hazardous chemicals transport pictogram label in the target image, the server evaluates whether the target image meets the preset image quality requirements. The preset image quality requirements include image clarity requirements, image integrity requirements, and image angle requirements. For the image clarity requirement, the server calculates the gradient information of the target image as a standard for measuring image clarity. If the image clarity is lower than the preset clarity threshold (such as the average gradient is less than 20), the target image is considered to be unqualified in terms of image clarity requirements. For image integrity, the server analyzes the position and size of the hazardous chemicals transport pictogram label in the target image to determine whether the hazardous chemicals transport pictogram label appears completely in the image. If the edge of the hazardous chemicals transport pictogram label is truncated by the image boundary, or the area of ​​the hazardous chemicals transport pictogram label is less than a preset proportion of the total area of ​​the target image (such as 30%), the target image is considered to not meet the image integrity requirements. For the image angle requirement, the server calculates the inclination of the hazardous chemicals transport pictogram label in the target image. If the angle between the hazardous chemicals transport pictogram and the image boundary exceeds the preset range (such as ±15°), the target image is considered to not meet the image angle requirement.

[0044] Only when the target image meets the image clarity requirements, image integrity requirements, and image angle requirements at the same time, the server determines that the AR glasses have successfully identified the hazardous chemicals transportation pictogram label. Otherwise, the server will determine that the AR glasses have not correctly aimed at the target. At this time, the server controls the AR glasses to issue a prompt message, prompting the user to adjust the shooting position and angle.

[0045] If the hazardous chemicals transport pictogram label is not detected in multiple consecutive frames of images within a preset time (such as 5 seconds), or the detected target image cannot meet the preset image quality requirements, the server will determine that the AR glasses have not recognized the hazardous chemicals transport pictogram label. At this time, the server guides the user to adjust the device position through voice or text prompts of the AR glasses to obtain a clear and complete hazardous chemicals transport pictogram.

[0046] Step S102: If it is determined that the AR glasses recognize the hazardous chemicals transportation pictographic label, it is determined whether the shooting posture of the AR glasses is a preset standard posture.

[0047] In step S102, it is determined whether the shooting posture of the AR glasses is a preset standard posture, specifically including: using the built-in gyroscope sensor of the AR glasses to detect the shooting posture of the AR glasses; calculating the deviation value between the shooting posture and the preset standard posture; determining whether the deviation value is greater than or equal to a preset deviation threshold; if it is determined that the deviation value is greater than or equal to the preset deviation threshold, generating a virtual auxiliary line through the display screen of the AR glasses to guide the user to adjust the shooting posture of the AR glasses to the preset standard posture.

[0048] Specifically, when the server determines that the AR glasses have been successfully aligned with the hazardous chemicals transportation pictogram, it will further determine whether the shooting posture of the AR glasses meets the preset standard posture. The preset standard posture usually refers to the relative position relationship between the AR glasses and the photographed object (i.e., the hazardous chemicals transportation pictogram label) that is facing and horizontal, which is conducive to obtaining a clear, complete, and distortion-free image. In order to detect the real-time shooting posture of the AR glasses, the server uses the built-in gyroscope sensor of the AR glasses. The gyroscope sensor can measure the angular velocity and acceleration information of the AR glasses in three-dimensional space in real time, and then calculate the shooting posture parameters of the AR glasses, such as pitch angle, roll angle, and yaw angle.

[0049] After the server obtains the shooting posture parameters of the AR glasses, it compares them with the preset standard posture and calculates the deviation between the two. The preset standard posture can be set in the embodiment of the present application as follows: pitch angle 0° and roll angle 0°, and yaw angle within a certain range (such as -10° to 10°). The deviation value is measured using the Euler angle distance.

[0050] Next, the server determines whether the calculated deviation value is greater than or equal to the preset deviation threshold. The setting of the preset deviation threshold takes into account factors such as the shooting scene and user experience, and this application does not limit this. If the server determines that the deviation value exceeds the preset deviation threshold, it determines that the shooting posture of the AR glasses is not standard. At this time, the server generates virtual auxiliary lines or auxiliary graphics through the display screen of the AR glasses, prompting the user how to move and rotate the AR glasses to make their shooting posture close to the preset standard posture.

[0051] For example, the server generates a vertical guideline and a horizontal guideline in the center of the display screen, which correspond to the pitch angle and roll angle under the preset standard posture. If the pitch angle of the AR glasses is too large, the vertical guideline will tilt upward; if the roll angle is too large, the horizontal guideline will rotate. The user can adjust the angle of the AR glasses so that the hazardous chemicals transportation pictogram in the target image coincides with the guideline, so that the shooting posture of the AR glasses can be close to the standard posture. Another guidance method is that the server generates a virtual "bubble" icon on the display screen, similar to the bubble in the spirit level. When the shooting posture of the AR glasses deviates from the preset standard posture, the bubble icon will deviate from the center of the screen; the user can adjust the angle of the AR glasses to make the bubble icon return to the circle in the center of the screen, which means that the shooting posture has been corrected.

[0052] Step S103: If it is determined that the shooting posture of the AR glasses is a preset standard posture, the target pictographic image of the hazardous chemicals transportation pictographic label is photographed by the AR glasses.

[0053] In step S103, when the server confirms that the shooting posture of the AR glasses is the preset standard posture, it enters the shooting and collection stage of the hazardous chemicals transportation pictographic label. Specifically, the server sends a shooting instruction to the user through the display screen of the AR glasses, guiding the user to shoot the target pictographic image containing the hazardous chemicals transportation pictographic label. Each time the shot is taken, the AR glasses transmit the collected target pictographic image to the server in real time. Through the above process, the server can collect the target pictographic image from the AR glasses.

[0054] Step S104: input the target pictographic image into a preset target detection model for processing to obtain the bounding box coordinates and category information corresponding to the target pictographic image.

[0055] In a possible implementation, before the target pictographic image is input into a preset target detection model for processing to obtain the bounding box coordinates and category information corresponding to the target pictographic image, the method further includes: performing a preprocessing operation on the target pictographic image, wherein the preprocessing operation includes image size normalization, image enhancement, and image noise removal.

[0056] Specifically, before the server inputs the target pictographic image into the preset target detection model, it also performs a preprocessing operation on the target pictographic image. The preprocessing operation includes image size normalization, image enhancement and image noise removal. First, the server normalizes the image size of the target pictographic image. The server uniformly scales the target pictographic image to a fixed size, such as 640x640 pixels. Secondly, the server enhances the normalized target pictographic image. The server uses a histogram equalization algorithm to automatically adjust the grayscale distribution of the image so that the contrast of the image is more uniform. Specifically, the histogram equalization algorithm counts the frequency of occurrence of each grayscale level in the target pictographic image, and calculates the grayscale mapping function based on the frequency distribution to map the grayscale value of the original image to a new grayscale range. Finally, the server removes noise from the enhanced target pictographic image. In order to remove image noise, the server adopts a smoothing filtering algorithm, such as mean filtering, median filtering, and Gaussian filtering. Based on the above algorithm, the server uses the pixel value distribution characteristics of the local area of ​​the image to perform weighted averaging or sorting processing on each pixel point, thereby eliminating isolated noise points. Taking Gaussian filtering as an example, the Gaussian filtering algorithm will perform weighted averaging on each pixel of the target pictographic image and its neighboring pixels, where the weight coefficient is determined by the Gaussian function. The closer the pixel is to the center point, the greater the weight, and the farther the pixel is, the smaller the weight. Weighted averaging can suppress high-frequency noise while retaining the low-frequency contour features of the image, making the edges and regions of the hazardous chemicals transportation pictogram more obvious and coherent.

[0057] Through the above series of preprocessing operations, the server can convert the originally collected target pictographic image into a standardized, high-quality, low-noise target pictographic image.

[0058] In step S104, the target pictographic image is input into a preset target detection model for processing to obtain bounding box coordinates and category information corresponding to the target pictographic image, specifically including: inputting the target pictographic image into the preset target detection model; extracting multi-scale features from the target pictographic image to obtain feature maps of multiple scales; fusing the feature maps of multiple scales to obtain a fused feature map; predicting the category information and position information of the target pictographic image based on the fused feature map to obtain bounding box coordinates and category information corresponding to the target pictographic image.

[0059] Specifically, the server inputs the preprocessed target pictographic image into a preset target detection model for processing to obtain the bounding box coordinates and category information of the key image area in the target pictographic image. The preset target detection model includes three main functional blocks: feature extraction functional block, feature fusion functional block and prediction functional block.

[0060] The function of the feature extraction block is to extract feature information that can represent the image content from the input target pictographic image. The feature extraction block adopts the structure of convolutional neural network (CNN) to convert the original target pictographic image into feature maps of multiple scales through multi-layer convolution and pooling operations. Feature maps of different scales have different receptive fields and semantic information, thereby capturing the feature patterns of key image areas of the target pictographic image at different sizes and positions.

[0061] For example, for a 800x600 pixel target pictographic image, the feature extraction function block may generate 5 scale feature maps, whose sizes are 200x150, 100x75, 50x38, 25x19 and 13x10. The larger scale feature map retains more detail information and is suitable for detecting small key image areas; the smaller scale feature map contains more context information and is suitable for detecting large key image areas.

[0062] Next, the feature fusion function block will fuse the feature maps of different scales to form a unified, multi-scale feature representation. For example, the feature fusion function block may upsample the feature maps of 25x19 and 13x10 to make their resolution the same as the feature map of 50x38. Then, the feature maps of these three scales are spliced ​​in the channel dimension to obtain a 150x114x256 fused feature map. This fused feature map not only retains the contextual information of the small-scale feature map, but also takes into account the detailed information of the large-scale feature map, which can provide a more comprehensive and accurate feature representation for the subsequent prediction function block.

[0063] Finally, the prediction function block predicts the category and position of the target pictographic image based on the fused feature map. The prediction function block includes two sub-function blocks: the classifier and the bounding box regressor. The classifier is used to predict the category of hazardous chemicals to which each key image area belongs, such as explosive transport pictograms, flammable gas transport pictograms, flammable solid transport pictograms, corrosive goods transport pictograms, etc. The bounding box regressor is used to predict the exact position and size of each key image area and output it in the form of bounding box coordinates.

[0064] For example, the prediction function block slides a 3x3 window on the fused feature map and generates a set of candidate regions for each window position. Then, each candidate region is classified and regressed. The classifier may output a probability vector of length N, indicating the confidence that the candidate region belongs to N hazardous chemical categories. The bounding box regressor may output a coordinate vector of length 4, indicating the center point coordinates and width and height of the candidate region.

[0065] Through the above process, the server uses a preset target detection model to detect key image areas from the target pictographic image and obtain its bounding box coordinates and category information.

[0066] For example, at a customs port, customs officers wear AR glasses to inspect the packaging of a batch of imported chemicals. After the AR glasses collect 10 target pictograms, the server inputs the target pictograms into the preset target detection model. The preset target detection model extracts the feature maps of each target pictogram at 5 scales, then performs feature fusion, and finally uses the classifier and bounding box regressor to predict the fused feature map. The results show that among the 10 hazardous chemical transportation pictograms, a total of 3 flammable gas transportation pictograms, 2 flammable solid transportation pictograms and 1 toxic goods transportation pictogram were detected.

[0067] Step S105: Control the AR glasses to generate a virtual label for the hazardous chemicals transportation pictographic label according to the bounding box coordinates and category information, and cover the virtual label on the hazardous chemicals transportation pictographic label.

[0068] In step S105, the AR glasses are controlled to generate a virtual label for the hazardous chemicals transportation pictographic label according to the bounding box coordinates and the category information, and the virtual label is overlaid on the hazardous chemicals transportation pictographic label, specifically including: selecting a label style of the corresponding category from a preset virtual label template library according to the category information; utilizing the spatial mapping function of the AR glasses to obtain the plane area where the hazardous chemicals transportation pictographic label is located; and overlaying the virtual label on the plane area corresponding to the hazardous chemicals transportation pictographic label.

[0069] Specifically, the server selects a label style of the corresponding category from the preset virtual label template library according to the category information of the target pictographic image. The preset virtual label template library is a pre-designed label style collection that contains standardized label templates for various common hazardous chemicals categories. Each standardized label template has specific visual elements such as color, pattern, text, and a unified size and layout.

[0070] For example, when the server detects that a target pictogram belongs to the flammable gas transport pictogram, it retrieves the style of the flammable gas label from the virtual label template library. The flammable gas label may have a red background with a black flame pattern in the middle, "flammable gas" in white text on the top, and the United Nations number and packaging category of the hazardous chemical in yellow numbers on the bottom.

[0071] Next, the server uses the spatial mapping function of the AR glasses to obtain the plane area where the target pictographic image is located. Spatial mapping can construct a digital three-dimensional model of the real environment and track the position and posture of the AR device in the environment in real time. Through spatial mapping, AR glasses can accurately sense and locate the plane area where the target pictographic image is located. For example, AR glasses can use their built-in depth sensors, such as ToF cameras or structured light sensors, to scan and model the plane area where the hazardous chemicals transportation pictographic label is located. By analyzing the point cloud data in the depth image, AR glasses can determine the plane's spatial position, size, normal vector and other geometric parameters. At the same time, AR glasses will also use the SLAM algorithm to track itself in real time relative to the plane area where the hazardous chemicals transportation pictographic label is located to maintain the spatial consistency of the virtual label with the real scene.

[0072] Finally, the server overlays the selected virtual label template on the plane area corresponding to the target pictographic image to form an augmented reality screen that combines the real and the virtual. Specifically, the server calculates the exact position and size of the virtual label on the plane area based on the bounding box coordinates of the hazardous chemical pictogram. The server then sends the graphic data of the virtual label to the AR glasses, which are rendered into a digital model of the actual scene by the graphics engine of the AR glasses. When the user observes the plane area where the hazardous chemical transport pictographic label is located through the AR glasses, he will see a virtual label that fits it, clearly indicating the category and number of the hazardous chemical.

[0073] Reference Figure 3The present application also provides a hazardous chemicals transportation pictogram classification device based on AR and AI, the device is a server, the server includes an AR glasses startup module 301, a judgment module 302, a determination module 303 and a processing module 304, wherein: the AR glasses startup module 301 starts the collection program of the AR glasses to determine whether the AR glasses recognize the hazardous chemicals transportation pictogram label; the judgment module 302 is used to determine whether the shooting posture of the AR glasses is a preset standard posture if it is determined that the AR glasses recognize the hazardous chemicals transportation pictogram label; the determination module Block 303 is used to capture the target pictographic image of the hazardous chemicals transportation pictographic label through the AR glasses if it is determined that the shooting posture of the AR glasses is a preset standard posture; the processing module 304 is used to input the target pictographic image into a preset target detection model for processing to obtain the bounding box coordinates and category information corresponding to the target pictographic image; the processing module 304 is also used to control the AR glasses to generate a virtual label of the hazardous chemicals transportation pictographic label according to the bounding box coordinates and category information, and cover the virtual label on the hazardous chemicals transportation pictographic label.

[0074] In a possible implementation, the AR glasses startup module 301 starts the acquisition program of the AR glasses, and before determining whether the AR glasses recognize the hazardous chemicals transportation pictogram label, the method further includes: the processing module 304 obtains a plurality of hazardous chemicals transportation pictogram samples and sample classifications corresponding to each of the hazardous chemicals transportation pictogram samples; the processing module 304 extracts the key image areas of the hazardous chemicals transportation pictogram samples, and adds corresponding hazardous chemicals category labels to each of the key image areas according to the sample classification to generate a labeling file; the processing module 304 randomly divides the labeling file into a training set and a test set according to a preset ratio; the processing module 304 inputs the training set into a preset YOLOv5 network model for training, and tests the preset YOLOv5 network model according to the test set to obtain the model accuracy; the processing module 304 determines whether the model accuracy is greater than or equal to a preset accuracy threshold; if the processing module 304 determines that the model accuracy is greater than or equal to the preset accuracy threshold, the training is stopped to obtain the preset target detection model.

[0075] In a possible implementation, the judgment module 302 judges whether the shooting posture of the AR glasses is a preset standard posture, specifically including: the processing module 304 uses the built-in gyroscope sensor of the AR glasses to detect the shooting posture of the AR glasses; the processing module 304 calculates the deviation value between the shooting posture and the preset standard posture; the judgment module 302 judges whether the deviation value is greater than or equal to the preset deviation threshold; if the processing module 304 determines that the deviation value is greater than or equal to the preset deviation threshold, a virtual auxiliary line is generated through the display screen of the AR glasses to guide the user to adjust the shooting posture of the AR glasses to the preset standard posture.

[0076] In a possible implementation, the processing module 304 inputs the target pictographic image into a preset target detection model for processing to obtain bounding box coordinates and category information corresponding to the target pictographic image, specifically including: the processing module 304 inputs the target pictographic image into the preset target detection model; extracts multi-scale features from the target pictographic image to obtain feature maps of multiple scales; the processing module 304 fuses the feature maps of multiple scales to obtain a fused feature map; the processing module 304 predicts the category information and position information of the target pictographic image based on the fused feature map to obtain bounding box coordinates and category information corresponding to the target pictographic image.

[0077] In a possible implementation, the processing module 304 controls the AR glasses to generate a virtual label for the hazardous chemicals transportation pictographic label according to the bounding box coordinates and category information, and covers the virtual label on the hazardous chemicals transportation pictographic label, specifically including: the processing module 304 selects a label style of the corresponding category from a preset virtual label template library according to the category information; the processing module 304 uses the spatial mapping function of the AR glasses to obtain the plane area where the hazardous chemicals transportation pictographic label is located; the processing module 304 covers the virtual label on the plane area corresponding to the hazardous chemicals transportation pictographic label.

[0078] In a possible implementation, the judgment module 302 judges whether the AR glasses recognize the hazardous chemicals transportation pictographic label, specifically including: the processing module 304 obtains the video stream collected in real time by the camera of the AR glasses, and the video stream includes multiple frames of images; the processing module 304 processes the target image to identify whether the hazardous chemicals transportation pictographic label exists in the target image; if the processing module 304 determines that the hazardous chemicals transportation pictographic label exists in the target image, it determines whether the target image meets the preset image quality requirements, and the preset image quality requirements include image clarity requirements, image integrity requirements, and image angle requirements; if the processing module 304 determines that the target image meets the preset image quality requirements, it determines that the AR glasses recognize the hazardous chemicals transportation pictographic label; if the processing module 304 determines that the hazardous chemicals transportation pictographic label does not exist in the target image or the target image does not meet the preset image quality requirements, it determines that the AR glasses do not recognize the hazardous chemicals transportation pictographic label.

[0079] In a possible implementation, before the processing module 304 inputs the target pictographic image into a preset target detection model for processing and obtains the bounding box coordinates and category information corresponding to the target pictographic image, the method further includes: the processing module 304 performs a preprocessing operation on the target pictographic image, and the preprocessing operation includes image size normalization, image enhancement, and image noise removal.

[0080] It should be noted that: when the device provided in the above embodiment realizes its function, only the division of the above functional modules is used as an example. In actual application, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0081] The present application also provides an electronic device. Figure 4 , Figure 4 The electronic device 400 may include: at least one processor 401 , at least one network interface 404 , a user interface 403 , a memory 405 , and at least one communication bus 402 .

[0082] The communication bus 402 is used to realize the connection and communication between these components.

[0083] The user interface 403 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 403 may also include a standard wired interface and a wireless interface.

[0084] The network interface 404 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0085] Among them, the processor 401 may include one or more processing cores. The processor 401 uses various interfaces and lines to connect various parts in the entire server, and executes various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 405, and calling data stored in the memory 405. Optionally, the processor 401 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 401 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 401, and it can be implemented separately through a chip.

[0086] Among them, the memory 405 may include a random access memory (Random Access Memory, RAM) and may also include a read-only memory (Read-Only Memory). Optionally, the memory 405 includes a non-transitory computer-readable storage medium. The memory 405 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 405 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 405 may optionally also be at least one storage device located away from the aforementioned processor 401. Refer to Figure 4, the memory 405 as a computer storage medium may include an operating system, a network communication module, a user interface module, and an application program for a hazardous chemicals transportation pictogram classification method based on AR and AI.

[0087] exist Figure 4 In the electronic device 400 shown, the user interface 403 is mainly used to provide an input interface for the user and obtain the data input by the user; and the processor 401 can be used to call the memory 405 to store an application program for a hazardous chemicals transportation pictogram classification method based on AR and AI. When executed by one or more processors 401, the electronic device 400 executes one or more of the methods described in the above embodiments. It should be noted that for the aforementioned method embodiments, for the sake of simple description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited to the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required for the present application.

[0088] The present application also provides a computer-readable storage medium, which stores instructions. When executed by one or more processors 401, the electronic device 400 executes one or more of the methods described in the above embodiments.

[0089] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0090] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0091] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0092] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0093] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned memory includes: various media that can store program codes, such as USB flash drives, mobile hard drives, magnetic disks or optical disks.

[0094] The above is only an exemplary embodiment of the present disclosure and cannot be used to limit the scope of the present disclosure. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the disclosure of the specification and the truth of practice, those skilled in the art will easily think of other embodiments of the present disclosure.

[0095] This application is intended to cover any variation, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the art not described in the present disclosure. The description and examples are to be regarded as exemplary only, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A pictogram classification method for hazardous chemicals transportation based on AR and AI, characterized in that: The method comprises: Starting the collection program of the AR glasses to determine whether the AR glasses recognize the hazardous chemicals transportation pictographic label; If it is determined that the AR glasses recognize the hazardous chemicals transportation pictographic label, determining whether the shooting posture of the AR glasses is a preset standard posture; If it is determined that the shooting posture of the AR glasses is a preset standard posture, the target pictographic image of the hazardous chemicals transportation pictographic label is photographed by the AR glasses; Inputting the target pictographic image into a preset target detection model for processing to obtain the bounding box coordinates and category information corresponding to the target pictographic image; The AR glasses are controlled to generate a virtual label for the hazardous chemicals transportation pictographic label according to the bounding box coordinates and the category information, and the virtual label is covered on the hazardous chemicals transportation pictographic label.

2. The method according to claim 1, characterized in that: Before starting the collection program of the AR glasses and determining whether the AR glasses recognize the hazardous chemicals transportation pictographic label, the method further includes: Obtaining a plurality of hazardous chemicals transport pictogram samples and sample classifications corresponding to each of the hazardous chemicals transport pictogram samples; Extracting the key image areas of the hazardous chemicals transportation pictogram sample, and adding corresponding hazardous chemicals category labels to each of the key image areas according to the sample classification, to generate a labeling file; Randomly divide the labeled files into a training set and a test set according to a preset ratio; Input the training set into a preset YOLOv5 network model for training, and test the preset YOLOv5 network model according to the test set to obtain model accuracy; Determining whether the model accuracy is greater than or equal to a preset accuracy threshold; If it is determined that the model accuracy is greater than or equal to the preset accuracy threshold, the training is stopped to obtain the preset target detection model.

3. The method according to claim 1, characterized in that The determining whether the shooting posture of the AR glasses is a preset standard posture specifically includes: Utilizing a gyroscope sensor built into the AR glasses to detect the shooting posture of the AR glasses; Calculating a deviation value between the shooting posture and a preset standard posture; Determining whether the deviation value is greater than or equal to a preset deviation threshold; If it is determined that the deviation value is greater than or equal to the preset deviation threshold, a virtual auxiliary line is generated through the display screen of the AR glasses to guide the user to adjust the shooting posture of the AR glasses to the preset standard posture.

4. The method according to claim 1, characterized in that: The step of inputting the target pictographic image into a preset target detection model for processing to obtain the bounding box coordinates and category information corresponding to the target pictographic image specifically includes: Inputting the target pictographic image into the preset target detection model; Extracting multi-scale features from the target pictographic image to obtain feature maps of multiple scales; Fusing the feature maps at multiple scales to obtain a fused feature map; According to the fused feature map, the category information and position information of the target pictographic image are predicted to obtain the bounding box coordinates and category information corresponding to the target pictographic image.

5. The method according to claim 1, characterized in that The controlling the AR glasses to generate a virtual label of the hazardous chemicals transportation pictographic label according to the bounding box coordinates and the category information, and covering the virtual label on the hazardous chemicals transportation pictographic label specifically includes: According to the category information, a label style of a corresponding category is selected from a preset virtual label template library; Using the spatial mapping function of the AR glasses, the plane area where the hazardous chemicals transportation pictographic label is located is obtained; The virtual label is covered on the plane area corresponding to the hazardous chemicals transportation pictographic label.

6. The method according to claim 1, characterized in that The determining whether the AR glasses recognize the hazardous chemicals transportation pictographic label specifically includes: Acquire a video stream collected in real time by a camera of the AR glasses, wherein the video stream includes multiple frames of images; Processing the target image to identify whether the hazardous chemicals transportation pictographic label exists in the target image; If it is determined that the hazardous chemicals transportation pictographic label exists in the target image, then judging whether the target image meets the preset image quality requirements, wherein the preset image quality requirements include image clarity requirements, image integrity requirements, and image angle requirements; If it is determined that the target image meets the preset image quality requirement, then it is determined that the AR glasses recognize the hazardous chemicals transportation pictographic label; If it is determined that the hazardous chemicals transportation pictographic label does not exist in the target image or the target image does not meet the preset image quality requirements, it is determined that the AR glasses have not recognized the hazardous chemicals transportation pictographic label.

7. The method according to claim 1, characterized in that Before inputting the target pictographic image into a preset target detection model for processing to obtain the bounding box coordinates and category information corresponding to the target pictographic image, the method further includes: The target pictographic image is preprocessed, wherein the preprocessing operation includes image size normalization, image enhancement, and image noise removal.

8. A hazardous chemicals transportation pictogram classification device based on AR and AI, characterized in that: The device comprises an AR glasses starting module (301), a judging module (302), a determining module (303) and a processing module (304), wherein: The AR glasses starting module (301) starts a collection program of the AR glasses and determines whether the AR glasses recognize the hazardous chemicals transportation pictographic label; The judgment module (302) is used to judge whether the shooting posture of the AR glasses is a preset standard posture if it is determined that the AR glasses recognize the hazardous chemicals transportation pictographic label; The determination module (303) is used to, if it is determined that the shooting posture of the AR glasses is a preset standard posture, shoot the target pictographic image of the hazardous chemicals transportation pictographic label through the AR glasses; The processing module (304) is used to input the target pictographic image into a preset target detection model for processing, and obtain the bounding box coordinates and category information corresponding to the target pictographic image; The processing module (304) is also used to control the AR glasses to generate a virtual label for the hazardous chemicals transportation pictographic label according to the boundary box coordinates and category information, and to cover the virtual label on the hazardous chemicals transportation pictographic label.

9. An electronic device, characterized in that: The electronic device (400) comprises a processor (401), a memory (405), a user interface (403) and a network interface (404), wherein the memory (405) is used to store instructions, the user interface (403) and the network interface (404) are used to communicate with other devices, and the processor (401) is used to execute the instructions stored in the memory (405) so that the electronic device (400) executes the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed, the method according to any one of claims 1 to 7 is executed.

Images