Big data analysis system and method for automotive systems

Abstract

The present application relates to the field of intelligent analysis, and provides a big data analysis system and method for automobile systems, which first fixes the automobile parts to be labeled by rotating the clamp assembly, then the label selection assembly selects the target label that matches the part, and then the label stripper cooperates with the robot arm to tear the target label from the label roll and accurately apply it to the designated marking area of ​​the part. After that, the camera detection assembly captures the label covering state image of the marking area, and finally identifies the label covering state based on the collected image to confirm whether the label is correctly applied as required. In this way, the automatic label covering quality detection of the automobile labeling system can be realized.

Description

Technical Field

[0001] The present application relates to the field of intelligent analysis, and more specifically, to a big data analysis system and method for automotive systems. Background Art

[0002] With the development of the automotive industry, the requirements for production efficiency and quality control are getting higher and higher. Especially in every step of the assembly line, it is very important to ensure that each component can be installed and labeled correctly. Labels not only help workers identify different automotive parts, but also may contain important information about these parts, such as serial numbers, production dates, etc. Therefore, labels must be accurately attached to the correct parts and keep them in good condition.

[0003] However, traditional technologies mainly rely on simple image processing algorithms, such as edge detection and color contrast, to inspect label coverage. These methods can only identify some basic visual features but cannot understand the semantic information of labels. For example, they may not be able to distinguish between different types of labels or understand the text content on the labels, which limits the system's ability to understand complex label designs and content, and thus affects the ability to identify label coverage.

[0004] Therefore, an optimized big data analysis solution for automotive systems is desired, which can realize automatic recognition of label semantics and covering status to improve the comprehensiveness of labeling quality inspection. Summary of the invention

[0005] In order to solve the above technical problems, the present application is proposed. The embodiments of the present application provide a big data analysis system and method for an automobile system.

[0006] According to one aspect of the present application, a big data analysis system for an automobile system is provided, wherein the automobile system is an automobile labeling system, which is used to label various automobile parts, and the automobile labeling system includes: a label stripper, a mechanical arm, a camera detection component, a label picking component, and a rotating fixture component, wherein the big data analysis system for an automobile system includes: a rotating fixture control module, which is used to control the rotating fixture component to fix the automobile part to be labeled; a label picking module, which is used to control the label picking component to select a target label that is compatible with the automobile part to be labeled; a target label covering module, which is used to control the label stripper and the mechanical arm to tear the target label from the label roll and cover the target label on the marking area of ​​the automobile part to be labeled; a label covering state image acquisition module, which is used to control the camera detection component to acquire the label covering state image of the marking area; a label covering state judgment module, which is used to perform label covering state recognition based on the label covering state image to judge whether the label covering state meets the requirements.

[0007] According to another aspect of the present application, a big data analysis method for an automobile system is provided, which is used to execute the big data analysis system for an automobile system as described above, and comprises: taking the label-covered state image as an image matching index, extracting a high-definition label reference image from a label image database; performing label semantic recognition on the label reference image to obtain a label semantic feature vector; performing multi-scale extraction and multi-dimensional feature interaction on the label-covered state image to obtain a label-covered state multi-scale interaction feature map; performing multi-modal prompt gate-guided interaction on the label semantic feature vector and the label-covered state multi-scale interaction feature map to obtain a label semantic-guided label-covered state cross-modal multi-dimensional representation feature map; obtaining a recognition result based on the label semantic-guided label-covered state cross-modal multi-dimensional representation feature map, and the recognition result is used to indicate whether the label-covered state meets the requirements.

[0008] Compared with the prior art, the big data analysis system and method for automobile systems provided by the present application first fixes the automobile parts to be labeled by rotating the clamp assembly, then the label selection assembly selects the target label that matches the part, and then the label stripper cooperates with the mechanical arm to tear off the target label from the label roll and accurately apply it to the designated marking area of ​​the part. After that, the camera detection assembly captures the label application state image of the marking area, and finally identifies the label application state based on the collected image to confirm whether the label is correctly applied as required. In this way, the automatic label application quality detection of the automobile labeling system can be realized. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0010] In the attached picture: Figure 1 4 is a system block diagram of a big data analysis system for an automotive system according to an embodiment of the present application.

[0011] Figure 2 4 is a block diagram of a label covering state judgment module in a big data analysis system for an automobile system according to an embodiment of the present application.

[0012] Figure 3 4 is a block diagram of a tag status analysis unit in a big data analysis system for an automobile system according to an embodiment of the present application.

[0013] Figure 4 It is a block diagram of a label semantic overlay state interaction unit in a big data analysis system for an automobile system according to an embodiment of the present application.

[0014] Figure 5 4 is a flowchart of a big data analysis method for an automotive system according to an embodiment of the present application. DETAILED DESCRIPTION

[0015] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0016] Based on the problems in the above-mentioned background technology, the present application proposes a big data analysis system for an automobile system, wherein the automobile system is an automobile labeling system, which is used to label various automobile parts, and the automobile labeling system includes: a label stripper, a robotic arm, a camera detection component, a label picking component, and a rotating clamp component. Figure 1 FIG. 1 is a system block diagram of a big data analysis system for an automobile system according to an embodiment of the present application. Figure 1As shown, in the big data analysis system 100 for automobile systems, it includes: a rotating fixture control module 110, used to control the rotating fixture assembly to fix the automobile parts to be labeled; a label selection module 120, used to control the label selection assembly to select a target label that is compatible with the automobile parts to be labeled; a target label covering module 130, used to control the label stripper and the robotic arm to tear the target label from the label roll, and cover the target label on the marking area of ​​the automobile parts to be labeled; a label covering state image acquisition module 140, used to control the camera detection assembly to collect the label covering state image of the marking area; a label covering state judgment module 150, used to perform label covering state recognition based on the label covering state image to judge whether the label covering state meets the requirements.

[0017] In an embodiment of the present application, the rotating fixture control module 110 is used to control the rotating fixture assembly to fix the automobile parts to be labeled. It should be understood that automobile parts vary in shape and size. During the labeling process, if the parts are not fixed, they are likely to move due to the operation of the robot arm, slight collisions during the label pasting process, or other external factors. The automobile parts are fixed by the rotating fixture assembly, and the position of the fixed automobile parts is stable, so that the robot arm can accurately apply the label to the marking area according to the preset path and parameters. In this case, the position accuracy of the label can be effectively controlled, which can greatly reduce the occurrence of skewed or tilted labeling. For example, for safety labels on car seats, fixing the seat components can ensure that the labels are accurately affixed to the specified position on the side of the seat, which is convenient for users to view and meets quality standards.

[0018] In the real automotive production environment, controlling the rotating fixture assembly to fix the automotive parts to be labeled is a precise and complex process. The following is a detailed description of its implementation process: The design of the rotating fixture assembly itself is crucial. Its structure needs to be tailored to the diverse shapes and characteristics of automotive parts. For parts with regular geometric shapes, such as engine cylinders, the fixture may adopt a fixed design with locating pins and a strong clamping device. The initial positioning is achieved by accurately matching the locating pins with specific holes on the cylinder block, and then a stable clamping force is applied with the help of the clamping device. For irregularly shaped parts, such as automotive interior panels, flexible fixtures or multi-point adjustable fixtures are more suitable. This type of fixture can be adaptively adjusted according to the complex curves and contours of the interior panel to ensure full fit with the surface of the component during the fixing process and prevent the component from shifting during subsequent operations. In terms of materials, the fixture body is mostly made of high-strength aluminum alloys or stainless steel to ensure sufficient structural strength and stability to withstand frequent operating stress. At the same time, soft materials such as rubber and nylon will be used to cover or pad the parts that are in direct contact with the components. This can not only effectively avoid scratches, indentations and other damages to the surface of automotive parts, but also increase friction and further improve the reliability of fixation.

[0019] The rotating mechanism of the rotary fixture assembly is a key part to achieve precise positioning. The rotation is usually driven by a motor to complete the rotation, and the combination of servo motors and high-precision encoders is widely used. The servo motor can accurately drive the rotating shaft to rotate according to the control signal, while the high-precision encoder monitors the angle change of the rotating shaft in real time and feeds this information back to the control system. Through this closed-loop control method, the rotary fixture can achieve extremely high angle control accuracy, and the error can be strictly controlled within a very small range, such as ±0.1 degrees or even less. This allows automotive parts to be accurately adjusted to the required angle position during the labeling process, ensuring the accuracy of labeling.

[0020] The control system plays a commanding role in the entire fixing process. The rotary fixture control module is deeply integrated with the overall control system of the automotive labeling system, and generally uses a programmable logic controller (PLC) as the core control unit. This control unit is responsible for receiving command information from various parts of the system, including instructions for component fixing in the labeling program. In terms of communication, the control module and the rotary fixture assembly rely on stable and efficient communication protocols and interfaces for information exchange. Communication interfaces such as Ethernet, RS-232, RS-485, and communication protocols such as Profinet, EtherNet / IP, Modbus, etc. based on these interfaces are widely used. These communication methods ensure the stable transmission of control signals in the system, ensuring that the rotary fixture assembly can respond to control instructions quickly and accurately and perform corresponding actions.

[0021] In order to achieve more precise control and operation monitoring, the rotating fixture assembly is equipped with various types of sensors. Position sensors, such as photoelectric switches and proximity switches, are cleverly placed at key positions of the fixture to accurately detect important position information such as the opening and closing state of the fixture and the rotation angle, and feed this data back to the control module in real time. Pressure sensors focus on monitoring the clamping force applied by the fixture to the automotive component. When the component is placed in the fixture and the clamping operation begins, the pressure sensor continuously measures the magnitude of the clamping force. Once the clamping force reaches the appropriate value pre-set in the system (this value is determined after considering various factors such as the material properties, shape structure, and labeling process requirements of the component), the control module will issue a command in time to stop the clamping action. This ensures that the component is firmly fixed on the fixture, preventing loosening and displacement during the labeling process that affect the labeling quality, and avoids damage to the component due to excessive clamping force, ensuring that the integrity and quality of the component are not affected.

[0022] In the actual operation process, the automotive parts are first accurately transported to the working area where the rotary fixture assembly is located through the conveying device of the production line. At this time, the automated loading equipment carefully places the parts on the rotary fixture. With its carefully designed positioning structure, the fixture quickly performs preliminary positioning operations based on the shape characteristics of the parts and the preset positioning reference. After that, the control module drives the clamping device of the fixture to start working according to the fixed instructions received. There are various types of clamping devices, including pneumatic clamping cylinders, hydraulic clamping cylinders or electric clamping mechanisms. Taking the pneumatic clamping cylinder as an example, when the control signal controls the gas to enter the cylinder through the solenoid valve, the gas pushes the piston to move, which in turn drives the clamping arm of the fixture to apply clamping force to the automotive parts. During the entire clamping process, the pressure sensor always closely monitors the changes in the clamping force.

[0023] If the labeling process requires the label to be attached to a specific angle position of the component, the control module will accurately control the rotation mechanism of the rotary fixture assembly to start according to the parameters pre-set in the system, and rotate the component to the corresponding precise angle. During the rotation process, the position sensor continuously feeds back the rotation angle information to the control module, and the control module accurately adjusts the rotation angle based on these real-time feedback data to ensure that the final angle error is within the extremely small allowable range. When the rotary fixture assembly completes the fixation and angle adjustment of the component, it will send a signal to the entire labeling system.

[0024] In an embodiment of the present application, the label selection module 120 is used to control the label selection component to select a target label that is compatible with the automobile part to be labeled. It should be understood that a car is a complex mechanical product composed of many parts with different functions and characteristics. Each part has its specific purpose and information that needs to be identified by a label. For example, engine parts may need to be labeled with information such as model, serial number, production date, performance parameters, etc., while interior parts may focus more on marking materials, cleaning methods, etc. The content, format, size, etc. of the labels required for different parts are different, so it is necessary to select a target label that is compatible with the automobile part to be labeled. When the label selection component can accurately select the compatible target label, the subsequent labeling process can be carried out more smoothly. There is no need to re-select or correct due to label errors, which reduces interruptions and delays in the production process, thereby improving the overall efficiency of automobile part labeling.

[0025] In a real automobile production environment, controlling the label selection component to select the target label that matches the automobile part to be labeled requires close cooperation of multiple links. The following is a detailed description of its implementation process: First, it is necessary to establish a powerful label information management system. The system contains a detailed and well-classified label database, which records comprehensive information of various labels, such as size, shape, color and content (text, digital code, barcode, QR code, etc.), and is associated with specific attributes of automobile parts. For example, the engine label records the model, serial number, production date and performance parameters, while the interior component label focuses on the material and cleaning method.

[0026] The structural design and functional realization of the label selection component itself are the core of the entire selection process. The component is equipped with storage devices such as label racks and reels to store labels, and integrates advanced technologies such as optical character recognition, barcode and QR code recognition to scan labels to obtain key feature information and compare it with the database in real time. It also has high-precision mechanical grasping and moving mechanisms (such as mechanical arms and grasping devices). Under the command of the control system, it can accurately locate and grasp the target label in three-dimensional space and move it to the appropriate position to cope with the complex arrangement and type distinction of labels in the storage device.

[0027] The control system plays the role of a command center in the label selection process. The label selection control module is integrated into the control system of the automobile labeling system, and is often precisely controlled through control software or embedded systems. When automobile parts enter the labeling process, the control system uses visual sensors and barcode scanners on the production line to obtain detailed information about the parts and transmit it to the control module. The control module queries the database for positioning and adapting target label information based on this information. For example, when the part to be labeled is identified as a car seat, the label information containing relevant content such as seat material, cleaning method, and safety precautions is screened out.

[0028] After determining the target tag, the control module transmits the detailed information of the target tag, such as the specific location in the storage device, unique identification features, etc., to the tag selection component through a stable and efficient communication link. After the tag selection component receives the control instruction, its identification unit quickly locates and accurately identifies the target tag in the storage device based on the target tag information. Subsequently, the mechanical grasping and moving mechanism performs the grasping action under the precise control of the control system. The mechanical arm or grasping device grasps the target tag smoothly and accurately according to the preset motion trajectory and action program, and takes it out of the storage device and places it in a position convenient for subsequent labeling operations, such as on a conveyor belt or a designated temporary storage area. During the entire label selection process, the system always monitors the working status of the label selection component in real time. If there are abnormal conditions such as label grasping failure, label information not matching the preset, or insufficient number of labels in the storage device during the grasping process, the system will immediately issue an alarm signal and take corresponding corrective measures according to the pre-set processing procedure. Possible measures include retrying grasping, automatically switching to the backup label storage location for picking, or triggering the manual intervention processing mechanism when necessary to ensure the continuity and accuracy of the automobile parts labeling work, ultimately achieving an efficient, accurate and reliable label picking process in the automobile labeling production process.

[0029] In the embodiment of the present application, the target label covering module 130 is used to control the label stripper and the mechanical arm to tear the target label from the label roll, and to cover the target label on the marking area of ​​the automobile part to be labeled. It should be understood that the marking area of ​​the automobile part is predetermined and is used to place the label to ensure the standardization and consistency of the label position. On the automobile production line, the label stripper and the mechanical arm are often used to complete the tearing and covering of the label. Through the precise motion control of the mechanical arm and the cooperation of the label stripper, the label can be removed from the label roll and placed in the correct position, so that the label of each component can be covered according to the design requirements, thereby meeting the standardization and normalization requirements in the automobile production process. It is also worth mentioning that by using the label stripper and the mechanical arm for labeling, the entire labeling process will be more stable. No matter how the shape and size of the automobile parts change, as long as the motion trajectory of the mechanical arm and the working parameters of the label stripper are pre-set, it can be ensured that each component can get a consistent labeling effect. This helps to maintain the stability and consistency of product quality during the automobile production process, and facilitates quality control and management.

[0030] The specific implementation process of controlling the label stripper and the robotic arm to tear the target label off the label roll and apply the target label to the marking area of ​​the automotive part to be labeled integrates advanced technologies in mechanical design, automatic control, sensor monitoring and precision motion planning, ensuring the efficiency and stability of labeling of automotive parts. The following is a detailed description of its implementation process: The core work of the label stripper revolves around label peeling, and its internal label peeling mechanism is the key part to achieve this function. Usually, a blade or peeling plate design is adopted. When the control system issues a start command, the label roll starts to rotate under the action of the drive device, and the label moves slowly and enters the peeling mechanism. The angle, sharpness and relative position of the blade to the label roll are finely adjusted, which can accurately cut into the adhesive layer between the label and the base paper, so that the label can be separated smoothly with minimal force and without damage. The surface material and shape of the peeling plate are carefully designed. Its smooth surface helps the label to detach smoothly from the backing paper and can guide the label to move in the predetermined direction, effectively preventing the label from wrinkling, curling or tearing during the peeling process, ensuring the integrity and quality of the label are intact.

[0031] In order to ensure the stability and accuracy of the label peeling process, the label stripping machine is equipped with a variety of sensors for real-time monitoring and feedback. The position sensor always keeps a close eye on the position of the label in the peeling mechanism to ensure that the label starts to peel at the right time to avoid damage to the label or incomplete peeling caused by peeling too early or too late. The tension sensor is responsible for monitoring the change in the tension of the label during the peeling process. This parameter is crucial to the quality of label peeling. When the tension is too large, it may indicate that the label has encountered obstacles during the peeling process, such as excessive friction between the label and the peeling mechanism or the label roll does not rotate smoothly; while too low tension may cause incomplete label peeling or the label to deviate during movement. Once the tension sensor detects an abnormal tension change, it will immediately feed back the signal to the control system. The control system responds quickly based on this feedback information, and ensures that the label peeling process is always carried out under stable tension conditions by adjusting the rotation speed of the label roll, optimizing the working parameters of the peeling mechanism, etc., to ensure the smoothness and consistency of label peeling.

[0032] The robot is responsible for accurately grabbing and moving labels during the entire labeling process, and its motion control and positioning accuracy directly affect the accuracy of labeling. The robot adopts an advanced motion control system, which is usually based on a high-precision servo motor drive and combined with a complex and precise motion control algorithm. The control system receives detailed motion instructions from the host computer, which cover key parameters such as the precise motion trajectory, speed change curve, and acceleration requirements of the robot in three-dimensional space. The servo motor accurately drives the various joints of the robot to move flexibly according to the control signal, so that the robot can achieve high-precision movement and positioning in space. For example, in the process of grabbing a label from a label stripper and moving it to the marking area of ​​​​the automotive part, the motion control system accurately calculates and controls the angle changes of each joint of the robot according to the pre-set optimal trajectory planning, ensuring that the robot can complete the grabbing and moving actions in the fastest and smoothest path, while ensuring that the position of the label is stable during the entire movement process without shaking or offset.

[0033] Some advanced robotic arm systems are also equipped with visual positioning systems to further improve the accuracy of label application. The visual positioning system uses a high-resolution camera to capture real-time images of automotive parts and labels, and then uses powerful image processing algorithms to analyze and process the images. These algorithms can accurately identify the marked area of ​​automotive parts and determine its precise position, shape, and posture in three-dimensional space; at the same time, they can also accurately obtain parameters such as the position, direction, and posture of the label. This visual information is fed back to the motion control system of the robotic arm in real time. The motion control system quickly adjusts the motion trajectory and posture of the robotic arm based on the visual feedback data to achieve real-time dynamic correction of the label application position. For example, when the robotic arm grabs the label and gradually approaches the automotive part, the visual positioning system can detect the slight deviation between the actual position of the marked area and the preset position. The motion control system immediately fine-tunes the movement of the robotic arm based on the deviation value to ensure that the label can be accurately applied to the center of the marked area, and the application error is controlled within a very small range, thereby achieving extremely high application accuracy.

[0034] The control system plays a core coordination role in the entire collaborative work process, and is responsible for accurately controlling the working sequence and action coordination of the label stripper and the robotic arm. In the label peeling stage, the control system first starts the label stripper, and at the same time controls the robotic arm to move to the best waiting position near the label stripper, ready to grab the label. When the label is successfully peeled off from the backing paper, the label stripper sends a signal containing detailed information such as the label position and posture to the robotic arm through the communication link. After receiving the signal, the robotic arm immediately starts the grabbing action according to the preset program, realizing a seamless connection from label peeling to grabbing.

[0035] The label stripper, robotic arm, control system and various sensors must be precisely integrated and calibrated during the installation and commissioning phases. The kinematic parameter calibration of the robotic arm is a key step in ensuring its motion accuracy. By using high-precision measuring instruments and professional calibration tools, the robotic arm's joint zero position, range of motion, geometric relationship between joints and other parameters are precisely measured and adjusted to ensure that the actual motion trajectory of the robotic arm is highly consistent with the preset trajectory. The peeling parameter calibration of the label stripper is equally important. It is necessary to optimize the settings of parameters such as blade position, peeling force, and rotational resistance of the label roll to ensure stable and reliable quality of label peeling. In addition, the calibration of communication and collaborative work between devices is also indispensable to ensure the accuracy and timeliness of data transmission, as well as the coordination and smoothness of the action coordination between devices.

[0036] In the actual operation process, after the label picking component selects the target label and passes the relevant information to the control system, the control system first performs a series of preparations for the label stripper. Check whether the label roll is correctly installed on the stripper to ensure that the label supply is sufficient and there is no jamming. At the same time, initialize the various working parameters of the label stripper, such as setting the initial rotation speed of the label roll and adjusting the initial position of the stripping mechanism. After everything is ready, the control system issues a command, the label stripper is officially started, the label roll begins to rotate at a uniform speed, and the label enters the stripping mechanism smoothly under the traction for stripping. At the same time, the robotic arm quickly moves to the designated waiting position according to the command of the control system, ready to grab the label that is about to be peeled off. When the sensor detects that the label is successfully peeled off from the base paper, it immediately sends a signal to the control system, and the control system then controls the robotic arm to perform precise grasping action. The robotic arm firmly grasps the edge or specific position of the label according to the preset grasping posture and strength to ensure that the label will not be deformed or damaged during the grasping process. After the grasping is successful, the robotic arm moves the label smoothly to the top of the marking area of ​​the automotive part according to the pre-planned motion trajectory. During the movement, the motion control system of the robot arm strictly controls the movement of the robot arm according to the preset speed and acceleration curve to ensure that the label's posture in space is always stable. When the robot arm reaches the top of the marking area, if it is equipped with a visual positioning system, it will further fine-tune the position and posture according to the visual feedback information; if there is no visual positioning system, the robot arm will slowly descend according to the preset laminating height and angle, and gently and accurately apply the label to the marking area. During the laminating process, the robot arm can apply appropriate pressure as needed to make the label fully fit the surface of the component to ensure the firmness and flatness of the laminating. After the laminating is completed, the robot arm carefully releases the label and quickly moves to the next working position according to the subsequent instructions of the control system, ready for the next label laminating task, thereby realizing the fully automated and high-precision operation of the label from peeling to laminating, which effectively guarantees the efficiency and quality of the label laminating work in the automobile production process.

[0037] In the embodiment of the present application, the label covering state image acquisition module 140 is used to control the camera detection component to acquire the label covering state image of the marked area. It should be understood that the label covering state image of the marked area acquired by the camera detection component contains the specific position information of the label in the marked area of ​​the automobile part, the degree of fit between the label and the surface of the automobile part, the label integrity information, etc. In the subsequent analysis of these various information in the acquired label covering state image, it can be judged whether the label covering is qualified.

[0038] In the embodiment of the present application, the label covering state judging module 150 is used to perform label covering state recognition based on the label covering state image to judge whether the label covering state meets the requirements.

[0039] Correspondingly, in the label covering state judgment module, the technical concept of the present application is to extract high-definition label reference images from the label image database, and use image processing and analysis technology based on machine learning and big data analysis to perform semantic recognition on the label reference images, and at the same time perform multi-scale extraction and multi-dimensional interaction on the label covering state images, so as to realize intelligent judgment on the label covering quality based on the semantic features of the label itself and the multi-scale interaction features under its covering state. In this way, the system can deeply analyze the content and type of the label, and identify its performance in a specific environment, while accurately detecting subtle defects in the label text and the label covering state, so as to more accurately judge whether the label is correctly covered, thereby significantly improving the comprehensiveness and intelligence of the label covering state inspection.

[0040] Figure 2 FIG. 1 is a block diagram of a label covering state judgment module in a big data analysis system for an automobile system according to an embodiment of the present application. Figure 2 As shown, the label covering state judgment module 150 includes: a label reference image matching selection unit 151, which is used to extract a high-definition label reference image from a label image database using the label covering state image as an image matching index; a label semantic recognition unit 152, which is used to perform label semantic recognition on the label reference image to obtain a label semantic feature vector; a label state analysis unit 153, which is used to perform multi-scale extraction and multi-dimensional feature interaction on the label covering state image to obtain a label covering state multi-scale interaction feature map; a label semantic covering state interaction unit 154, which is used to perform multi-modal prompt gate guided interaction on the label semantic feature vector and the label covering state multi-scale interaction feature map to obtain a label semantic guided label covering state cross-modal multi-dimensional representation feature map; and a recognition unit 155, which is used to obtain a recognition result based on the label semantic guided label covering state cross-modal multi-dimensional representation feature map, and the recognition result is used to indicate whether the label covering state meets the requirements.

[0041] In an embodiment of the present application, the label reference image matching selection unit 151 is used to extract a high-definition label reference image from a label image database using the label pasting state image as an image matching index. It should be understood that the keyword high-definition used to describe the label reference image specifically refers to the label reference image having sufficiently high resolution, clarity and detail richness, and being able to clearly display the complete pattern, text, symbols and key elements such as the edge of the label. The label reference image is usually taken under ideal conditions, and it can clearly display the tiny serial number, barcode, security mark and other details on the label, so as to accurately perform semantic analysis later. Specifically, the resolution of the high-definition label reference image should be higher than that of the label pasting state image, so as to perform more accurate analysis on the details. For example, the resolution of the label pasting state image is 1024×768 pixels; while the high-definition label reference image may be 2048×1536 pixels or higher. In addition to the appearance information of the label, the label reference image extracted from the label image database also contains semantic information such as text content, symbols, barcodes or QR codes. This information is helpful for subsequent auxiliary judgment of whether the label pasting state meets the requirements.

[0042] In an embodiment of the present application, the label semantic recognition unit 152 is used to perform label semantic recognition on the label reference image to obtain a label semantic feature vector. Specifically, in an embodiment of the present application, the label semantic recognition unit is used to: pass the label reference image through a label semantic recognizer based on a CNN-LSTM model to obtain the label semantic feature vector. It should be understood that the label reference image contains complex patterns, text and other information, which often have rich semantic meanings. Therefore, in order to capture and mine the implicit feature information contained in the reference image and perform contextual semantic understanding of this information, in the technical solution of the present application, the label reference image is passed through a label semantic recognizer based on a CNN-LSTM model to obtain a label semantic feature vector. It can be understood that CNN is good at processing image data and can automatically extract local features in the image. Therefore, the CNN (convolutional neural network) can extract spatial features such as edges, textures and shapes from the label reference image to better characterize the deep feature information in the image. LSTM is good at capturing dependencies in local data and can understand the contextual relationship between these features. That is, LSTM (Long Short-Term Memory Network) can analyze the semantic dependency between these local features. Therefore, the combination of these two models can capture the complex semantic features of the label, improve the accuracy of understanding the label content, and make the generated label semantic feature vector accurately reflect the actual content of the label and its expression in a specific environment, which is helpful for detailed analysis of the label covering status in subsequent steps.

[0043] In the embodiment of the present application, the label state analysis unit 153 is used to perform multi-scale extraction and multi-dimensional feature interaction on the label covering state image to obtain a multi-scale interactive feature map of the label covering state. Specifically, Figure 3 FIG. 1 is a block diagram of a tag status analysis unit in a big data analysis system for an automobile system according to an embodiment of the present application. Figure 3 As shown, the label state analysis unit 153 includes: a label covering state image multi-scale feature extraction subunit 1531, which is used to pass the label covering state image through a covering state multi-scale extractor based on a hollow pyramid network to obtain a label covering state shallow feature map and a label covering state semantic feature map; a label covering state multi-scale interaction subunit 1532, which is used to pass the label covering state shallow feature map and the label covering state semantic feature map through a covering state multi-dimensional feature interaction encoder to obtain the label covering state multi-scale interaction feature map.

[0044] In an embodiment of the present application, the label-covered state image multi-scale feature extraction subunit 1531 is used to pass the label-covered state image through a covering state multi-scale extractor based on a hollow pyramid network to obtain a label-covered state shallow feature map and a label-covered state semantic feature map. Accordingly, considering that the label-covered state image contains rich state image information, such as local details in the shallow covering, such as edges and textures, it also contains higher-level and abstract semantic information, such as the overall layout and positional relationship of the label. Moreover, features of different scales are crucial to understanding the label covering state. Shallow features focus on local details, while deep features can capture higher-level semantic information. Therefore, in order to capture and extract feature information at different levels to improve the ability to understand the label covering state, in the technical solution of the present application, the label covering state image is passed through a covering state multi-scale extractor based on a dilated pyramid network to utilize dilated convolution to insert "holes" or "holes" in the standard convolution kernel, that is, skip some pixels in the input feature map to achieve capture information of a broader context, and obtain a shallow feature map of the label covering state and a semantic feature map of the label covering state.

[0045] In the embodiment of the present application, the label covering state multi-scale interaction subunit 1532 is used to obtain the label covering state multi-scale interaction feature map by passing the label covering state shallow feature map and the label covering state semantic feature map through the covering state multi-dimensional feature interaction encoder. Specifically, in an embodiment of the present application, the label-covered state multi-scale interaction subunit is used to: fuse the label-covered state shallow feature map and the label-covered state semantic feature map to obtain a label-covered state multi-scale feature map; use convolution layers with convolution kernels of 1×3, 3×1 and 3×3 to convolutionally encode the label-covered state multi-scale feature map to obtain a first label-covered state convolutional encoding feature map, a second label-covered state convolutional encoding feature map and a third label-covered state convolutional encoding feature map; calculate the mean of corresponding positions in the first label-covered state convolutional encoding feature map, the second label-covered state convolutional encoding feature map and the third label-covered state convolutional encoding feature map, and input the obtained label-covered state multi-scale convolutional encoding feature map into a hole convolutional coding module to obtain a label-covered state multi-scale perceptual feature map; perform residual connection on the label-covered state multi-scale convolutional encoding feature map and the label-covered state multi-scale perceptual feature map to obtain the label-covered state multi-scale interaction feature map.

[0046] It should be understood that in order to further interactively encode the two types of features, the shallow features of the label covering state and the semantic features of the label covering state, to generate a more comprehensive and representative feature, which helps the system to more accurately understand the actual covering of the label, in the technical solution of the present application, the shallow feature map of the label covering state and the semantic feature map of the label covering state are passed through the covering state multi-dimensional feature interactive encoder to obtain the multi-scale interactive feature map of the label covering state. Specifically, first, the shallow feature map of the label covering state (usually containing low-level visual information in the image, such as edges, corners, etc.) is fused with the semantic feature map (carrying higher-level information, such as object categories, positional relationships, etc.) to obtain preliminary multi-scale features. Then, in order to capture the feature changes of the label covering state in different directions, the fused feature map is encoded with convolution kernels of different shapes (1×3, 3×1, and 3×3). In particular, the 1×3 and 3×1 convolution kernels focus on features in the horizontal and vertical directions, respectively, while the 3×3 convolution kernel is used to capture more comprehensive spatial information in the local area of ​​the label coverage state. Afterwards, the feature information in different directions is integrated by calculating the mean of the convolved features at each corresponding position. Next, the multi-scale convolutional coding features generated above are input into a dilated convolutional coding module to better capture long-range dependencies, which is conducive to understanding the label coverage of the entire scene. Finally, the original multi-scale convolutional coding features are combined with the multi-scale perceptual features after dilated convolution processing through residual connections to finally obtain a multi-scale interactive feature representation that can reflect the quality of label coverage.

[0047] In an embodiment of the present application, the label semantics covering state interaction unit 154 is used to perform a multimodal prompt gate guided interaction on the label semantic feature vector and the label covering state multi-scale interactive feature map to obtain a label semantic guided label covering state cross-modal multidimensional representation feature map. It should be understood that the label semantic feature vector and the label covering state multi-scale interactive feature map respectively represent two different modal information (here is text / semantic information and image / visual information) data. In order to utilize the complementary advantages of information from different fields to improve the model's understanding of complex scenes, especially when the information of a single modality is not enough to make an accurate judgment. For example, in the label recognition task, it may be difficult to distinguish certain subtle differences relying solely on images, and adding semantic information can help the model better understand these differences. Therefore, in the technical solution of the present application, the label semantic feature vector and the label covering state multi-scale interactive feature map are subjected to a multimodal prompt gate guided interaction to obtain a label semantic guided label covering state cross-modal multidimensional representation feature map. In particular, by introducing the interactive mechanism guided by the multimodal cue gate, the advantages of each modality can be fully utilized, so that the generated feature map contains not only visual information but also richer semantic content. At the same time, the model can selectively focus on certain specific aspects during the processing, thereby improving the ability to understand complex scenes.

[0048] Specifically, Figure 4 FIG. 1 is a block diagram of a label semantics covering state interaction unit in a big data analysis system for an automobile system according to an embodiment of the present application. Figure 4As shown, the label semantic covering state interaction unit 154 includes: a label semantic autocorrelation encoding secondary subunit 1541, which is used to calculate the product between the label semantic feature vector and the transposed vector of the label semantic feature vector to obtain a label semantic autocorrelation encoding matrix; a label covering state feature map decoupling secondary subunit 1542, which is used to perform feature decoupling along the channel dimension on the label covering state multi-scale interaction feature map to obtain a set of label covering state local feature matrices; a label semantic-covering state weight calculation secondary subunit 1543, which is used to calculate the weights of the label semantic autocorrelation encoding matrix and each label covering state local feature matrix in the set of the label covering state local feature matrices. The interactive attention weights between them are used to obtain a set of label semantics-covering state cross-modal semantics interaction attention weights; the label semantics-covering state interaction secondary sub-unit 1544 is used to perform cross-modal interaction on the set of label semantics autocorrelation encoding matrices and the label covering state local feature matrices to obtain a set of label semantics-covering state cross-modal interaction local feature matrices; the label semantics-covering state interaction optimization secondary sub-unit 1545 is used to perform cross-modal interaction optimization on the set of label semantics-covering state cross-modal semantics interaction attention weights and the set of label semantics-covering state cross-modal interaction local feature matrices to obtain the label semantics-guided label covering state cross-modal multidimensional representation feature map.

[0049] Specifically, first, the product of the tag semantic feature vector and the transposed vector of the tag semantic feature vector is calculated to obtain a tag semantic autocorrelation encoding matrix. The above process can be expressed as: ;in, is the label semantic feature vector, is matrix multiplication, for The transposed vector of is the label semantic autocorrelation encoding matrix.

[0050] That is, the outer product concept in matrix operation is used to reveal the strength of the relationship between different label semantic feature elements within the label semantic feature vector. The label semantic autocorrelation encoding matrix generated by this calculation method not only contains the information of the original label semantic feature vector, but also contains the correlation information between label semantic features, which is conducive to the subsequent interaction with the local features of the label covering state.

[0051] Next, the multi-scale interactive feature map of the label covering state is subjected to feature decoupling along the channel dimension to obtain a set of local feature matrices of the label covering state. The above process can be expressed as follows: ;in, is the multi-scale interactive feature map of the label covering state, For Perform feature decoupling operations. and They are the first and the second in the set of local feature matrices of the label-covered state. and The local feature matrix of the label-covered state.

[0052] It should be understandable that the details of the label covering state may be reflected in specific local areas, and it is these details that determine whether the label covering meets the requirements. In order to allow the model to focus more accurately on the details of each local area of ​​the label covering state, thereby increasing the attention to the local details of the label covering state, it is necessary to perform feature decoupling along the channel dimension of the multi-scale interaction feature map of the label covering state. By decoupling the features along the channel dimension, the key information of the local area of ​​the label covering can be presented more clearly and independently, which increases the distinguishability of the local features of the label covering state and is more conducive to the implementation of fine-grained cross-modal interaction.

[0053] Specifically, in an embodiment of the present application, the label semantics-covering state weight calculation secondary subunit is used to: perform a linear transformation on the label semantics autocorrelation coding matrix to obtain a label semantics query coding matrix and a label semantics value coding matrix; perform converter cross-modal prompt information encoding on each label covering state local feature matrix in the set of the label semantics query coding matrix, the label semantics value coding matrix and the label covering state local feature matrix to obtain a set of label semantics-covering state cross-modal prompt information semantic coding matrices; perform gated decoding on the set of label semantics-covering state cross-modal prompt information semantic coding matrices to obtain a set of label semantics-covering state cross-modal semantic interaction attention weights.

[0054] Subsequently, the label semantic autocorrelation encoding matrix is ​​linearly transformed to obtain a label semantic query encoding matrix and a label semantic value encoding matrix. The above process can be expressed as follows: ;in, is the label semantic autocorrelation encoding matrix, is the query embedding matrix, is the label semantic query encoding matrix, is the value embedding matrix, is the label semantic value encoding matrix, is matrix multiplication.

[0055] That is, through linear mapping, the label semantic autocorrelation encoding matrix is ​​converted into a form suitable for subsequent attention mechanism calculation, where the calculated label semantic query encoding matrix is ​​used to find the correlation with the label covering state characteristics, while the label semantic value encoding matrix preserves the important information of the original label semantic features. This conversion is the key to achieving cross-modal interaction, which allows text / semantic information and image / visual information to be integrated in a unified way.

[0056] Then, each label covering state local feature matrix in the set of the label semantic query encoding matrix, the label semantic value encoding matrix and the label covering state local feature matrix is ​​respectively encoded with the converter cross-modal prompt information to obtain a set of label semantic-covering state cross-modal prompt information semantic encoding matrices. The above process can be expressed as: ;in, is the first in the set of local feature matrices of the label-covered state The local feature matrix of the label-covered state, is the label semantic query encoding matrix, is the label semantic value encoding matrix, is matrix multiplication, yes The transposed matrix of yes The scale is the matrix width multiplied by the matrix height, yes function, yes and The label semantics between the two - the semantic encoding matrix of the cross-modal prompt information of the covering state.

[0057] In this process, the converter can effectively capture the complex interaction patterns between the label semantic features and the local features of the label covering state through its core self-attention mechanism. The set of label semantics-covering state cross-modal prompt information semantic encoding matrices generated in this way not only retains the feature information of the original label semantics, but also integrates the supplementary information from the local features of the label covering state. In other words, the converter can establish a deep semantic connection between the two modalities of original text and image.

[0058] Next, the set of the label semantics-covering state cross-modal prompt information semantic encoding matrices is gated and decoded to obtain the set of label semantics-covering state cross-modal semantic interaction attention weights. The above process can be expressed as: ;in, yes function, yes and The label semantics-covered state cross-modal prompt information semantic encoding matrix between, is the decoding weight matrix, It is a decoder. yes Corresponding label semantics-covered state cross-modal semantic interaction attention weights.

[0059] That is, by gated decoding of the semantic encoding matrix of each label semantic-covering state cross-modal prompt information, the importance of label semantic features and label covering state features can be adjusted, which helps the model to allocate attention resources more flexibly to more accurately capture the complementarity between the two modal information of original text and image.

[0060] Then, the set of the label semantic autocorrelation encoding matrix and the label covering state local feature matrix is ​​cross-modally interacted to obtain the set of label semantic-covering state cross-modal interaction local feature matrices. And finally, the set of label semantic-covering state cross-modal semantic interaction attention weights and the set of label semantic-covering state cross-modal interaction local feature matrices are cross-modally optimized to obtain the label semantics-guided label covering state cross-modal multidimensional representation feature map. The above two steps can be comprehensively expressed by the formula: ;in, is the label semantic autocorrelation encoding matrix, and They are the first and the second in the set of local feature matrices of the label-covered state. and The local feature matrix of the label-covered state, and They are and The corresponding label semantics-covered state cross-modal semantic interaction attention weight, It is the point multiplication by position. is the characteristic coupling along the channel dimension, It is the label semantics that guides the cross-modal multi-dimensional representation feature map of the label covering state.

[0061] That is, in the step of cross-modal interaction between the label semantic autocorrelation encoding matrix and the set of label covering state local feature matrices, the complementary information of label semantic features and label covering state local features at the same spatial position can be effectively captured by position point multiplication, which is conducive to enhancing the spatial consistency of interactive feature representation. Finally, the obtained semantic interaction attention weights and interactive local features are combined to achieve weighted fusion of different modal features, generating a label semantic guided label covering state cross-modal multidimensional representation feature map with stronger expression ability and less redundant information.

[0062] In an embodiment of the present application, the recognition unit 155 is used to obtain a recognition result based on the label semantics to guide the cross-modal multidimensional representation feature map of the label covering state, and the recognition result is used to indicate whether the label covering state meets the requirements. Specifically, in an embodiment of the present application, the recognition unit is used to: pass the label semantics to guide the cross-modal multidimensional representation feature map of the label covering state through a label covering identifier based on a classifier to obtain the recognition result, and the recognition result is used to indicate whether the label covering state meets the requirements. It should be understood that a classifier is a machine learning model, and its main function is to divide it into different categories according to the input feature data. Specifically, it takes the label semantics to guide the cross-modal multidimensional representation feature map of the label covering state as input. This feature map contains rich information extracted from the label covering state image and the label reference image. The classifier will analyze and judge these complex features based on pre-trained rules and parameters, so as to determine whether the label covering state belongs to the two categories of "meeting the requirements" or "not meeting the requirements". In this way, automatic and intelligent label covering quality detection can be achieved. In particular, in an embodiment of the present application, a possible implementation method of passing the label semantics-guided label covering state cross-modal multidimensional representation feature graph through a label covering identifier based on a classifier to obtain a recognition result indicating whether the label covering state meets the requirements is as follows: each label semantics-guided label covering state cross-modal multidimensional representation feature matrix in the label semantics-guided label covering state cross-modal multidimensional representation feature graph is expanded into a one-dimensional feature vector according to a row vector or a column vector, and then cascaded to obtain a label covering state classification feature vector; the fully connected layer of the classifier is used to fully connect the label covering state classification feature vector to obtain a label covering state fully connected encoded classification feature vector; the label covering state fully connected encoded classification feature vector is input into the Softmax classification function of the classifier to obtain the recognition result.

[0063] In particular, when the label semantic feature vector and the label covering state multi-scale interaction feature map respectively represent the label semantic features of the label reference image and the multi-scale interaction image semantic features of the label covering state image, after performing cross-modal interaction optimization encoding based on the cross-modal prompt gate, the label semantics-guided label covering state cross-modal multidimensional representation feature map will also have significant interaction optimization distribution space structure differences due to the differences in cross-modal semantic prompts, affecting the convergence consistency of the classifier and thus affecting the accuracy of the recognition results obtained by the label covering identifier.

[0064] Therefore, in one example, the label semantics-guided label covering state cross-modal multidimensional representation feature graph is optimized, and the optimization process includes: calculating the absolute value sum of all feature values ​​of the label semantics-guided label covering state cross-modal multidimensional representation feature graph and the square root of the sum of squares to obtain the first label semantics-guided label covering state cross-modal multidimensional space structure value and the second label semantics-guided label covering state cross-modal multidimensional space structure value. The process can be expressed by the formula: ;in, Indicates each feature value of the cross-modal multi-dimensional representation feature map of the label semantics-guided label covering state, Indicates the cross-modal multidimensional space structure value of the first label semantics-guided label covering state, Represents the cross-modal multidimensional space structure value of the second label semantically guided label covering state.

[0065] Determine the total number of feature values ​​of all feature values ​​of the cross-modal multi-dimensional representation feature map of the label semantic guidance label covering state .

[0066] For each eigenvalue of the label semantics-guided label covering state cross-modal multidimensional representation feature map, the first label semantics-guided label covering state cross-modal multidimensional spatial structure value minus the product of the eigenvalue and the total number of eigenvalues ​​is calculated to obtain the first label semantics-guided label covering state cross-modal multidimensional long-range dependency value. The process can be expressed by the formula: ;in, Indicates each feature value of the cross-modal multi-dimensional representation feature map of the label semantics-guided label covering state, Indicates the cross-modal multidimensional space structure value of the first label semantics-guided label covering state, The total number of eigenvalues ​​of all eigenvalues ​​of the cross-modal multi-dimensional representation feature graph of the label semantics-guided label covering state, It represents the cross-modal multi-dimensional long-range dependency value of the label covering state guided by the semantics of the first label.

[0067] The product of the square root of the total number of eigenvalues ​​multiplied by the eigenvalues ​​minus the cross-modal multidimensional spatial structure value of the second label semantics-guided label covering state is calculated to obtain the cross-modal multidimensional long-range dependency value of the second label semantics-guided label covering state. The process can be expressed by the formula: ;in, The total number of eigenvalues ​​of all eigenvalues ​​of the cross-modal multi-dimensional representation feature graph of the label semantics-guided label covering state, Indicates each feature value of the cross-modal multi-dimensional representation feature map of the label semantics-guided label covering state, Represents the cross-modal multidimensional space structure value of the second label semantics-guided label covering state, It represents the cross-modal multi-dimensional long-range dependency value of the second label semantics-guided label covering state.

[0068] The exponential value calculated by taking the first label semantics-guided label covering state cross-modal multidimensional long-range dependency value as the exponent of the natural constant and the inverse of the second label semantics-guided label covering state cross-modal multidimensional long-range dependency value are weighted summed to obtain the optimized eigenvalue corresponding to each eigenvalue. The process can be expressed by the formula: ;in, It represents the multi-dimensional long-range dependency value of the label covering state guided by the semantics of the first label, Indicates the multi-dimensional long-range dependency value of the second label semantics-guided label covering state across modalities, and represents the weighted hyperparameter, It represents the optimized feature value corresponding to each feature value of the cross-modal multi-dimensional representation feature graph of the label semantic-guided label covering state.

[0069] The optimized feature values ​​are combined into an optimized label semantics-guided label covering state cross-modal multi-dimensional representation feature graph.

[0070] That is, in view of the possible spatial structure loss in the high-dimensional space of the feature set of the label semantics-guided label covering state cross-modal multidimensional representation feature graph, which leads to the weight of the classifier implicitly inferring the spatial structure information based on the features, resulting in inconsistent convergence, a long-distance feature dependency relationship is established based on the overall feature scale of the label semantics-guided label covering state cross-modal multidimensional representation feature graph relative to the spatial structure representation of the label semantics-guided label covering state cross-modal multidimensional representation feature graph, so as to establish the feature local connectivity of the label semantics-guided label covering state cross-modal multidimensional representation feature graph, and capture the spatial ambiguous information of the object feature value through the unstructured feature value point prediction of the label semantics-guided label covering state cross-modal multidimensional representation feature graph, thereby improving the spatial inductive bias perception ability of the feature set of the label semantics-guided label covering state cross-modal multidimensional representation feature graph, improving the convergence consistency of the classifier, and improving the accuracy of the recognition result obtained by the classifier-based label covering identifier of the label semantics-guided label covering state cross-modal multidimensional representation feature graph.

[0071] In summary, the label covering state judgment module 150 is clearly explained. It uses image processing and analysis technology based on machine learning and big data analysis to perform semantic recognition on the high-definition label reference image extracted from the label image database, and performs multi-scale extraction and multi-dimensional interaction on the collected label covering state image, so as to realize intelligent judgment on the label covering quality based on the semantic features of the label itself and the multi-scale interaction features under its covering state. In this way, the system can deeply analyze the content and type of the label, and identify its performance in a specific environment, while accurately detecting the subtle defects of the label text and the label covering state, so as to more accurately judge whether the label is correctly covered, thereby significantly improving the comprehensiveness and intelligence of the label covering state inspection.

[0072] In summary, the big data analysis system 100 for automobile systems based on the embodiment of the present application is explained, which first fixes the automobile parts to be labeled by rotating the clamp assembly, then the label selection assembly selects the target label that matches the part, and then the label stripper cooperates with the robot arm to tear off the target label from the label roll and accurately apply it to the designated marking area of ​​the part. After that, the camera detection component captures the label covering state image of the marking area, and finally identifies the label covering state based on the collected image to confirm whether the label is correctly applied as required. In this way, the automatic label covering quality detection of the automobile labeling system can be realized.

[0073] In an embodiment of the present application, a big data analysis method for an automobile system is also provided, which is used to execute the big data analysis system for an automobile system as described above. Figure 5 FIG. 1 is a flow chart of a big data analysis method for an automobile system according to an embodiment of the present application. Figure 5 As shown, in the big data analysis method for automobile systems, it includes: S110, taking the label covering state image as the image matching index, extracting a high-definition label reference image from the label image database; S120, performing label semantic recognition on the label reference image to obtain a label semantic feature vector; S130, performing multi-scale extraction and multi-dimensional feature interaction on the label covering state image to obtain a label covering state multi-scale interactive feature map; S140, performing multi-modal prompt gate guided interaction on the label semantic feature vector and the label covering state multi-scale interactive feature map to obtain a label semantic guided label covering state cross-modal multi-dimensional representation feature map; S150, based on the label semantic guided label covering state cross-modal multi-dimensional representation feature map, obtaining a recognition result, and the recognition result is used to indicate whether the label covering state meets the requirements.

[0074] Here, those skilled in the art can understand that the specific operations of each step in the above-mentioned big data analysis method for automobile system have been referred to above. Figures 1 to 4The big data analysis system for an automotive system has been described in detail, and therefore, its repeated description will be omitted.

[0075] In summary, the big data analysis method for automobile systems based on the embodiment of the present application is explained, which first fixes the automobile parts to be labeled by rotating the clamp assembly, then the label selection assembly selects the target label that matches the part, and then the label stripper cooperates with the robot arm to tear off the target label from the label roll and accurately apply it to the designated marking area of ​​the part. After that, the camera detection component captures the label covering state image of the marking area, and finally identifies the label covering state based on the collected image to confirm whether the label is correctly applied as required. In this way, the automatic label covering quality detection of the automobile labeling system can be realized.

Claims

1. A big data analysis system for an automotive system, wherein: The automobile system is an automobile labeling system, which is used to label various automobile parts. The automobile labeling system includes: a label stripper, a mechanical arm, a camera detection component, a label picking component, and a rotating fixture component. It is characterized in that the big data analysis system for the automobile system includes: a rotating fixture control module, which is used to control the rotating fixture component to fix the automobile parts to be labeled; a label picking module, which is used to control the label picking component to select a target label that is compatible with the automobile parts to be labeled; a target label covering module, which is used to control the label stripper and the mechanical arm to tear the target label from the label roll and cover the target label on the marking area of ​​the automobile parts to be labeled; a label covering state image acquisition module, which is used to control the camera detection component to collect the label covering state image of the marking area; a label covering state judgment module, which is used to perform label covering state recognition based on the label covering state image to judge whether the label covering state meets the requirements; Among them, the label covering state judgment module includes: a label reference image matching selection unit, which is used to extract a high-definition label reference image from a label image database using the label covering state image as an image matching index; a label semantic recognition unit, which is used to perform label semantic recognition on the label reference image to obtain a label semantic feature vector; a label state analysis unit, which is used to perform multi-scale extraction and multi-dimensional feature interaction on the label covering state image to obtain a label covering state multi-scale interaction feature map; a label semantic covering state interaction unit, which is used to perform multi-modal prompt gate-guided interaction on the label semantic feature vector and the label covering state multi-scale interaction feature map to obtain a label semantic-guided label covering state cross-modal multi-dimensional representation feature map; a recognition unit, which is used to obtain a recognition result based on the label semantic-guided label covering state cross-modal multi-dimensional representation feature map, and the recognition result is used to indicate whether the label covering state meets the requirements; Among them, the label state analysis unit includes: a label covering state image multi-scale feature extraction subunit, which is used to pass the label covering state image through a covering state multi-scale extractor based on a hollow pyramid network to obtain a label covering state shallow feature map and a label covering state semantic feature map; a label covering state multi-scale interaction subunit, which is used to pass the label covering state shallow feature map and the label covering state semantic feature map through a covering state multi-dimensional feature interaction encoder to obtain the label covering state multi-scale interaction feature map.

2. The big data analysis system for automobile systems according to claim 1, characterized in that: The label semantic recognition unit is used to: pass the label reference image through a label semantic recognizer based on a CNN-LSTM model to obtain the label semantic feature vector.

3. The big data analysis system for automobile systems according to claim 2, characterized in that: The label covering state multi-scale interaction sub-unit is used to: fuse the label covering state shallow feature map and the label covering state semantic feature map to obtain the label covering state multi-scale feature map; Convolutionally encode the multi-scale feature map of the label-covered state using convolutional layers with convolutional kernels of 1×3, 3×1, and 3×3, respectively, to obtain a first convolutional encoding feature map of the label-covered state, a second convolutional encoding feature map of the label-covered state, and a third convolutional encoding feature map of the label-covered state; calculate the means of corresponding positions in the first convolutional encoding feature map of the label-covered state, the second convolutional encoding feature map of the label-covered state, and the third convolutional encoding feature map of the label-covered state, and input the obtained multi-scale convolutional encoding feature map of the label-covered state into a hole convolutional coding module to obtain a multi-scale perceptual feature map of the label-covered state; The multi-scale convolutional coding feature map of the label-covered state and the multi-scale perceptual feature map of the label-covered state are residually connected to obtain the multi-scale interactive feature map of the label-covered state.

4. The big data analysis system for automobile systems according to claim 3, characterized in that: The label semantics covering state interaction unit includes: a label semantics autocorrelation encoding secondary subunit, which is used to calculate the product between the label semantic feature vector and the transposed vector of the label semantic feature vector to obtain a label semantics autocorrelation encoding matrix; a label covering state feature map decoupling secondary subunit, which is used to perform feature decoupling along the channel dimension on the label covering state multi-scale interaction feature map to obtain a set of label covering state local feature matrices; a label semantics-covering state weight calculation secondary subunit, which is used to calculate the interaction weight between the label semantics autocorrelation encoding matrix and each label covering state local feature matrix in the set of the label covering state local feature matrix. The second-level sub-unit of label semantics-covering state interaction is used to perform cross-modal interaction on the set of label semantics autocorrelation encoding matrix and the set of label covering state local feature matrix to obtain the set of label semantics-covering state cross-modal interaction local feature matrix; the second-level sub-unit of label semantics-covering state interaction optimization is used to perform cross-modal interaction optimization on the set of label semantics-covering state cross-modal semantic interaction attention weights and the set of label semantics-covering state cross-modal interaction local feature matrix to obtain the label semantics-guided label covering state cross-modal multidimensional representation feature map.

5. The big data analysis system for automobile systems according to claim 4, characterized in that: The label semantics-covering state weight calculation secondary subunit is used to: perform linear transformation on the label semantics autocorrelation encoding matrix to obtain a label semantics query encoding matrix and a label semantics value encoding matrix; perform converter cross-modal prompt information encoding on each label covering state local feature matrix in the set of the label semantics query encoding matrix, the label semantics value encoding matrix and the label covering state local feature matrix to obtain a set of label semantics-covering state cross-modal prompt information semantic encoding matrices; perform gated decoding on the set of label semantics-covering state cross-modal prompt information semantic encoding matrices to obtain a set of label semantics-covering state cross-modal semantic interaction attention weights.

6. The big data analysis system for automobile systems according to claim 5, characterized in that: The recognition unit is used to: pass the label semantics-guided label covering state cross-modal multi-dimensional representation feature map through a classifier-based label covering identifier to obtain the recognition result, and the recognition result is used to indicate whether the label covering state meets the requirements.

7. A big data analysis method for an automobile system, used to execute the big data analysis system for an automobile system as claimed in claim 1, characterized in that: The big data analysis method for automobile systems includes: taking the label-covered state image as an image matching index, extracting a high-definition label reference image from a label image database; performing label semantic recognition on the label reference image to obtain a label semantic feature vector; performing multi-scale extraction and multi-dimensional feature interaction on the label-covered state image to obtain a label-covered state multi-scale interactive feature map; performing multi-modal prompt gate-guided interaction on the label semantic feature vector and the label-covered state multi-scale interactive feature map to obtain a label semantic-guided label-covered state cross-modal multi-dimensional representation feature map; and obtaining a recognition result based on the label semantic-guided label-covered state cross-modal multi-dimensional representation feature map, wherein the recognition result is used to indicate whether the label-covered state meets the requirements.

8. The big data analysis method for automobile systems according to claim 7, characterized in that: Performing label semantic recognition on the label reference image to obtain a label semantic feature vector includes: passing the label reference image through a label semantic recognizer based on a CNN-LSTM model to obtain the label semantic feature vector.

Images