A wheel hub detection visual angle image generation method, system and storage medium

Abstract

This invention discloses a method, system, and storage medium for generating perspective images for wheel hub inspection, relating to the field of wheel hub inspection. The method includes: acquiring original wheel hub images acquired by multiple image acquisition devices located at a wheel hub inspection station; and generating a corresponding wheel hub perspective inspection image for each image acquisition device based on the original wheel hub image and a pre-determined wheel hub perspective mapping relationship. This method, by pre-establishing a wheel hub perspective mapping relationship and ensuring that corresponding positioning points in the wheel hub perspective inspection images from different image acquisition devices are symmetrically distributed relative to the central axis of the wheel hub in the wheel hub inspection coordinate system, allows the images acquired by each image acquisition device to be uniformly converted to a coordinate system based on the geometric features of the wheel hub itself.

Description

Technical Field

[0001] This invention relates to the field of wheel hub inspection, and in particular to a method, system, and storage medium for generating viewpoint images for wheel hub inspection. Background Technology

[0002] The surface quality of wheel hubs directly affects driving safety, and comprehensive defect inspection is required before leaving the factory. Current technology typically involves deploying multiple image acquisition devices around the wheel hub inspection station to obtain multi-view images, but the following problems still exist: First, the wheel hub is a three-dimensional rotating structure with large differences in spatial orientation in different areas. There is a lack of a unified coordinate reference between images acquired independently by different image acquisition devices, making it difficult to ensure that the same wheel hub feature in images from different perspectives corresponds accurately in spatial position. This results in the loss of perspective in areas such as the spoke angle region and the inner curved surface of the rim.

[0003] Second, in existing technologies, the arrangement of multiple image acquisition devices is usually done by uniform distribution or manual experience, which lacks systematic matching between the three-dimensional structure of the wheel hub and the field of view of each image acquisition device. On the one hand, this results in viewpoint redundancy in some areas and wastes hardware resources. On the other hand, unreasonable viewpoint distribution may lead to insufficient coverage in some areas, which needs to be compensated by increasing the number of devices, thus increasing the system cost.

[0004] Third, in industrial inspection environments, image acquisition equipment operates continuously for extended periods, inevitably leading to malfunctions such as connection drops, abnormal frame rates, and degraded image quality. Existing detection technologies are limited in scope and cannot comprehensively cover the various types of malfunctions at the hardware, data, and software levels, making them prone to missed detections or misjudgments. Furthermore, they lack grading standards based on the severity of malfunctions.

[0005] Fourth, existing technologies can only issue simple alarms or stop detection after detecting the failure of image acquisition equipment. They lack the ability to reschedule and replan the paths of the remaining available equipment. As a result, the detection task has to be interrupted until manual intervention is required, which seriously restricts the reliability of wheel hub inspection systems in unmanned continuous production lines. Summary of the Invention

[0006] The purpose of this invention is to provide a method, system, and storage medium for generating viewpoint images for wheel hub inspection, in order to solve the problems mentioned in the background art.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a method for generating a viewpoint image for wheel hub detection, comprising: Acquire raw wheel hub images from multiple image acquisition devices located at the wheel hub inspection station; For each of the image acquisition devices, a corresponding wheel hub perspective detection image is generated based on the original wheel hub image and a pre-determined wheel hub perspective mapping relationship. The wheel hub perspective mapping relationship is the correspondence between the positioning points in the wheel hub perspective detection image and the corresponding points in the original wheel hub image; the corresponding positioning points in the wheel hub perspective detection images corresponding to different image acquisition devices are symmetrically distributed with respect to the central axis of the wheel hub in the wheel hub detection coordinate system.

[0008] Preferably, several feature points on the surface of the wheel hub are selected as positioning points in the wheel hub detection coordinate system. The positioning points are evenly distributed along the circumference of the wheel hub and are symmetrical with respect to the central axis of the wheel hub. Establish the three-dimensional coordinates of each positioning point The corresponding pixel position of this positioning point in the original wheel hub image The mapping relationship between them satisfies: ;in, It is a 3×4 projection matrix; the projection matrix Obtained through calibration; Based on the mapping relationship, a wheel hub view lookup table is generated. The lookup table records the mapping relationship between each pixel position in the wheel hub view detection image and the corresponding pixel position in the original wheel hub image.

[0009] Preferably, the number and spatial pose of the multiple image acquisition devices set at the wheel hub inspection station are determined by a path planning algorithm based on the three-dimensional model of the wheel hub and the field of view of the image acquisition devices, so that the original wheel hub images acquired by the multiple image acquisition devices in their respective acquisition poses can cover the entire surface of the wheel hub to be inspected. The path planning algorithm includes the following steps: S1. Obtain the triangular facet model of the wheel hub to be detected, perform Gaussian mapping on the triangular facet model, and map the normal vector of each triangular facet of the triangular facet model onto the Gaussian sphere to form a Gaussian point cloud. S2. Based on the angle between the normal vectors and the topological adjacency relationship between Gaussian points in the Gaussian point cloud, the triangular patch model is divided into several independent regions, such that the angle between the normal vectors of any two Gaussian points in the same independent region is less than a preset angle threshold, and the triangular patches in the independent region are topologically adjacent. S3. For each independent region, calculate the average normal vector of all triangular facet normal vectors within that independent region. Find adjacent independent regions whose angle with the average normal vector is smaller than the field of view of the image acquisition device, and merge the found adjacent independent regions with the corresponding independent regions to obtain several merged detection regions. S4. Construct the OBB bounding box for each detection region. For each OBB bounding box, select the candidate viewpoints that can be completely surrounded by the visual cone model of the image acquisition device from among the candidate viewpoints uniformly sampled on a sphere with the center of the OBB bounding box as the center and a preset radius. S5. Perform occlusion detection on the selected candidate viewpoints: Starting from the candidate viewpoint, emit rays to each vertex of the OBB bounding box. If the ray intersects with other triangular facets in the triangular facet model other than the detection area where the OBB bounding box is located, it is determined that the candidate viewpoint is occluded. Adjust the spatial pose of the candidate viewpoint until there is no occlusion. Use the adjusted candidate viewpoint as the acquisition pose of the image acquisition device corresponding to the detection area. S6. Summarize the acquisition poses corresponding to each detection area to obtain the number and spatial pose of the image acquisition devices.

[0010] Preferably, after acquiring the original wheel hub images acquired by multiple image acquisition devices located at the wheel hub inspection station, the method further includes: Obtain the operating status information of each of the image acquisition devices; Based on the operating status information, a comprehensive failure detection is performed on each of the image acquisition devices; If any of the image acquisition devices is detected to be malfunctioning, a corresponding processing strategy will be executed according to the severity of the malfunction.

[0011] Preferably, the comprehensive failure detection includes at least two of the following: Hardware status detection: Obtain the connection status, operating temperature, power supply voltage and real-time frame rate of the image acquisition device; when a connection is broken, the operating temperature exceeds the preset temperature range, the power supply voltage deviates from the rated voltage by more than the preset voltage deviation range, or the real-time frame rate is lower than the product of the set frame rate and the preset frame rate ratio threshold, it is determined that the image acquisition device has a hardware abnormality. Image quality detection: Calculate the sharpness index of the original wheel hub image. Average brightness and noise level The clarity index adopts Calculate, when When the average brightness is below a first preset threshold, the clarity is deemed substandard; the average brightness is determined using... Calculate, when A brightness abnormality is determined when the noise level is below a second preset threshold or above a third preset threshold; the noise level is determined using... Calculation, where ,when The noise level is considered excessive when it exceeds the fourth preset threshold. Event log detection: Read the operation log of the image acquisition device and check whether there are buffer overflow event records, image acquisition job interruption event records, or external trigger signal missing event records in the log; Multi-view cross-validation: When there is an overlapping area between the original wheel hub images acquired by multiple image acquisition devices, feature points in the overlapping area of ​​each image are extracted and feature matching is performed; the images of one of the image acquisition devices are removed in turn, and the overlapping area is reconstructed in three dimensions using the remaining images and the reprojection error is calculated with the images that have not been removed; when the reprojection error after removing a certain device increases by more than a fifth preset threshold compared with before removal, the removed device is determined to be invalid.

[0012] Preferably, based on the detection results of the comprehensive failure detection, the severity of the failure is determined according to the following rules: When the detection result is a single frame loss or a single image processing timeout, the severity of the failure is determined to be a minor anomaly. When the severity of the failure is a minor anomaly, the current frame is discarded and the log is recorded without interrupting the detection process. When the detection result is that the image acquisition device is disconnected but can be reconnected, or the inference engine inside the image acquisition device is malfunctioning, the severity of the failure is determined to be moderate. When the severity of the failure is moderate, the acquisition channel of the failed image acquisition device is suspended, the computing and storage resources occupied by the channel are released, and automatic reconnection is attempted according to the preset exponential backoff strategy until the reconnection is successful or the number of reconnection reaches the sixth preset threshold, after which it is determined to be severe. When the detection result indicates that the image acquisition device hardware is permanently damaged, or the automatic reconnection fails after the moderate anomaly handling stage, the severity of the failure is determined to be a severe anomaly. When the severity of the failure is a severe anomaly, the failed image acquisition device is marked, and the path planning algorithm is re-executed based on the currently available image acquisition devices to determine the number and spatial pose of the updated image acquisition devices, and the updated image acquisition devices are controlled to re-acquire the original wheel hub image. When the detection result is system memory overflow or main process unresponsive, the severity of the failure is determined to be a fatal exception. When the severity of the failure is a fatal exception, a high-level alarm is triggered, the current system state and error context are saved to non-volatile storage media, and the watchdog timer triggers a hardware reset to restart the system or process when the system unresponsiveness exceeds the seventh preset threshold.

[0013] Preferably, generating a wheel hub perspective detection image based on the original wheel hub image and a pre-determined wheel hub perspective mapping relationship includes: Determine the size and resolution of the target wheel hub perspective detection image; Based on the mapping relationship recorded in the wheel hub perspective mapping relationship, find the pixel position of each pixel in the target wheel hub perspective detection image in the original wheel hub image; For non-integer pixel positions after mapping, the pixel value at that position is calculated using interpolation. The calculated pixel value is then assigned to the corresponding pixel in the target wheel hub perspective detection image to generate the wheel hub perspective detection image.

[0014] Preferably, the wheel hub view lookup table records the mapping relationship between the position of each pixel in the wheel hub view detection image and the corresponding sub-pixel coordinate position in the original wheel hub image.

[0015] A system for generating viewpoint images for wheel hub detection, used to perform an image generation method, includes: The image acquisition module is used to acquire the original wheel hub images captured by multiple image acquisition devices set up at the wheel hub inspection station; The image generation module is used to generate a wheel hub perspective detection image corresponding to each of the image acquisition devices, based on the original wheel hub image and a pre-determined wheel hub perspective mapping relationship. The wheel hub perspective mapping relationship is the correspondence between the positioning points in the wheel hub perspective detection image and the corresponding points in the original wheel hub image; the corresponding positioning points in the wheel hub perspective detection images corresponding to different image acquisition devices are symmetrically distributed with respect to the central axis of the wheel hub in the wheel hub detection coordinate system.

[0016] A computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of an image generation method.

[0017] The technical effects and advantages of this invention are as follows: This wheel hub inspection method generates viewpoint images by pre-establishing a wheel hub viewpoint mapping relationship and ensuring that the corresponding positioning points in the wheel hub viewpoint inspection images from different image acquisition devices are symmetrically distributed relative to the central axis of the wheel hub in the wheel hub inspection coordinate system. This allows the images acquired by each image acquisition device to be uniformly transformed into a coordinate system based on the geometric features of the wheel hub itself, ensuring that each viewpoint image covers the corresponding circumferentially symmetrical area of ​​the wheel hub surface, avoiding the loss of viewpoints in some areas, and providing multi-view inspection images with good spatial consistency and clear positional correspondence for subsequent image stitching and defect detection.

[0018] This wheel hub inspection method uses a viewpoint image generation approach. Based on the 3D model of the wheel hub and the field of view of the image acquisition devices, a path planning algorithm determines the number and spatial pose of multiple image acquisition devices at the wheel hub inspection station. This ensures that the images acquired by each device in its respective acquisition pose cover the entire surface of the wheel hub to be inspected. This approach avoids viewpoint redundancy caused by manual placement or uniform sampling in existing technologies. It completes the inspection task with the minimum number of devices while ensuring complete coverage, reducing hardware costs and station space occupancy.

[0019] This wheel hub inspection method uses a perspective image generation approach. It comprehensively employs at least two of the following detection methods—hardware status detection, image quality detection, event log detection, and multi-view cross-validation—to detect failures in image acquisition equipment. It comprehensively assesses the equipment's health status from four dimensions: physical layer, data layer, software layer, and redundancy check layer, overcoming the shortcomings of single-dimensional detection which is prone to missed detections or misjudgments. Furthermore, based on the detection results, it classifies the faults according to specific rules: minor anomalies are classified as single frame drops or timeouts; moderate anomalies are classified as equipment reconnection possible or inference engine malfunctions; severe anomalies are classified as permanent hardware damage or reconnection failures; and fatal anomalies are classified as system memory overflows or main process unresponsiveness. This allows the system to handle faults differently based on their specific circumstances.

[0020] This wheel hub inspection system uses a viewpoint image generation method and a hierarchical processing strategy. For minor anomalies, the current frame is discarded while the process continues to ensure uninterrupted operation. For moderate anomalies, the acquisition channel is paused and automatically reconnected according to an exponential backoff strategy to attempt recovery. For severe anomalies, failed equipment is marked and the path planning module is triggered to re-execute path planning based on currently available equipment, enabling the remaining equipment to effectively cover the entire surface of the wheel hub to be inspected, preventing the entire line from shutting down due to the failure of a single device. For fatal anomalies, alarms are triggered, the error context is saved, and a watchdog timer triggers a hardware reset to ensure system recoverability. This hierarchical strategy achieves a reasonable balance between ensuring inspection continuity and system stability, significantly improving the reliability and adaptability of the wheel hub inspection system on unmanned production lines. Attached Figure Description

[0021] Figure 1 This is a flowchart of the overall method of the present invention; Figure 2 This is a flowchart showing the detailed steps of path planning in this invention; Figure 3 A detailed flowchart of the steps for generating the detection image in this invention is provided. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] This invention provides, for example Figures 1-3 The method for generating a viewpoint image for wheel hub detection, as shown, includes: Acquire raw wheel hub images from multiple image acquisition devices located at the wheel hub inspection station; For each of the image acquisition devices, a corresponding wheel hub perspective detection image is generated based on the original wheel hub image and a pre-determined wheel hub perspective mapping relationship. The wheel hub perspective mapping relationship is the correspondence between the positioning points in the wheel hub perspective detection image and the corresponding points in the original wheel hub image; the corresponding positioning points in the wheel hub perspective detection images corresponding to different image acquisition devices are symmetrically distributed with respect to the central axis of the wheel hub in the wheel hub detection coordinate system.

[0024] By pre-establishing a wheel hub perspective mapping relationship and ensuring that the corresponding positioning points in the wheel hub perspective detection images from different image acquisition devices are symmetrically distributed relative to the central axis of the wheel hub in the wheel hub detection coordinate system, the images acquired by each image acquisition device can be uniformly transformed into a coordinate system based on the geometric features of the wheel hub itself. This ensures that each perspective image covers the corresponding circumferentially symmetrical area of ​​the wheel hub surface, avoids missing perspectives in some areas, and provides multi-view detection images with good spatial consistency and clear positional correspondence for subsequent image stitching and defect detection.

[0025] Furthermore, several feature points on the surface of the wheel hub are selected as positioning points in the wheel hub detection coordinate system. The positioning points are evenly distributed along the circumference of the wheel hub and are symmetrical with respect to the central axis of the wheel hub. Establish the three-dimensional coordinates of each positioning point The corresponding pixel position of this positioning point in the original wheel hub image The mapping relationship between them satisfies: ;in, It is a 3×4 projection matrix; the projection matrix Obtained through calibration; Based on the mapping relationship, a wheel hub view lookup table is generated. The lookup table records the mapping relationship between each pixel position in the wheel hub view detection image and the corresponding pixel position in the original wheel hub image.

[0026] Furthermore, the number and spatial pose of the multiple image acquisition devices set at the wheel hub inspection station are determined by the path planning algorithm based on the three-dimensional model of the wheel hub and the field of view of the image acquisition devices, so that the original wheel hub images acquired by the multiple image acquisition devices in their respective acquisition poses can cover the entire surface of the wheel hub to be inspected. The path planning algorithm includes the following steps: S1. Obtain the triangular facet model of the wheel hub to be detected, perform Gaussian mapping on the triangular facet model, and map the normal vector of each triangular facet of the triangular facet model onto the Gaussian sphere to form a Gaussian point cloud. S2. Based on the angle between the normal vectors and the topological adjacency relationship between Gaussian points in the Gaussian point cloud, the triangular patch model is divided into several independent regions, such that the angle between the normal vectors of any two Gaussian points in the same independent region is less than a preset angle threshold, and the triangular patches in the independent region are topologically adjacent. S3. For each independent region, calculate the average normal vector of all triangular facet normal vectors within that independent region. Find adjacent independent regions whose angle with the average normal vector is smaller than the field of view of the image acquisition device, and merge the found adjacent independent regions with the corresponding independent regions to obtain several merged detection regions. S4. Construct the OBB bounding box for each detection region. For each OBB bounding box, select the candidate viewpoints that can be completely surrounded by the visual cone model of the image acquisition device from among the candidate viewpoints uniformly sampled on a sphere with the center of the OBB bounding box as the center and a preset radius. S5. Perform occlusion detection on the selected candidate viewpoints: Starting from the candidate viewpoint, emit rays to each vertex of the OBB bounding box. If the ray intersects with other triangular facets in the triangular facet model other than the detection area where the OBB bounding box is located, it is determined that the candidate viewpoint is occluded. Adjust the spatial pose of the candidate viewpoint until there is no occlusion. Use the adjusted candidate viewpoint as the acquisition pose of the image acquisition device corresponding to the detection area. S6. Summarize the acquisition poses corresponding to each detection area to obtain the number and spatial pose of the image acquisition devices.

[0027] Furthermore, after acquiring the original wheel hub images captured by multiple image acquisition devices set up at the wheel hub inspection station, the method further includes: Obtain the operating status information of each of the image acquisition devices; Based on the operating status information, a comprehensive failure detection is performed on each of the image acquisition devices; If any of the image acquisition devices is detected to be malfunctioning, a corresponding processing strategy will be executed according to the severity of the malfunction.

[0028] Furthermore, the comprehensive failure detection includes at least two of the following: Hardware status detection: Obtain the connection status, operating temperature, power supply voltage and real-time frame rate of the image acquisition device; when a connection is broken, the operating temperature exceeds the preset temperature range, the power supply voltage deviates from the rated voltage by more than the preset voltage deviation range, or the real-time frame rate is lower than the product of the set frame rate and the preset frame rate ratio threshold, it is determined that the image acquisition device has a hardware abnormality. Image quality detection: Calculate the sharpness index of the original wheel hub image. Average brightness and noise level The clarity index adopts Calculate, when When the average brightness is below a first preset threshold, the clarity is deemed substandard; the average brightness is determined using... Calculate, when A brightness abnormality is determined when the noise level is below a second preset threshold or above a third preset threshold; the noise level is determined using... Calculation, where ,when The noise level is considered excessive when it exceeds the fourth preset threshold. Event log detection: Read the operation log of the image acquisition device and check whether there are buffer overflow event records, image acquisition job interruption event records, or external trigger signal missing event records in the log; Multi-view cross-validation: When there is an overlapping area between the original wheel hub images acquired by multiple image acquisition devices, feature points in the overlapping area of ​​each image are extracted and feature matching is performed; the images of one of the image acquisition devices are removed in turn, and the overlapping area is reconstructed in three dimensions using the remaining images and the reprojection error is calculated with the images that have not been removed; when the reprojection error after removing a certain device increases by more than a fifth preset threshold compared with before removal, the removed device is determined to be invalid.

[0029] Furthermore, based on the detection results of the comprehensive failure detection, the severity of the failure is determined according to the following rules: When the detection result is a single frame loss or a single image processing timeout, the severity of the failure is determined to be a minor anomaly. When the severity of the failure is a minor anomaly, the current frame is discarded and the log is recorded without interrupting the detection process. When the detection result is that the image acquisition device is disconnected but can be reconnected, or the inference engine inside the image acquisition device is malfunctioning, the severity of the failure is determined to be moderate. When the severity of the failure is moderate, the acquisition channel of the failed image acquisition device is suspended, the computing and storage resources occupied by the channel are released, and automatic reconnection is attempted according to the preset exponential backoff strategy until the reconnection is successful or the number of reconnection reaches the sixth preset threshold, after which it is determined to be severe. When the detection result indicates that the image acquisition device hardware is permanently damaged, or the automatic reconnection fails after the moderate anomaly handling stage, the severity of the failure is determined to be a severe anomaly. When the severity of the failure is a severe anomaly, the failed image acquisition device is marked, and the path planning algorithm is re-executed based on the currently available image acquisition devices to determine the number and spatial pose of the updated image acquisition devices, and the updated image acquisition devices are controlled to re-acquire the original wheel hub image. When the detection result is system memory overflow or main process unresponsive, the severity of the failure is determined to be a fatal exception. When the severity of the failure is a fatal exception, a high-level alarm is triggered, the current system state and error context are saved to non-volatile storage media, and the watchdog timer triggers a hardware reset to restart the system or process when the system unresponsiveness exceeds the seventh preset threshold.

[0030] Furthermore, generating a wheel hub perspective detection image based on the original wheel hub image and a pre-determined wheel hub perspective mapping relationship includes: Determine the size and resolution of the target wheel hub perspective detection image; Based on the mapping relationship recorded in the wheel hub perspective mapping relationship, find the pixel position of each pixel in the target wheel hub perspective detection image in the original wheel hub image; For non-integer pixel positions after mapping, the pixel value at that position is calculated using interpolation. The calculated pixel value is then assigned to the corresponding pixel in the target wheel hub perspective detection image to generate the wheel hub perspective detection image.

[0031] Furthermore, the wheel hub view lookup table records the mapping relationship between the position of each pixel in the wheel hub view detection image and the corresponding sub-pixel coordinate position in the original wheel hub image.

[0032] Based on the 3D model of the wheel hub and the field of view of the image acquisition devices, a path planning algorithm determines the number and spatial pose of multiple image acquisition devices at the wheel hub inspection station. This ensures that the images acquired by each device in its respective acquisition pose cover the entire surface of the wheel hub to be inspected. This approach avoids viewpoint redundancy caused by manual placement or uniform sampling in existing technologies. It completes the inspection task with the minimum number of devices while ensuring complete coverage, thus reducing hardware costs and station space occupancy.

[0033] By comprehensively employing at least two of the following detection methods—hardware status detection, image quality detection, event log detection, and multi-view cross-validation—to perform failure detection on image acquisition devices, the system comprehensively assesses the device's health status from four dimensions: physical layer, data layer, software layer, and redundancy check layer. This overcomes the shortcomings of relying on a single detection dimension, which can easily lead to missed detections or false positives. Furthermore, based on the detection results, the system is further categorized into three levels: minor anomalies (single frame loss or timeout), moderate anomalies (device reconnection capability or inference engine malfunction), severe anomalies (permanent hardware damage or reconnection failure), and fatal anomalies (system memory overflow or main process unresponsiveness). This allows the system to handle faults differently depending on their specific circumstances.

[0034] Through a tiered processing strategy, minor anomalies are handled by discarding the current frame and continuing operation to ensure uninterrupted workflow. Moderate anomalies involve pausing the acquisition channel and automatically reconnecting to attempt recovery using an exponential backoff strategy. Severe anomalies involve marking failed devices and triggering the path planning module to re-execute path planning based on currently available devices, enabling the remaining devices to effectively cover the entire surface of the wheel hub to be inspected, preventing a single device failure from causing a complete line shutdown. Fatal anomalies trigger alarms, save the error context, and are resolved by a watchdog timer-triggered hardware reset to ensure system recoverability. This tiered strategy achieves a reasonable balance between ensuring continuous inspection and system stability, significantly improving the reliability and adaptability of the wheel hub inspection system on unmanned production lines.

[0035] A system for generating viewpoint images for wheel hub detection, used to perform an image generation method, includes: The image acquisition module is used to acquire the original wheel hub images captured by multiple image acquisition devices set up at the wheel hub inspection station; The image generation module is used to generate a wheel hub perspective detection image corresponding to each of the image acquisition devices, based on the original wheel hub image and a pre-determined wheel hub perspective mapping relationship. The wheel hub perspective mapping relationship is the correspondence between the positioning points in the wheel hub perspective detection image and the corresponding points in the original wheel hub image; the corresponding positioning points in the wheel hub perspective detection images corresponding to different image acquisition devices are symmetrically distributed with respect to the central axis of the wheel hub in the wheel hub detection coordinate system.

[0036] A computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of an image generation method.

[0037] The workflow is as follows: Phase 1: Preparing the Wheel Hub Perspective Mapping Relationship This stage is completed offline during system deployment and is used to establish the mapping relationship between the three-dimensional coordinates of the wheel hub surface and the two-dimensional pixel positions of the original wheel hub image, providing a coordinate transformation benchmark for the generation of subsequent viewpoint images.

[0038] Step 1.1: Select the positioning point In the wheel hub inspection coordinate system, several feature points on the wheel hub surface are selected as positioning points (such as spoke edge points, bolt hole center points, rim edge points, etc., which have clear geometric significance). These positioning points are evenly distributed along the circumference of the wheel hub and are symmetrical in spatial position with respect to the central axis of the wheel hub.

[0039] Step 1.2: Establish the mapping relationship between 3D coordinates and 2D pixel positions. Let the three-dimensional homogeneous coordinates of a certain positioning point be... The pixel position corresponding to this positioning point in the original wheel hub image is: The mapping relationship between the two satisfies: ; in, The projection matrix is ​​3×4, obtained through calibration. During calibration, a checkerboard calibration board can be used at the wheel hub inspection station to calibrate the image acquisition equipment and obtain the projection matrix of each image acquisition device. The specific value.

[0040] Step 1.3: Generate a wheel hub view lookup table Based on the above mapping relationship, a wheel hub view lookup table is generated. This lookup table records the mapping relationship between each pixel position in the wheel hub view detection image and the corresponding pixel position (sub-pixel coordinates) in the original wheel hub image, which can be quickly looked up when generating subsequent view images.

[0041] Phase Two: Path Planning to Determine the Layout of Image Acquisition Equipment This stage is completed offline during system deployment and is used to determine the number and spatial orientation of image acquisition devices that need to be placed at the wheel hub inspection station to ensure that all surfaces to be inspected can be covered.

[0042] Step 2.1: Obtain the 3D model of the wheel hub and generate a Gaussian point cloud. Obtain the triangular facet model of the wheel hub to be inspected as the 3D model of the wheel hub. Perform Gaussian mapping on the triangular facet model, mapping the normal vector of each triangular facet onto a Gaussian sphere to form a Gaussian point cloud.

[0043] Step 2.2: Divide into independent areas Based on the angle between the normal vectors of Gaussian points in the Gaussian point cloud and their topological adjacency, the triangular patch model is divided into several independent regions. The division criterion is: the angle between the normal vectors of any two Gaussian points within the same independent region is less than a preset angle threshold. Furthermore, the triangular facets within this independent region are topologically adjacent.

[0044] Step 2.3: Region Merging For each independent region, calculate the average normal vector of all triangular facet normals within that region. Find the angle between the average normal vector and the image acquisition device's field of view. The adjacent independent regions are merged with the current region to obtain several merged detection regions.

[0045] Step 2.4: Generate candidate viewpoints Construct bounding boxes for each detection region using OBBs (Directed Bounding Boxes). For each OBB bounding box, define a sphere with the center of the OBB bounding box as the center and a preset radius. Candidate viewpoints are uniformly sampled on the sphere, and candidate viewpoints from which the visible cone model of the image acquisition device can completely enclose the OBB bounding box are selected.

[0046] Step 2.5: Occlusion Detection and Pose Adjustment Occlusion detection is performed on the selected candidate viewpoints: Starting from the candidate viewpoint, rays are emitted to each vertex of the OBB bounding box. If a ray intersects with any other triangle in the triangle model except for the detection area where the OBB bounding box is located, then the candidate viewpoint is determined to be occluded.

[0047] For candidate viewpoints with occlusion, their spatial pose (position and orientation) is adjusted until there is no occlusion from that viewpoint. The adjusted candidate viewpoint is then used as the acquisition pose of the image acquisition device corresponding to the detection area.

[0048] Step 2.6: Summarize the output By summarizing the acquisition poses corresponding to each detection area, the number of image acquisition devices required to be arranged at the wheel hub inspection station and their respective spatial poses are obtained.

[0049] Phase 3: Image Acquisition and Comprehensive Failure Detection This stage is executed online during each wheel hub inspection, controlling each image acquisition device to acquire original wheel hub images, and performing comprehensive failure detection on the acquisition process and results.

[0050] Step 3.1: Perform image acquisition Multiple image acquisition devices located at the wheel hub inspection station are controlled to synchronously or sequentially acquire original wheel hub images in their respective acquisition poses.

[0051] Each image acquisition device acquires original images of different detection areas of the wheel hub. The corresponding positioning points in the wheel hub perspective detection images from different image acquisition devices are symmetrically distributed with respect to the central axis of the wheel hub in the wheel hub detection coordinate system.

[0052] Step 3.2: Obtain running status information While or after acquiring the original wheel hub image, obtain the operating status information of each image acquisition device.

[0053] Step 3.3: Comprehensive Failure Detection Based on the operational status information, a comprehensive failure detection is performed on each image acquisition device. The comprehensive failure detection utilizes at least two of the following four methods: (1) Hardware status detection: Acquire the connection status, operating temperature, power supply voltage, and real-time frame rate of the image acquisition device. If any of the following conditions are detected, the device is deemed to have a hardware malfunction: Connection lost; operating temperature exceeds preset temperature range; power supply voltage deviates from rated voltage by more than preset voltage deviation range; real-time frame rate is lower than the product of set frame rate and preset frame rate ratio threshold.

[0054] (2) Image quality detection: Calculate the sharpness index of the original wheel hub image. Average brightness and noise level .

[0055] Clarity index uses Calculate; when Below the first preset threshold The image clarity was deemed insufficient at that time. Average brightness is adopted Calculate; when Below the second preset threshold Or higher than the third preset threshold The brightness is determined to be abnormal at that time; Noise level adopted Calculation, where ,when Higher than the fourth preset threshold The noise level was determined to be excessive at that time. (3) Event log detection: Read the operation log of the image acquisition device and check if the following events are recorded in the log: Buffer overflow event; image acquisition job interruption event; external trigger signal missing event.

[0056] (4) Multi-view cross-validation: When there is an overlapping area between the original wheel hub images acquired by multiple image acquisition devices, feature points within the overlapping area of ​​each image are extracted and feature matching is performed. Images from one of the image acquisition devices are sequentially removed, and the remaining images are used to perform 3D reconstruction of the overlapping area. The reprojection error is then calculated by comparing the reprojection error with the remaining images. If the reprojection error increases by more than a fifth preset threshold after removing an image from a particular device, the reprojection error is considered complete. If so, the removed device is deemed invalid.

[0057] Step 3.4: Determine the severity of the failure Based on the results of the comprehensive failure detection, the severity of the failure is determined according to the following rules: Minor anomalies: The detection result is a single frame loss or a single image processing timeout; Moderate anomaly: The detection result is that the image acquisition device is disconnected but can be reconnected, or the inference engine inside the image acquisition device is malfunctioning. Serious anomaly: The detection result indicates that the image acquisition device hardware is permanently damaged, or that automatic reconnection failed after the moderate anomaly handling stage; Fatal anomaly: The detection result is system memory overflow or main process unresponsive.

[0058] If no anomalies are detected, proceed directly to the fifth stage (view image generation).

[0059] Phase 4: Graded Processing Based on the severity of the failure determined in step 3.4, this stage executes the corresponding handling strategy.

[0060] Scenario 1: Minor abnormality When the failure severity is a minor anomaly (single frame loss or single image processing timeout): The current frame is discarded and logged; the detection process is not interrupted and the system continues to run.

[0061] Scenario 2: Moderate abnormality When the failure severity is moderate (device connection is lost but can be reconnected, or inference engine malfunctions): Suspend the acquisition channel of the malfunctioning image acquisition device; release the computing and storage resources occupied by the channel; attempt automatic reconnection according to the preset exponential backoff strategy (reconnection intervals are 1s, 2s, 4s, 8s, ...); if reconnection is successful, resume normal acquisition of the channel; if the number of reconnections reaches the sixth preset threshold... If the failure still fails, the failure level will be upgraded to severe anomaly, and the process will proceed to Case 3.

[0062] Scenario 3: Severe Abnormality When the failure severity is classified as severe anomaly (permanent hardware damage or reconnection failure): Mark the failed image acquisition devices; based on the currently available image acquisition devices, re-execute the second-stage path planning algorithm to determine the number and spatial pose of the updated image acquisition devices; control the updated image acquisition devices to re-acquire the original wheel hub image; after the re-acquisition is completed, return to step 3.2 to re-perform the comprehensive failure detection.

[0063] Scenario 4: Fatal Abnormality When the failure severity is a fatal exception (system memory overflow or main process unresponsive): Trigger a high-level alarm (audio-visual alarm or remote notification); save the current system state and error context to non-volatile storage media (for post-fault analysis); trigger a watchdog timer when the system is unresponsive for more than the seventh preset threshold. When this occurs, a hardware reset is triggered to restart the system or process.

[0064] Fifth stage: Generation of wheel hub perspective detection images This stage is executed after successful image acquisition and failure detection, and is used to generate a viewpoint detection image that can be used by subsequent defect detection algorithms.

[0065] Step 5.1: Determine the target image parameters Determine the size (width W′ and height H′) and resolution of the target wheel hub view detection image.

[0066] Step 5.2: Pixel position mapping Based on the wheel hub view lookup table generated in the first stage (which records the mapping relationship between the position of each pixel in the wheel hub view detection image and the corresponding sub-pixel coordinate position in the original wheel hub image), find the pixel position in the original wheel hub image corresponding to each pixel in the target wheel hub view detection image.

[0067] Step 5.3: Interpolation Calculation For the non-integer pixel positions (sub-pixel coordinates) obtained after mapping, the pixel value at that position is calculated using an interpolation method (such as bilinear interpolation or bicubic interpolation). The calculated pixel value is then assigned to the corresponding pixel point in the target wheel hub perspective detection image, generating the wheel hub perspective detection image corresponding to the image acquisition device.

[0068] Step 5.4: Multi-device image integration Repeat steps 5.1 to 5.3 for each image acquisition device to generate a corresponding wheel hub perspective detection image for each device. Since the positioning points in each perspective detection image are symmetrically distributed relative to the central axis of the wheel hub, these images have a clear spatial correspondence and can be used as a complete set of wheel hub full-circumference perspective detection images for output to the subsequent defect detection module for defect identification.

[0069] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for generating a viewpoint image for wheel hub detection, characterized in that, include: Acquire raw wheel hub images from multiple image acquisition devices located at the wheel hub inspection station; For each of the image acquisition devices, a corresponding wheel hub perspective detection image is generated based on the original wheel hub image and a pre-determined wheel hub perspective mapping relationship. The wheel hub perspective mapping relationship is the correspondence between the positioning points in the wheel hub perspective detection image and the corresponding points in the original wheel hub image; the corresponding positioning points in the wheel hub perspective detection images corresponding to different image acquisition devices are symmetrically distributed with respect to the central axis of the wheel hub in the wheel hub detection coordinate system.

2. The method for generating a viewing angle image for wheel hub detection according to claim 1, characterized in that, In the hub detection coordinate system, several feature points on the surface of the hub are selected as positioning points. The positioning points are evenly distributed along the circumference of the hub and are symmetrical with respect to the central axis of the hub. Establish the three-dimensional coordinates of each positioning point The corresponding pixel position of this positioning point in the original wheel hub image The mapping relationship between them satisfies: ; Wherein, M is a 3×4 projection matrix; the projection matrix M is obtained through calibration; Based on the mapping relationship, a wheel hub view lookup table is generated. The lookup table records the mapping relationship between each pixel position in the wheel hub view detection image and the corresponding pixel position in the original wheel hub image.

3. The method for generating a viewing angle image for wheel hub detection according to claim 1, characterized in that, The number and spatial pose of the multiple image acquisition devices set at the wheel hub inspection station are determined by a path planning algorithm based on the three-dimensional model of the wheel hub and the field of view of the image acquisition devices, so that the original wheel hub images acquired by the multiple image acquisition devices in their respective acquisition poses can cover the entire surface of the wheel hub to be inspected. The path planning algorithm includes the following steps: S1. Obtain the triangular facet model of the wheel hub to be detected, perform Gaussian mapping on the triangular facet model, and map the normal vector of each triangular facet of the triangular facet model onto the Gaussian sphere to form a Gaussian point cloud. S2. Based on the angle between the normal vectors and the topological adjacency relationship between Gaussian points in the Gaussian point cloud, the triangular patch model is divided into several independent regions, such that the angle between the normal vectors of any two Gaussian points in the same independent region is less than a preset angle threshold, and the triangular patches in the independent region are topologically adjacent. S3. For each independent region, calculate the average normal vector of all triangular facet normal vectors within that independent region. Find adjacent independent regions whose angle with the average normal vector is smaller than the field of view of the image acquisition device, and merge the found adjacent independent regions with the corresponding independent regions to obtain several merged detection regions. S4. Construct the OBB bounding box for each detection region. For each OBB bounding box, select the candidate viewpoints that can be completely surrounded by the visual cone model of the image acquisition device from among the candidate viewpoints uniformly sampled on a sphere with the center of the OBB bounding box as the center and a preset radius. S5. Perform occlusion detection on the selected candidate viewpoints: Starting from the candidate viewpoint, emit rays to each vertex of the OBB bounding box. If the ray intersects with other triangular facets in the triangular facet model other than the detection area where the OBB bounding box is located, it is determined that the candidate viewpoint is occluded. Adjust the spatial pose of the candidate viewpoint until there is no occlusion. Use the adjusted candidate viewpoint as the acquisition pose of the image acquisition device corresponding to the detection area. S6. Summarize the acquisition poses corresponding to each detection area to obtain the number and spatial pose of the image acquisition devices.

4. The method for generating a viewpoint image for wheel hub detection according to claim 1, characterized in that, After acquiring the original wheel hub images captured by multiple image acquisition devices set up at the wheel hub inspection station, the method further includes: Obtain the operating status information of each of the image acquisition devices; Based on the operating status information, a comprehensive failure detection is performed on each of the image acquisition devices; If any of the image acquisition devices is detected to be malfunctioning, a corresponding processing strategy will be executed according to the severity of the malfunction.

5. The method for generating a viewpoint image for wheel hub detection according to claim 4, characterized in that, The comprehensive failure detection includes at least two of the following: Hardware status detection: Obtain the connection status, operating temperature, power supply voltage and real-time frame rate of the image acquisition device; when a connection is broken, the operating temperature exceeds the preset temperature range, the power supply voltage deviates from the rated voltage by more than the preset voltage deviation range, or the real-time frame rate is lower than the product of the set frame rate and the preset frame rate ratio threshold, it is determined that the image acquisition device has a hardware abnormality. Image quality detection: Calculate the sharpness index of the original wheel hub image. Average brightness and noise level The clarity index adopts Calculate, when When the average brightness is below a first preset threshold, the clarity is deemed substandard; the average brightness is determined using... Calculate, when A brightness abnormality is determined when the noise level is below a second preset threshold or above a third preset threshold; the noise level is determined using... Calculation, where ,when The noise level is considered excessive when it exceeds the fourth preset threshold. Event log detection: Read the operation log of the image acquisition device and check whether there are buffer overflow event records, image acquisition job interruption event records, or external trigger signal missing event records in the log; Multi-view cross-validation: When there is an overlapping area between the original wheel hub images acquired by multiple image acquisition devices, feature points in the overlapping area of ​​each image are extracted and feature matching is performed; the images of one of the image acquisition devices are removed in turn, and the overlapping area is reconstructed in three dimensions using the remaining images and the reprojection error is calculated with the images that have not been removed; when the reprojection error after removing a certain device increases by more than a fifth preset threshold compared with before removal, the removed device is determined to be invalid.

6. The method for generating a viewpoint image for wheel hub detection according to claim 4, characterized in that, Based on the results of the comprehensive failure detection, the severity of the failure is determined according to the following rules: When the detection result is a single frame loss or a single image processing timeout, the severity of the failure is determined to be a minor anomaly. When the severity of the failure is a minor anomaly, the current frame is discarded and the log is recorded without interrupting the detection process. When the detection result is that the image acquisition device is disconnected but can be reconnected, or the inference engine inside the image acquisition device is malfunctioning, the severity of the failure is determined to be moderate. When the severity of the failure is moderate, the acquisition channel of the failed image acquisition device is suspended, the computing and storage resources occupied by the channel are released, and automatic reconnection is attempted according to the preset exponential backoff strategy until the reconnection is successful or the number of reconnection reaches the sixth preset threshold, after which it is determined to be severe. When the detection result indicates that the image acquisition device hardware is permanently damaged, or the automatic reconnection fails after the moderate anomaly handling stage, the severity of the failure is determined to be a severe anomaly. When the severity of the failure is a severe anomaly, the failed image acquisition device is marked, and the path planning algorithm is re-executed based on the currently available image acquisition devices to determine the number and spatial pose of the updated image acquisition devices, and the updated image acquisition devices are controlled to re-acquire the original wheel hub image. When the detection result is system memory overflow or main process unresponsive, the severity of the failure is determined to be a fatal exception. When the severity of the failure is a fatal exception, a high-level alarm is triggered, the current system state and error context are saved to non-volatile storage media, and the watchdog timer triggers a hardware reset to restart the system or process when the system unresponsiveness exceeds the seventh preset threshold.

7. The method for generating a viewpoint image for wheel hub detection according to claim 1, characterized in that, The step of generating a wheel hub perspective detection image based on the original wheel hub image and a pre-determined wheel hub perspective mapping relationship includes: Determine the size and resolution of the target wheel hub perspective detection image; Based on the mapping relationship recorded in the wheel hub perspective mapping relationship, find the pixel position of each pixel in the target wheel hub perspective detection image in the original wheel hub image; For non-integer pixel positions after mapping, the pixel value at that position is calculated by interpolation. The calculated pixel value is then assigned to the corresponding pixel in the target wheel hub perspective detection image to generate the wheel hub perspective detection image.

8. The method for generating a viewing image for wheel hub detection according to claim 1, characterized in that, The wheel hub view lookup table records the mapping relationship between the position of each pixel in the wheel hub view detection image and the corresponding sub-pixel coordinate position in the original wheel hub image.

9. A system for generating viewpoint images for wheel hub detection, used to perform the method according to any one of claims 1-8, characterized in that, include: The image acquisition module is used to acquire the original wheel hub images captured by multiple image acquisition devices set up at the wheel hub inspection station; The image generation module is used to generate a wheel hub perspective detection image corresponding to each of the image acquisition devices, based on the original wheel hub image and a pre-determined wheel hub perspective mapping relationship. The wheel hub perspective mapping relationship is the correspondence between the positioning points in the wheel hub perspective detection image and the corresponding points in the original wheel hub image; the corresponding positioning points in the wheel hub perspective detection images corresponding to different image acquisition devices are symmetrically distributed with respect to the central axis of the wheel hub in the wheel hub detection coordinate system.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the method as described in any one of claims 1-8.

Images