Dynamic self-adaptive sensing equipment deviation rectification method and system

Abstract

The invention belongs to the technical field of automobile vision sensors, and discloses a dynamic self-adaptive sensing equipment deviation correction method and system. The method comprises the following steps: S1, determining a blind area between adjacent vision sensors calibrated in advance and corresponding to a vehicle body; s2, acquiring an image corresponding to the previous period of parking of the vehicle body and recording the image as a first image; s3, acquiring an image of the vehicle body in the current period, recording the image as a monitoring image, and correcting the image corresponding to each blind area in the monitoring image by using the first image and the monitoring image to obtain the corrected image corresponding to each blind area; according to the method, the images corresponding to the blind areas in the vehicle body monitoring images can be corrected, more accurate parking environment information is provided for a driver, and the risk of accidents such as collision and scraping when the vehicle is parked for a short time is reduced.

Description

Technical Field

[0001] This invention relates to the field of automotive vision sensor technology, and more specifically, to a method and system for correcting deviations in a dynamic adaptive sensing device. Background Technology

[0002] A vision sensor is a device that can convert optical image information into electrical signals. It is widely used in industrial inspection, robot vision, security monitoring, and 360-degree panoramic imaging of automobiles. With the continuous development of automotive intelligence and safety technology, vision sensors play a crucial role as key components for automobiles to perceive the external environment, such as CMOS image sensors. In the automotive field, 360° panoramic imaging systems have become an important feature for improving driving safety and convenience. The system is usually composed of four visual sensors (cameras) installed at the front, rear, left, and right of the vehicle. By collecting image information from different angles around the vehicle and using image stitching and processing technology, it presents the driver with a complete panoramic image of the vehicle's surrounding environment, effectively reducing the driver's blind spots and improving safety in scenarios such as parking and low-speed driving. However, when a car is briefly parked (during which the driver generally does not get out of the car to walk around and inspect it), due to the physical characteristics of the four visual sensors and their installation positions, blind spots (such as under the car or at the four corners of the vehicle) can easily appear between adjacent visual sensors in a 360-degree panoramic imaging system. The existence of these blind spots greatly increases the risk of collisions, scratches, and other accidents when the car is briefly parked. Therefore, it is necessary to correct the blind spots caused by the positions of adjacent visual sensors in a 360-degree panoramic imaging system to reduce the risk of accidents during brief stops.

[0003] In view of this, the present invention proposes a dynamic adaptive sensing device correction method and system to solve the above problems. Summary of the Invention

[0004] To overcome the aforementioned deficiencies of the prior art and to achieve the above objectives, the present invention provides the following technical solution: a method for correcting bias in a dynamic adaptive sensing device, comprising the following steps: S1. Determine the blind spots between each pre-calibrated adjacent visual sensor on the vehicle body; S2. Obtain the image of the vehicle body in the previous cycle before parking and record it as the first image; S3. Acquire the image of the vehicle body in the current cycle and record it as the monitoring image. Use the first image and the monitoring image to correct the image corresponding to each blind zone in the monitoring image to obtain the corrected image corresponding to each blind zone. Specifically, the correction process in step S3 includes: acquiring basic images of each blind zone using the first image, establishing a monitoring network and generating predicted trajectories of each object in the corresponding blind zone based on the movement trajectory of each object, and adding the predicted trajectories of each object to the basic image of the corresponding blind zone, thereby completing the correction of the images corresponding to each blind zone in the monitoring image.

[0005] Furthermore, S1 specifically includes: S11. Establish a virtual calibration space, which is composed of multiple virtual three-dimensional grids; S12. Place the vehicle body at the center of the virtual calibration space, simulate the specifications of each visual sensor corresponding to the vehicle body, and take pictures of the virtual calibration space. Use the space formed by the virtual three-dimensional grid corresponding to each visual sensor as the shooting space range of each visual sensor. S13. Mark the non-overlapping virtual 3D grids between the shooting space ranges corresponding to each adjacent visual sensor, and use the non-overlapping virtual 3D grids as the blind zones between each adjacent visual sensor.

[0006] Furthermore, S2 specifically includes: S21. Obtain the time point corresponding to the image of the vehicle body that matches the blind zone between each adjacent visual sensor before the vehicle body stops and record it as the first time point. Take the time interval between the first time point and the time point corresponding to the vehicle body being completely stationary as the previous cycle. S22. Extract the image corresponding to the previous cycle and use it as the first image.

[0007] Furthermore, S3 specifically includes: S31. After the vehicle comes to a stop, the surrounding environment of the vehicle is monitored by various vision sensors to obtain monitoring images. S32. Use the first image to fill in the images corresponding to the blind areas between adjacent visual sensors in the monitoring image to obtain the basic images of each blind area. S33. Determine the monitoring range of each visual sensor on the vehicle body and establish a monitoring network. Monitor the movement trajectory of each object on the monitoring network and generate the predicted trajectory of each object in the corresponding blind zone based on the movement trajectory. Add the predicted trajectory of each object to the base image of the corresponding blind zone, thus completing the correction of the image corresponding to each blind zone in the monitoring image.

[0008] Furthermore, S32 specifically includes: S321. Mark the spatial boundary of each blind zone in the monitoring image based on the blind zone between each adjacent visual sensor; S322. Based on the spatial range boundary corresponding to each blind zone, extract the image of each blind zone in the first image, and fill the extracted images of each blind zone in the first image into the images corresponding to the blind zones between adjacent visual sensors in the monitoring image to obtain the basic image of each blind zone.

[0009] Furthermore, S33 specifically includes: S331. Obtain the monitoring range of each visual sensor after the vehicle body is stationary, and establish a monitoring network for monitoring the environment and objects around the vehicle body based on the monitoring range of each visual sensor. S332. Divide the objects on the monitoring network into static objects and dynamic objects. Static objects can be converted into dynamic objects. Static objects are objects that do not have displacement on the monitoring network, while dynamic objects are objects that have displacement on the monitoring network. S333. Determine the shape and specification information of each object on the monitoring network and bind and associate the shape and specification information with the corresponding object. Extract the outline information of each object based on the shape and specification information of each object. S334. Obtain the movement trajectory of dynamic objects on the monitoring network. When a dynamic object enters a blind zone, automatically generate a predicted trajectory of the dynamic object based on the movement trajectory. Add the dynamic object that has entered the blind zone and the predicted trajectory to the blind zone of the monitoring area, thereby completing the correction of the images corresponding to each blind zone in the monitoring image.

[0010] Furthermore, the step of acquiring the movement trajectory of dynamic objects on the monitoring network specifically includes: Number the moving objects on the monitoring network and mark their location information on the monitoring network; By connecting the marked location information in chronological order, the movement trajectory of dynamic objects on the monitoring network can be obtained.

[0011] Furthermore, the step of automatically generating the predicted trajectory of a dynamic object based on its movement trajectory specifically includes: Obtain the tangent sequence of the dominant edge of a dynamic object on its movement trajectory, and generate a predicted trajectory of the dynamic object based on the changing trend of the tangent in the tangent sequence. The tangent sequence consists of the tangents of the dominant edge of the dynamic object at n time points on its movement trajectory. If the trend of the change in the tangent angle is zero or close to zero, then the movement trajectory of the dynamic object is used as the predicted trajectory. If the trend of the chamfer is to gradually decrease, then obtain the decreasing speed corresponding to the chamfer, and use the movement trajectory corresponding to the decreasing speed as the predicted trajectory of the dynamic object in the blind zone. If the trend of the chamfer angle is to gradually increase, then obtain the increase rate corresponding to the chamfer angle, and use the movement trajectory corresponding to the increase rate as the predicted trajectory of the dynamic object in the blind zone. If the trend of the chamfer change is alternating between increasing and decreasing, and if the increase and decrease are regular, then the movement trajectory corresponding to the regular increase and decrease is used as the predicted trajectory of the dynamic object in the blind zone. If the increase and decrease are irregular, then the range of motion and displacement velocity of the dynamic object are obtained, and m possible candidate trajectories are generated in the blind zone corresponding to the dynamic object based on the range of motion and displacement velocity. The optimal trajectory among the m possible candidate trajectories is used as the predicted trajectory of the dynamic object in the blind zone.

[0012] Furthermore, the step of obtaining the optimal trajectory from m possible candidate trajectories as the predicted trajectory of the dynamic object in the blind zone specifically includes: Determine the maximum changing angle and maximum displacement velocity of the dynamic object on the monitoring network, and set the activity range of the dynamic object based on the maximum changing angle and maximum displacement velocity; By randomly generating m possible candidate trajectories within the Monte Carlo sampling activity range, and filtering the m possible candidate trajectories based on the set physical constraints, the similarity value between the remaining candidate trajectories after filtering and the trajectory of the moving object is calculated using DTW distance, and the candidate trajectory with the highest similarity value is taken as the predicted trajectory of the moving object in the blind zone. Among these, physical constraints include the feasibility of terrain conditions.

[0013] The present invention also includes: a dynamic adaptive sensing device correction system, comprising: The blind spot calibration module is used to determine the blind spots between each pre-calibrated adjacent visual sensor on the vehicle body; The image acquisition module is used to acquire the image of the vehicle body in the previous cycle before parking and record it as the first image; The image correction module is used to acquire images of the vehicle body in the current cycle and record them as monitoring images. It uses the first image and the monitoring images to correct the images corresponding to each blind spot in the monitoring images, so as to obtain the corrected images corresponding to each blind spot. In the image correction module, the correction process specifically includes: acquiring the basic image of each blind zone using the first image, establishing a monitoring network and generating the predicted trajectory of each object in the corresponding blind zone based on the movement trajectory of each object, and adding the predicted trajectory of each object to the basic image of the corresponding blind zone, thereby completing the correction of the image corresponding to each blind zone in the monitoring image.

[0014] The technical effects and advantages of the dynamic adaptive sensing device correction method and system of the present invention are as follows: By establishing a virtual calibration space to pre-calibrate the blind spots between adjacent visual sensors, and using the extracted first image as the base image for each blind spot, the image information of static objects within the blind spot can be filled in. By establishing a monitoring network and monitoring dynamic objects on the monitoring network, the movement trajectory of dynamic objects can be obtained. Based on the changing trend of the chamfer in the chamfer sequence corresponding to the movement trajectory of the dynamic object, if the changing trend shows a regular change, the predicted trajectory of the dynamic object in the corresponding blind spot can be generated. Finally, by adding the dynamic object and the predicted trajectory to the base image of the corresponding blind spot, the image corresponding to each blind spot in the monitoring image is corrected, providing the driver with more accurate parking environment information and reducing the risk of collisions, scratches and other accidents when the car is briefly stopped. Attached Figure Description

[0015] Figure 1 This is a flowchart illustrating a dynamic adaptive sensing device correction method according to the present invention. Figure 2 This is a schematic diagram of the process of S33 of the present invention; Figure 3 This is a schematic diagram of the structure of a dynamic adaptive sensing device correction system according to the present invention; Figure 4 This is a schematic diagram of the structure for monitoring online movement trajectories according to the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] When a car is briefly stopped (during which the driver typically does not get out of the car to walk around and inspect it), blind spots (such as under the car or at the four corners) can easily appear between adjacent visual sensors in a 360° panoramic imaging system due to the physical characteristics and installation location of the four visual sensors. These blind spots significantly increase the risk of collisions and scrapes during brief stops. Therefore, this paper proposes a dynamic adaptive sensor correction method and system to address these issues and correct the blind spots created by the positions of adjacent visual sensors in a 360° panoramic imaging system.

[0018] Please see Figure 1 , Figure 2 as well as Figure 4As shown in this embodiment, a dynamic adaptive sensing device correction method includes the following steps: S1. Determine the blind zone between each pre-calibrated adjacent visual sensor on the vehicle body. It should be noted that by constructing a virtual calibration space and simulating 360-degree panoramic imaging for shooting, the spatial range of each blind zone can be obtained, and each blind zone can be quantified with high precision. S2. Acquire the image of the vehicle body in the previous cycle before parking and record it as the first image. It should be noted that the first image can accurately reflect the environmental information related to the blind spot before the vehicle stops, and can provide a reliable data basis for subsequent correction of the blind spot of the monitoring image based on the first image, so that the corrected image is more in line with the actual scene. S3. Acquire images of the vehicle body in the current cycle and record them as monitoring images. Use the first image and the monitoring images to correct the images corresponding to each blind zone in the monitoring images to obtain the corrected images corresponding to each blind zone. Specifically, the correction process in step S3 includes: acquiring basic images of each blind zone using the first image, establishing a monitoring network and generating predicted trajectories of each object in the corresponding blind zone based on the movement trajectory of each object, and adding the predicted trajectories of each object to the basic image of the corresponding blind zone, thus completing the correction of the images corresponding to each blind zone in the monitoring images. It should be noted that by establishing a monitoring network and monitoring the movement trajectory of dynamic objects on the monitoring network, and generating predicted trajectories of dynamic objects in the corresponding blind zones based on the changing trend of the chamfer sequence corresponding to the movement trajectory of the dynamic objects, it is easier to complete the correction of the images corresponding to each blind zone in the monitoring images, providing drivers with more accurate parking environment information.

[0019] In this embodiment, a virtual calibration space is established to pre-calibrate the blind spots between adjacent visual sensors. The extracted first image is used as the base image for each blind spot, thereby filling in the image information of static objects within the blind spot. By establishing a monitoring network and monitoring dynamic objects on the monitoring network, the movement trajectory of the dynamic objects can be obtained. Based on the changing trend of the chamfer in the chamfer sequence corresponding to the movement trajectory of the dynamic object, if the changing trend is regular, a predicted trajectory of the dynamic object in the corresponding blind spot is generated. If the changing trend is irregular, Monte Carlo sampling is used to randomly generate m possible candidate trajectories within the activity range, and physical constraints are used to filter the candidate trajectories. DTW distance is used to calculate the candidate trajectory most similar to the historical actual trajectory of the dynamic object as the predicted trajectory of this dynamic object. Finally, by adding the dynamic object and the predicted trajectory to the base image of the corresponding blind spot, the correction of the image corresponding to each blind spot in the monitoring image is completed, providing the driver with more accurate parking environment information.

[0020] As an optional embodiment: S1 specifically includes: S11. Establish a virtual calibration space, which is composed of multiple virtual 3D meshes (e.g., the virtual 3D mesh has a size of 10mm). 10mm 10mm, can be in a three-dimensional grid pattern); S12. Place the vehicle body at the center of the virtual calibration space, simulate the specifications of each visual sensor corresponding to the vehicle body, and take pictures of the virtual calibration space. Use the space formed by the virtual three-dimensional grid corresponding to each visual sensor as the shooting space range of each visual sensor. S13. Mark the non-overlapping virtual 3D grids between the shooting space ranges corresponding to each adjacent visual sensor, and use the non-overlapping virtual 3D grids as the blind zones between each adjacent visual sensor.

[0021] It should be noted that by simulating the shooting effect of the vehicle's 360-degree panoramic image in a virtual calibration space, and using a virtual three-dimensional grid with non-overlapping shooting space ranges between adjacent visual sensors, the spatial range of each blind zone can be obtained, enabling high-precision quantification of each blind zone. After calibration in the virtual space, the blind zones can be verified in a real-world scenario. The verification process involves using the vehicle's 360-degree panoramic image to capture a real-world scene, placing an object at the boundary of the calibrated blind zone for testing, and comparing the placed object with the boundary of the blind zone. If the placed object is not displayed within the blind zone but is displayed outside the blind zone boundary, the calibrated blind zone is considered to have passed verification, ensuring the accuracy of the calibrated blind zones.

[0022] As an optional embodiment: S2 specifically includes: S21. Obtain the time point corresponding to the image of the vehicle body that matches the blind zone between each adjacent visual sensor before the vehicle body stops and record it as the first time point. Take the time interval between the first time point and the time point corresponding to the vehicle body being completely stationary as the previous cycle. S22. Extract the image corresponding to the previous cycle and use it as the first image.

[0023] It should be noted that the matching method is as follows: using the SURF or SIFT algorithm, the last frame of the image after the vehicle stops is matched with each frame of the image preceding the stopping time. The time point corresponding to the image where the boundary of the blind spot is fully displayed is taken as the first time point (if the time points corresponding to the boundary of each blind spot are different, the time point farthest from the current time point is taken as the first time point). Thus, the first image can accurately reflect the environmental information related to the blind spot before the vehicle stops, and can provide a reliable data foundation for subsequent correction of the blind spot of the monitoring image based on the first image, making the corrected image more consistent with the actual scene.

[0024] As an optional embodiment: S3 specifically includes: S31. After the vehicle comes to a stop, the surrounding environment of the vehicle is monitored by various vision sensors to obtain monitoring images. S32. Use the first image to fill in the images corresponding to the blind areas between adjacent visual sensors in the monitoring image to obtain the basic images of each blind area. S33. Determine the monitoring range of each visual sensor on the vehicle body and establish a monitoring network. Monitor the movement trajectory of each object on the monitoring network and generate the predicted trajectory of each object in the corresponding blind zone based on the movement trajectory. Add the predicted trajectory of each object to the base image of the corresponding blind zone, thus completing the correction of the image corresponding to each blind zone in the monitoring image.

[0025] It should be noted that the monitoring images can reflect the current environmental conditions around the vehicle. By using the first image to fill in the images corresponding to the blind spots between adjacent visual sensors in the monitoring images, the basic images of each blind spot are obtained, which initially compensates for the image gaps in the blind spots. By using the image corresponding to the first time point in the first image to fill the blind spots as the basic image (when using the image as the basic image, it needs to be matched and aligned using the SURF or SIFT algorithm. The alignment method can be assisted by the spatial boundary of the blind spot. When used as the basic image, static objects in this image are retained, while dynamic objects are not retained. The determination of static and dynamic objects can be made by whether the objects in the first image have moved), it can not only fill in the image information of static objects in the blind spots, but also predict and supplement the dynamic situation in the blind spots based on the movement trajectory of dynamic objects. This greatly improves the accuracy and completeness of the blind spot images, making the 360-degree panoramic image and its corresponding blind spots more realistically reflect the actual situation of the vehicle's surrounding environment, providing drivers with more comprehensive and accurate parking assistance information, and effectively reducing the risk of parking accidents caused by missing blind spot information.

[0026] As an optional embodiment: S32 specifically includes: S321. Mark the spatial boundary of each blind zone in the monitoring image based on the blind zone between each adjacent visual sensor; S322. Based on the spatial range boundary corresponding to each blind zone, extract the image of each blind zone in the first image, and fill the extracted images of each blind zone in the first image into the images corresponding to the blind zones between adjacent visual sensors in the monitoring image to obtain the basic image of each blind zone.

[0027] It should be noted that by using the spatial boundaries corresponding to each blind zone, the blind zones of the monitoring image can be accurately filled, which facilitates the acquisition of basic images of each blind zone. This provides a more accurate basis for predicting the trajectory of dynamic objects entering the blind zone and for image correction of each blind zone, thereby improving the accuracy and reliability of the blind zone correction and deviation correction process.

[0028] As an optional embodiment: S33 specifically includes: S331. Obtain the monitoring range of each visual sensor after the vehicle body is stationary, and establish a monitoring network for monitoring the environment and objects around the vehicle body based on the monitoring range of each visual sensor. S332. Divide the objects on the monitoring network into static objects and dynamic objects. Static objects can be converted into dynamic objects. Static objects are objects that do not have displacement on the monitoring network, while dynamic objects are objects that have displacement on the monitoring network. S333. Determine the shape and specification information of each object on the monitoring network and bind and associate the shape and specification information with the corresponding object. Extract the outline information of each object based on the shape and specification information of each object. S334. Obtain the movement trajectory of dynamic objects on the monitoring network. When a dynamic object enters a blind zone, automatically generate a predicted trajectory of the dynamic object based on the movement trajectory. Add the dynamic object that has entered the blind zone and the predicted trajectory to the blind zone of the monitoring area, thereby completing the correction of the images corresponding to each blind zone in the monitoring image.

[0029] It should be noted that the contour information of an object includes the positional relationship between the contour convex points of the object. The contour convex points can be calculated using the convex hull algorithm. The positional relationship includes the distance and angle between adjacent contour convex points. The shape and specification information of each object helps to mark each object, while the contour information helps to observe whether the object has deformed. The transformation of a static object into a dynamic object can be determined by whether it has displacement on the monitoring network. By establishing a monitoring network to monitor the environment around the vehicle and classifying the objects on the monitoring network into dynamic and static objects, it is easier to obtain the movement trajectory of dynamic objects on the monitoring network and to predict the movement trajectory of dynamic objects entering blind spots. This enables the correction of the corresponding images of each blind spot, reduces accidents caused by failure to detect dynamic objects in blind spots or collisions with dynamic objects, and provides drivers with more reliable parking environment information.

[0030] As an optional embodiment: the step of obtaining the movement trajectory of dynamic objects on the monitoring network specifically includes: Number the moving objects on the monitoring network and mark their location information on the monitoring network; By connecting the marked location information in chronological order, the movement trajectory of dynamic objects on the monitoring network can be obtained.

[0031] It should be noted that by using labels and location markers, different dynamic objects can be clearly distinguished and their positional information changes can be accurately recorded. By connecting the positional information in chronological order, the continuity and accuracy of the movement trajectory of dynamic objects can be ensured. This provides a reliable data foundation for generating predicted trajectories based on the movement trajectory, which helps to improve the accuracy of predicting the movement of dynamic objects in blind spots. It can also improve the image correction effect of each blind spot in 360-degree panoramic images, making the images of each blind spot more realistically reflect the movement of dynamic objects.

[0032] As an optional embodiment: the step of automatically generating the predicted trajectory of a dynamic object based on its movement trajectory specifically includes: Obtain the tangent sequence of the dominant edge of a dynamic object on its movement trajectory, and generate a predicted trajectory of the dynamic object based on the changing trend of the tangent in the tangent sequence. The tangent sequence consists of the tangents of the dominant edge of the dynamic object at n time points on its movement trajectory. If the trend of the chamfer change is zero or close to zero, then the trajectory of the moving object is used as the predicted trajectory. For example, refer to... Figure 4 , Figure 4 The trend of the change in the tangent angle is zero; If the trend of the chamfer is to gradually decrease, then obtain the decreasing speed corresponding to the chamfer, and use the movement trajectory corresponding to the decreasing speed as the predicted trajectory of the dynamic object in the blind zone. If the trend of the chamfer angle is to gradually increase, then obtain the increase rate corresponding to the chamfer angle, and use the movement trajectory corresponding to the increase rate as the predicted trajectory of the dynamic object in the blind zone. If the trend of the chamfer change is alternating between increasing and decreasing, and if the increase and decrease are regular, then the movement trajectory corresponding to the regular increase and decrease is used as the predicted trajectory of the dynamic object in the blind zone. If the increase and decrease are irregular, then the range of motion and displacement velocity of the dynamic object are obtained, and m possible candidate trajectories are generated in the blind zone corresponding to the dynamic object based on the range of motion and displacement velocity. The optimal trajectory among the m possible candidate trajectories is used as the predicted trajectory of the dynamic object in the blind zone.

[0033] It should be noted that the dominant edge of a dynamic object is its longest straight line projected onto the monitoring network (if multiple identical longest straight lines exist, one can be randomly selected, such as the major axis direction of a vehicle's projection plane); the tangent angle is the angle between the direction vector of the dominant edge of the dynamic object and the moving direction vector (forward direction) of the dynamic object; the trend of change is obtained by the difference between the tangent angle of the dynamic object at the current time point and the tangent angle at the previous time point; by obtaining the trend of change of the tangent angle sequence corresponding to the movement trajectory of the dynamic object on the monitoring network, and generating the corresponding predicted trajectory based on the trend, the predicted trajectory is made to better match the actual movement of the dynamic object. This system improves the accuracy of predicting the movement of dynamic objects in blind spots, helping to more realistically correct or supplement the images of dynamic objects in blind spots. This enhances the accuracy and real-time performance of the 360° panoramic imaging system, providing drivers with more reliable information about the dynamic environment around the vehicle and reducing potential dangers or accident risks caused by the movement of dynamic objects in blind spots. In addition, if multiple identical dynamic objects exist, a distinguishing marker is added to each dynamic object to facilitate differentiation. Each of the identical dynamic objects is monitored, and dynamic objects that have not left the blind spot are recorded and sent to the driver for early warning.

[0034] As an optional embodiment: the step of obtaining the optimal trajectory from m possible candidate trajectories as the predicted trajectory of the dynamic object in the blind zone specifically includes: Determine the maximum changing angle and maximum displacement velocity of the dynamic object on the monitoring network, and set the activity range of the dynamic object based on the maximum changing angle and maximum displacement velocity; By randomly generating m possible candidate trajectories within the Monte Carlo sampling activity range, and filtering the m possible candidate trajectories based on the set physical constraints, the similarity value between the remaining candidate trajectories after filtering and the trajectory of the moving object is calculated using DTW distance, and the candidate trajectory with the highest similarity value is taken as the predicted trajectory of the moving object in the blind zone. Among these, physical constraints include the feasibility of terrain conditions.

[0035] It should be noted that by utilizing Monte Carlo sampling, m possible candidate trajectories are randomly generated within the activity range. For example, to determine the last known state of a dynamic object before entering the blind zone, including position information (x0, y0), velocity v, and maximum tangent angle change Δθmax, candidate trajectories are defined with velocities within the range of v ± Δv (considering acceleration and deceleration) and turning angles within the range of [θ0 - Δθmax, θ0 + Δθmax]. m = 200 candidate trajectories are randomly generated within the activity range, each consisting of k = 5 position information points (e.g., one position information point is generated every 0.2 seconds). The specific process is as follows: an initial disturbance is randomly generated, including a velocity change range Δv (e.g., ±0.5 m / s) and a turning angle change. The azimuth angle varies within the range Δθ. Based on the initial state and disturbance, the position (x, y) of each key point is calculated. The generated trajectory needs to meet physical constraints (i.e., the candidate trajectory must be terrain-feasible, where terrain feasibility means that there are no obstacles such as puddles or potholes on the ground or that need to be detoured). Finally, candidate trajectories that do not meet the physical constraints are filtered out, leaving about 100 candidate trajectories. This sampling method can fully explore the possible movement trajectory of dynamic objects within the spatial boundary range of the corresponding blind zone within a certain range. Thus, by setting the activity range and physical constraints, unreasonable candidate trajectories can be effectively eliminated to ensure that the candidate trajectories conform to the actual movement law of dynamic objects and the limitations of the terrain environment. By using DTW distance calculation to select candidate trajectories, the predicted trajectory most similar to the actual movement trajectory of the object is found, further improving the accuracy of predicting the movement of dynamic objects in blind spots. This makes the images added to the blind spots more realistic and accurate, improving the quality of 360-degree panoramic images and providing drivers with more precise parking environment information. For example, let's define the movement trajectory A of a dynamic object: its coordinate sequence is [(1,1),(2,2),(3,3),(4,4),(5,5)]. There are three candidate trajectories B, C, and D. Candidate trajectory B: [(1,1),(2,2),(3.5,3.5),(4.5,4.5)]. Candidate trajectory C: [(1,1),(2.5,2.5),(3,3),(4,4),(5.5,5.5)], Candidate trajectory D: [(1,1),(2,3),(3,4),(4,5),(5,6)]. When calculating the DTW distance, a distance matrix is ​​first constructed, and the Euclidean distance between the actual trajectory A and the corresponding points of each candidate trajectory is calculated. The DTW distances between A and B, C, and D are 0.8, 1.2, and 2.5, respectively. The DTW distance between A and B is the smallest, indicating that B and A are the most similar. The similarity value between B and A is the highest. Therefore, candidate trajectory B is used as the final predicted trajectory to supplement the dynamic object image in the blind zone. In addition, the entry and exit times of each dynamic object entering the corresponding blind spot need to be marked. If the monitoring network does not detect that the object has left the 360-degree panoramic image before the vehicle starts and is not within the 360-degree panoramic image, a warning will be issued to the driver to reduce the risk of accidents.

[0036] Please see Figure 3 As shown in the figure, the dynamic adaptive sensing device correction system described in this embodiment includes a blind zone calibration module, an image acquisition module, and an image correction module. The aforementioned blind spot calibration module is used to determine the blind spots between each pre-calibrated adjacent visual sensor on the vehicle body. It should be noted that by constructing a virtual calibration space and simulating 360-degree panoramic imaging, the spatial range of each blind spot can be obtained, and each blind spot can be quantified with high precision. The aforementioned image acquisition module is used to acquire the image of the vehicle body in the previous cycle before parking and record it as the first image. It should be noted that the first image can accurately reflect the environmental information related to the blind spot before the vehicle stops, and can provide a reliable data basis for subsequent correction of the blind spot of the monitoring image based on the first image, so that the corrected image is more in line with the actual scene. The aforementioned image correction module is used to acquire images of the vehicle body in the current cycle and record them as monitoring images. It then uses the first image and the monitoring images to correct the images corresponding to each blind spot in the monitoring images, thus obtaining the corrected images for each blind spot. Specifically, the correction process in the image correction module includes: acquiring basic images of each blind spot using the first image; establishing a monitoring network and generating predicted trajectories of each object within its corresponding blind spot based on the object's movement trajectory; and adding the predicted trajectories of each object to the basic image of the corresponding blind spot. This completes the correction of the images corresponding to each blind spot in the monitoring images. It should be noted that by establishing a monitoring network and monitoring the movement trajectories of dynamic objects on the network, and by generating predicted trajectories of dynamic objects in their corresponding blind spots based on the changing trends of the chamfered angles in the chamfered angle sequence corresponding to the object's movement trajectory, the correction of the images corresponding to each blind spot in the monitoring images can be easily completed, providing the driver with more accurate parking environment information.

[0037] In this embodiment, a virtual calibration space is established to pre-calibrate the blind spots between adjacent visual sensors. The extracted first image is used as the base image for each blind spot, thereby filling in the image information of static objects within the blind spot. By establishing a monitoring network and monitoring dynamic objects on the monitoring network, the movement trajectory of the dynamic objects can be obtained. Based on the changing trend of the chamfer in the chamfer sequence corresponding to the movement trajectory of the dynamic object, if the changing trend shows a regular change, a predicted trajectory of the dynamic object in the corresponding blind spot is generated. Finally, by adding the dynamic object and the predicted trajectory to the base image of the corresponding blind spot, the image correction of each blind spot in the monitoring image is completed, providing the driver with more accurate parking environment information.

[0038] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0039] In the several embodiments provided by this invention, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only one method, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0040] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

[0041] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for correcting deviations in a dynamic adaptive sensing device, characterized in that, Includes the following steps: S1. Determine the blind spots between each pre-calibrated adjacent visual sensor on the vehicle body; S2. Obtain the image of the vehicle body in the previous cycle before parking and record it as the first image; S3. Acquire the image of the vehicle body in the current cycle and record it as the monitoring image. Use the first image and the monitoring image to correct the image corresponding to each blind zone in the monitoring image to obtain the corrected image corresponding to each blind zone. Specifically, the correction process in step S3 includes: acquiring basic images of each blind zone using the first image, establishing a monitoring network and generating predicted trajectories of each object in the corresponding blind zone based on the movement trajectory of each object, and adding the predicted trajectories of each object to the basic image of the corresponding blind zone, thereby completing the correction of the images corresponding to each blind zone in the monitoring image.

2. The method for correcting deviations in a dynamic adaptive sensing device according to claim 1, characterized in that, S1 specifically includes: S11. Establish a virtual calibration space, which is composed of multiple virtual three-dimensional grids; S12. Place the vehicle body at the center of the virtual calibration space, simulate the specifications of each visual sensor corresponding to the vehicle body, and take pictures of the virtual calibration space. Use the space formed by the virtual three-dimensional grid corresponding to each visual sensor as the shooting space range of each visual sensor. S13. Mark the non-overlapping virtual 3D grids between the shooting space ranges corresponding to each adjacent visual sensor, and use the non-overlapping virtual 3D grids as the blind zones between each adjacent visual sensor.

3. The method for correcting deviations in a dynamic adaptive sensing device according to claim 2, characterized in that, S2 specifically includes: S21. Obtain the time point corresponding to the image of the vehicle body that matches the blind zone between each adjacent visual sensor before the vehicle body stops and record it as the first time point. Take the time interval between the first time point and the time point corresponding to the vehicle body being completely stationary as the previous cycle. S22. Extract the image corresponding to the previous cycle and use it as the first image.

4. The method for correcting deviations in a dynamic adaptive sensing device according to claim 3, characterized in that, S3 specifically includes: S31. After the vehicle comes to a stop, the surrounding environment of the vehicle is monitored by various vision sensors to obtain monitoring images. S32. Use the first image to fill in the images corresponding to the blind areas between adjacent visual sensors in the monitoring image to obtain the basic images of each blind area. S33. Determine the monitoring range of each visual sensor on the vehicle body and establish a monitoring network. Monitor the movement trajectory of each object on the monitoring network and generate the predicted trajectory of each object in the corresponding blind zone based on the movement trajectory. Add the predicted trajectory of each object to the base image of the corresponding blind zone, thus completing the correction of the image corresponding to each blind zone in the monitoring image.

5. The method for correcting deviations in a dynamic adaptive sensing device according to claim 4, characterized in that, Specifically, S32 includes: S321. Mark the spatial boundary of each blind zone in the monitoring image based on the blind zone between each adjacent visual sensor; S322. Based on the spatial range boundary corresponding to each blind zone, extract the image of each blind zone in the first image, and fill the extracted images of each blind zone in the first image into the images corresponding to the blind zones between adjacent visual sensors in the monitoring image to obtain the basic image of each blind zone.

6. The method for correcting deviations in a dynamic adaptive sensing device according to claim 5, characterized in that, Specifically, S33 includes: S331. Obtain the monitoring range of each visual sensor after the vehicle body is stationary, and establish a monitoring network for monitoring the environment and objects around the vehicle body based on the monitoring range of each visual sensor. S332. Divide the objects on the monitoring network into static objects and dynamic objects. Static objects can be converted into dynamic objects. Static objects are objects that do not have displacement on the monitoring network, while dynamic objects are objects that have displacement on the monitoring network. S333. Determine the shape and specification information of each object on the monitoring network and bind and associate the shape and specification information with the corresponding object. Extract the outline information of each object based on the shape and specification information of each object. S334. Obtain the movement trajectory of dynamic objects on the monitoring network. When a dynamic object enters a blind zone, automatically generate a predicted trajectory of the dynamic object based on the movement trajectory. Add the dynamic object that has entered the blind zone and the predicted trajectory to the blind zone of the monitoring area, thereby completing the correction of the images corresponding to each blind zone in the monitoring image.

7. The method for correcting deviations in a dynamic adaptive sensing device according to claim 6, characterized in that, The steps for obtaining the movement trajectory of dynamic objects on the monitoring network specifically include: Number the moving objects on the monitoring network and mark their location information on the monitoring network; By connecting the marked location information in chronological order, the movement trajectory of dynamic objects on the monitoring network can be obtained.

8. The method for correcting deviations in a dynamic adaptive sensing device according to claim 7, characterized in that, The steps for automatically generating the predicted trajectory of a dynamic object based on its movement trajectory specifically include: Obtain the tangent sequence of the dominant edge of a dynamic object on its movement trajectory, and generate a predicted trajectory of the dynamic object based on the changing trend of the tangent in the tangent sequence. The tangent sequence consists of the tangents of the dominant edge of the dynamic object at n time points on its movement trajectory. If the trend of the change in the tangent angle is zero or close to zero, then the movement trajectory of the dynamic object is used as the predicted trajectory. If the trend of the chamfer is to gradually decrease, then obtain the decreasing speed corresponding to the chamfer, and use the movement trajectory corresponding to the decreasing speed as the predicted trajectory of the dynamic object in the blind zone. If the trend of the chamfer angle is to gradually increase, then obtain the increase rate corresponding to the chamfer angle, and use the movement trajectory corresponding to the increase rate as the predicted trajectory of the dynamic object in the blind zone. If the trend of the chamfer change is alternating between increasing and decreasing, and if the increase and decrease are regular, then the movement trajectory corresponding to the regular increase and decrease is used as the predicted trajectory of the dynamic object in the blind zone. If the increase and decrease are irregular, then the range of motion and displacement velocity of the dynamic object are obtained, and m possible candidate trajectories are generated in the blind zone corresponding to the dynamic object based on the range of motion and displacement velocity. The optimal trajectory among the m possible candidate trajectories is used as the predicted trajectory of the dynamic object in the blind zone.

9. The method for correcting deviations in a dynamic adaptive sensing device according to claim 8, characterized in that, The step of obtaining the optimal trajectory from m possible candidate trajectories as the predicted trajectory of the dynamic object in the blind zone specifically includes: Determine the maximum changing angle and maximum displacement velocity of the dynamic object on the monitoring network, and set the activity range of the dynamic object based on the maximum changing angle and maximum displacement velocity; By randomly generating m possible candidate trajectories within the Monte Carlo sampling activity range, and filtering the m possible candidate trajectories based on the set physical constraints, the similarity value between the remaining candidate trajectories after filtering and the trajectory of the moving object is calculated using DTW distance, and the candidate trajectory with the highest similarity value is taken as the predicted trajectory of the moving object in the blind zone. Among these, physical constraints include the feasibility of terrain conditions.

10. A method and system for correcting the bias of a dynamic adaptive sensing device, used to implement the method for correcting the bias of a dynamic adaptive sensing device as described in any one of claims 1 to 9, characterized in that, include: The blind spot calibration module is used to determine the blind spots between each pre-calibrated adjacent visual sensor on the vehicle body; The image acquisition module is used to acquire the image of the vehicle body in the previous cycle before parking and record it as the first image; The image correction module is used to acquire images of the vehicle body in the current cycle and record them as monitoring images. It uses the first image and the monitoring images to correct the images corresponding to each blind spot in the monitoring images, so as to obtain the corrected images corresponding to each blind spot. In the image correction module, the correction process specifically includes: acquiring the basic image of each blind zone using the first image, establishing a monitoring network and generating the predicted trajectory of each object in the corresponding blind zone based on the movement trajectory of each object, and adding the predicted trajectory of each object to the basic image of the corresponding blind zone, thereby completing the correction of the image corresponding to each blind zone in the monitoring image.

Images