Method for adjusting splicing seam of spliced image

Abstract

The invention discloses a method for adjusting a splicing seam of a spliced image in the technical field of image processing, and aims to solve the technical problem that an existing image splicing method cannot perform quick image adjustment on a moving object which is about to enter a view blind area. The method comprises the steps that real-time early warning of a moving object can be carried out according to the recognition result of an object recognition model on an original image, the rapid response requirement is met, and the relation between the world coordinate azimuth angle of a recognition frame of the moving object and the initial splicing seam azimuth angle serves as the judgment standard for the moving object to enter a view blind area. The adjusting angle is directly called from the preset mapping table according to the world coordinate azimuth angle of the recognition frame so as to adjust and control the azimuth angle of the splicing seam, and the effects of rapidly responding and rapidly eliminating the view blind area can be achieved.

Description

Technical Field

[0001] The invention relates to a method for adjusting a splicing seam of spliced images, and belongs to the technical field of image processing. Background Art

[0002] As safety requirements for construction machinery continue to increase, the application of panoramic video systems for excavators is gradually increasing. Panoramic video can eliminate blind spots in the vision of equipment operators and improve the safety of equipment use.

[0003] Existing panoramic surround view systems for excavators typically use a fixed seam approach. This involves correcting and stitching images from multiple cameras through an image processing unit to create a seamless, complete panoramic bird's-eye view, which is then displayed on the excavator's monitor. If a pedestrian or other moving object approaches the seam, two separate projections of the same object, one on each side of the seam, are created by two different cameras. When the bird's-eye view is stitched together, both projections are covered and cannot be displayed, causing the pedestrian or other object to disappear from the image, creating a blind spot. This situation poses a threat to the operational safety of the equipment.

[0004] Therefore, the existing image stitching method cannot quickly adjust the image of a moving object that is about to enter the blind spot of the visual field. Summary of the Invention

[0005] The purpose of the present application is to overcome the deficiencies in the prior art and provide a method for adjusting the seams of spliced images that reduces the impact of blind spots in the field of view and ensures rapid processing of image adjustments.

[0006] To achieve the above objectives, this application is implemented using the following technical solutions: In a first aspect, the present application provides a method for adjusting a stitching seam of stitched images, comprising: Acquire an original image, and obtain a recognition result of the original image by an object recognition model; In response to identifying the moving object, generating an identification frame of the moving object; In response to the angular difference between the world coordinate azimuth of the identification frame and the initial seam azimuth being less than a set threshold, an adjustment angle is retrieved from a preset mapping table according to the world coordinate azimuth of the identification frame, and the seam azimuth is updated according to the adjustment angle.

[0007] In some embodiments of the first aspect, obtaining a recognition result of the object recognition model on the original image includes: Uploading the original image to a host controller configured with an object recognition model at intervals of a preset first period; In response to identifying a moving object, the original image is uploaded to the object recognition cloud at intervals of a preset second period, where the second period is shorter than the first period.

[0008] In some embodiments of the first aspect, the response to the world coordinate azimuth of the identification frame and the seam azimuth being less than a set threshold includes: Obtaining pixel coordinates of the identification frame; Obtaining the world coordinates of the recognition frame according to the pixel coordinates and the camera's external parameter calibration; The world coordinate azimuth of the identification frame is obtained according to the world coordinate of the identification frame.

[0009] In some embodiments of the first aspect, acquiring the original image includes: Get the camera's intrinsic calibration; Obtaining an intrinsic parameter matrix and distortion parameters according to the intrinsic parameter calibration; The original image is subjected to reverse mapping interpolation correction based on the intrinsic parameter matrix and the distortion parameter.

[0010] In some embodiments of the first aspect, the recognition objects of the object recognition model include pedestrians in walking, standing, and squatting postures, and also include vehicles.

[0011] In some embodiments of the first aspect, the updated seam azimuth is used to perform Poisson fusion on the original images to obtain a panoramic image.

[0012] In a second aspect, the present application further provides a device for adjusting a stitching seam of stitched images, the device for adjusting a stitching seam of stitched images comprising: A host controller, configured to output a recognition result based on the original image and the object recognition model; An image processor, configured to execute the method for adjusting a stitching seam of stitched images according to any one of the embodiments of the first aspect; The host controller is signal-connected to the image processor.

[0013] In a third aspect, the present application also provides a computer device comprising a processor and a memory connected to the processor, wherein a computer program is stored in the memory. When the computer program is executed by the processor, the steps of the method for adjusting the stitching seam of the stitched image as described in any embodiment of the first aspect are performed.

[0014] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method for adjusting the stitching seam of stitched images described in any embodiment of the first aspect.

[0015] In a fifth aspect, the present application also provides a computer program product, including a computer program / instruction, characterized in that when the computer program / instruction is executed by a processor, the steps of the method for adjusting the stitching seam of stitched images described in any embodiment of the first aspect are implemented.

[0016] Compared with the prior art, the present invention has the following beneficial effects: The method for adjusting the stitching seam of a stitched image provided in the present application can provide real-time early warning of a moving object based on the recognition result of the original image by an object recognition model, thereby meeting the requirement of rapid response. The relationship between the world coordinate azimuth of the recognition frame of the moving object and the initial stitching seam azimuth is used as a simple and clear criterion for judging whether the moving object is about to enter a blind spot of the field of view. The adjustment angle is directly called up in a preset mapping table according to the world coordinate azimuth of the recognition frame to adjust the stitching seam azimuth, which can achieve the effect of rapid response and rapid elimination of blind spots of the field of view. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present application or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0018] Figure 1 is a flowchart of the steps of the method for adjusting the stitching seam of stitched images provided in this embodiment; Figure 2 Schematic diagram of the structure of the device for adjusting the stitching seam of stitched images provided in this embodiment; Figure 3 is a schematic block diagram of the principles of the computer device provided in this embodiment; In the figure: 1. Host controller; 2. Image processor. DETAILED DESCRIPTION

[0019] The technical solution of the present invention is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present application and the specific features in the embodiments are detailed descriptions of the technical solution of the present application, rather than limitations on the technical solution of the present application. Unless there is a conflict, the embodiments of the present application and the technical features in the embodiments can be combined with each other.

[0020] The term "and / or" in this document simply describes a relationship between related objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, or B exists alone. Additionally, the character " / " in this document generally indicates an "or" relationship between the related objects. Example 1

[0021] Figure 1 This is a flow chart of a method for adjusting the seam of stitched images in the first embodiment of the present invention. This flow chart only shows the logical sequence of the method described in this embodiment. In other possible embodiments of the present invention, different methods may be used without conflict. Figure 1 The steps shown or described are accomplished in the order shown.

[0022] See also Figure 1 , the method of this implementation specifically includes the following steps: Obtain an original image and obtain a recognition result of the object recognition model on the original image; the original image is an unstitched image captured by a camera. Since the object recognition model has high requirements for computing resources, a distributed deployment architecture can be adopted to separate the model from the image processing system. After sending the original image to the object recognition model, the recognition result output by the object recognition model is obtained; In response to identifying a moving object, generating an identification frame of the moving object; in subsequent steps, using the identification frame instead of pixels of the moving object to adjust the stitching seam, thereby reducing the amount of calculation and improving control efficiency; In response to the angular difference between the world coordinate azimuth of the identification frame and the initial stitching seam azimuth being less than a set threshold, indicating that the pixels of the moving object in the identification frame are about to enter the blind spot of the field of view of the stitched image, an adjustment angle is retrieved from a preset mapping table according to the world coordinate azimuth of the identification frame, and the stitching seam azimuth is updated according to the adjustment angle. Angle adjustment according to the preset mapping table can achieve a faster response than angle calculation.

[0023] The method for adjusting the stitching seam of a stitched image provided in this embodiment can provide real-time early warning of a moving object based on the recognition result of the original image by the object recognition model, meeting the requirement of rapid response. The relationship between the world coordinate azimuth of the recognition frame of the moving object and the initial stitching seam azimuth is used as a simple and clear criterion for judging whether the moving object is about to enter a blind spot of the field of view. The adjustment angle is directly retrieved from a preset mapping table based on the world coordinate azimuth of the recognition frame to adjust the stitching seam azimuth, which can achieve a rapid response and quickly eliminate blind spots of the field of view. Example 2

[0024] This embodiment provides a method for adjusting a stitching seam of stitched images. This embodiment is optimized based on the first embodiment to improve the technical effect and refine the technical solution. For details not fully described in this embodiment, please refer to the first embodiment.

[0025] As one embodiment, the object recognition model obtains the recognition result of the original image, including uploading the original image to the host controller 1 configured with the object recognition model at intervals of a preset first period; in response to the recognition of a moving object, uploading the original image to the object recognition cloud at intervals of a preset second period, wherein the second period is shorter than the first period. Under normal conditions, the subject executing the method for adjusting the stitching seam of the stitched image provided in this embodiment uploads the original image to the host controller 1 at a relatively low frequency. When the host controller 1 sends the result of identifying a moving object to the subject executing the method for adjusting the stitching seam of the stitching image provided in this embodiment, the original image is uploaded at a higher frequency to ensure that the recognition frame can be obtained in real time, while taking into account the saving of computing resources, so as to meet the requirements of multiple cameras on the excavator being processed by the same host controller 1 for image recognition.

[0026] In one embodiment, the response to the world coordinate azimuth angle of the recognition frame and the seam azimuth angle being less than a set threshold includes: Obtain the pixel coordinates of the recognition frame; obtain the world coordinates of the recognition frame based on the pixel coordinates and the camera's extrinsic calibration, that is, use the camera's extrinsic calibration to convert the pixel coordinates to obtain the actual azimuth of the moving object relative to the camera; obtain the world coordinate azimuth of the recognition frame based on the world coordinates of the recognition frame.

[0027] Because some cameras around the excavator use ultra-wide-angle lenses to obtain a larger field of view and reduce the number of cameras deployed, the original image needs to be corrected to obtain a more accurate world coordinate azimuth in subsequent steps. Acquiring the original image includes obtaining the camera's intrinsic calibration; obtaining an intrinsic parameter matrix and distortion parameters based on the intrinsic calibration; and performing reverse mapping interpolation correction on the original image based on the intrinsic parameter matrix and distortion parameters.

[0028] As one embodiment, the object recognition model recognizes objects including pedestrians in walking, standing, and squatting postures, and also includes vehicles.

[0029] As one embodiment, the updated stitching seam azimuth is used to perform Poisson fusion on the original images to obtain a panoramic image.

[0030] As one embodiment, after obtaining the updated stitching seam azimuth according to the adjustment angle, the original images are stitched according to the new stitching seam azimuth, and the mapping table pre-sets the stitching seam azimuths corresponding to the world coordinate azimuths of multiple recognition frames.

[0031] In one embodiment, in response to identifying a moving object, an alarm message is output. The excavator operator's central control screen can alert the operator based on the alarm message and provide a recognition image and the type of person or vehicle identified. The distance of the moving object can also be calculated based on the rectified original images captured by the two cameras and the camera's layout coordinates relative to the excavator body. Example 3

[0032] refer to Figure 2 This embodiment provides a device for adjusting a stitching seam of stitched images, comprising: Host controller 1, used for outputting recognition results based on the original image and the object recognition model; An image processor 2, configured to execute the method for adjusting the stitching seam of stitched images according to any one of the first and second embodiments; The host controller 1 and the image processor 2 are connected via a CAN bus.

[0033] The host controller 1 may be arranged on the vehicle controller VCU.

[0034] The method for adjusting the stitching seams of stitched images provided in Example 1 or 2 can be applied to the device for adjusting the stitching seams of stitched images provided in this embodiment, and has functional modules and beneficial effects corresponding to the execution method. The device for adjusting the stitching seams of stitched images provided in this embodiment has the same technical effects as those in Example 1 or 2, and will not be repeated here. Example 4

[0035] This embodiment provides a computer device, including a processor and a memory connected to the processor, wherein a computer program is stored in the memory. When the computer program is executed by the processor, the steps of the method for adjusting the stitching seam of stitched images provided in Embodiment 1 or 2 are performed.

[0036] The computer device may be a server or an electronic terminal. As one embodiment, Figure 3The computer device includes a processor, a memory, an input / output interface (I / O) and a communication interface. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data obtained and generated in the method for adjusting the splicing seam of the spliced image. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, the method for adjusting the splicing seam of the spliced image provided in embodiment one or two is implemented.

[0037] Those skilled in the art will understand that Figure 3 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0038] The computer device provided in this embodiment has the same technical effects as those in Embodiment 1 or 2, and will not be described in detail here. Example 5

[0039] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the steps of the method for adjusting the stitching seam of stitched images provided in the first or second embodiment are implemented.

[0040] The computer-readable storage medium provided in this embodiment has the same technical effects as those in Embodiment 1 or 2, and will not be described in detail here. Example 6

[0041] This embodiment provides a computer program product having a computer program stored thereon, which, when executed by a processor, implements the steps of the method for adjusting the stitching seam of stitched images provided in Embodiment 1 or Embodiment 2. The computer program product provided in this embodiment can be transmitted, distributed, and downloaded in the form of a signal via the Internet.

[0042] The computer program product provided in this embodiment has the same technical effects as those in Embodiment 1 or 2, and will not be described in detail here.

[0043] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0044] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0045] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0046] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0047] Furthermore, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature identified with "first," "second," etc., may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "plurality" means two or more.

[0048] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense. For example, they may refer to fixed connections, detachable connections, or integral connections; mechanical connections or electrical connections; direct connections or indirect connections through an intermediate medium; and internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0049] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for adjusting a stitching seam of stitched images, characterized in that: include, Acquire an original image, and obtain a recognition result of the original image by an object recognition model; In response to identifying the moving object, generating an identification frame of the moving object; In response to the angular difference between the world coordinate azimuth of the identification frame and the initial seam azimuth being less than a set threshold, an adjustment angle is retrieved from a preset mapping table according to the world coordinate azimuth of the identification frame, and the seam azimuth is updated according to the adjustment angle.

2. The method for adjusting the stitching seam of stitched images according to claim 1, wherein: The obtaining of the recognition result of the object recognition model on the original image includes: Uploading the original image to a host controller (1) configured with an object recognition model at intervals of a preset first period; In response to identifying a moving object, the original image is uploaded to the object recognition cloud at intervals of a preset second period, where the second period is shorter than the first period.

3. The method for adjusting the stitching seam of stitched images according to claim 1, wherein: The step of responding to the world coordinate azimuth angle of the recognition frame and the seam azimuth angle being less than a set threshold value includes: Obtaining pixel coordinates of the identification frame; Obtaining the world coordinates of the recognition frame according to the pixel coordinates and the camera's external parameter calibration; The world coordinate azimuth of the identification frame is obtained according to the world coordinate of the identification frame.

4. The method for adjusting the stitching seam of stitched images according to claim 1, wherein: The obtaining of the original image includes: Get the camera's intrinsic calibration; Obtaining an intrinsic parameter matrix and distortion parameters according to the intrinsic parameter calibration; The original image is subjected to reverse mapping interpolation correction based on the intrinsic parameter matrix and the distortion parameter.

5. The method for adjusting the stitching seam of stitched images according to claim 1, wherein: The object recognition model can recognize pedestrians in walking, standing, and squatting postures, as well as vehicles.

6. The method for adjusting the stitching seam of stitched images according to claim 1, wherein: The updated stitching seam azimuth is used to perform Poisson fusion between the original images to obtain a panoramic image.

7. A device for adjusting the seam of spliced images, characterized in that: The device for adjusting the stitching seam of the stitched images comprises: A host controller (1) is used to output a recognition result based on the original image and the object recognition model; An image processor (2), configured to execute the method for adjusting a stitching seam of stitched images according to any one of claims 1 to 6; The host controller (1) and the image processor (2) are signal-connected.

8. A computer device, characterized in that: The method comprises a processor and a memory connected to the processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the steps of the method for adjusting the stitching seam of stitched images according to any one of claims 1 to 6 are executed.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method for adjusting the stitching seam of stitched images according to any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the steps of the method for adjusting the stitching seam of stitched images according to any one of claims 1 to 6 are implemented.

Images