Method and system for collecting front view image of working machine and working machine

Abstract

The invention relates to the technical field of image display, in particular to a front-view image acquisition method and acquisition system of an operation machine and the operation machine, and the front-view image acquisition method comprises the following steps: controlling two image acquisition pieces to respectively carry out front-view image acquisition on the operation machine to obtain two first front-view images; position calibration is carried out on the two image acquisition pieces, so that preset calibration points of the working device in the two first front view images are distributed in a mirror symmetry mode; obtaining two groups of coordinate points of the working device in the two first front view images; generating an outer contour corresponding to the working device according to the two groups of coordinate points; performing image processing on the two first front-view images to obtain two second front-view images without outer contours; and fitting and splicing the two second front-view images to obtain an unshielded front-view image of the working machine. By the adoption of the front-view image acquisition method, the unshielded front-view image of the operation machine can be obtained, and the view blind area of a driver during operation is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image display, in particular to a front view image acquisition method and system of a working machine, the working machine and a storage medium. BACKGROUND

[0002] When excavators are working, the right front is often blocked by the boom, bucket and other working devices, which is in the blind area of the field of view, and is prone to safety risks. The common look-around system or right front auxiliary camera can play a role in supplementing the blind area, but the view is scattered, and the driver is difficult to form a unified connection between the scene in front of the eyes and the screen auxiliary scene.

[0003] The existing forward blind area supplement method of the excavator mainly includes the following two kinds, one of which is to add a front view camera, and the other is to supplement the blind area through 360-degree look-around. The front view camera is generally installed on the right front of the excavator to supplement the field of view blocked by the boom and bucket, but the front view has obvious parallax with the driver's visual view, and the driver needs to combine the information of the two views, which leads to misjudgment. The 360-degree look-around can combine multiple views, but the front view is still blocked and cannot restore the complete front view. SUMMARY

[0004] The purpose of the embodiment of the present application is to provide a front view image acquisition method and system of a working machine, the working machine and a storage medium, to solve the technical problem of the existing working machine that the front view is blocked.

[0005] In order to achieve the above-mentioned purpose, the first aspect of the present application provides a front view image acquisition method of a working machine, the working machine comprising a vehicle body, a working device arranged at the front end of the vehicle body, and two image acquisition devices arranged on the vehicle body and located on both sides of the working device, the two image acquisition devices are used to acquire the front view image of the working machine, the front view image acquisition method comprises: controlling the two image acquisition devices to acquire the front view image of the working machine respectively and obtaining two first front view images; positioning the two image acquisition devices to make the preset calibration points of the working device in the two first front view images mirror-symmetrically distributed; obtaining two groups of coordinate points of the working device in the two first front view images; generating the outer contour corresponding to the working device according to the two groups of coordinate points; performing image processing on the two first front view images to obtain two second front view images without the outer contour; fitting and splicing the two second front view images to obtain the non-blocked front view image of the working machine.

[0006] In the embodiment of the present application, the step of position calibration of the two image acquisition units to make the preset calibration points of the working device in the two first front view images mirror-symmetrically distributed comprises: adjusting the positions of the two image acquisition units to make the two image acquisition units at the same height and symmetrically arranged relative to the working device; adjusting the acquisition directions of the two image acquisition units according to the first front view images to obtain two third front view images, wherein the vertical coordinate axis of the first front view images is perpendicular to the horizontal direction; and performing software calibration on the two third front view images to make the two groups of preset calibration points mirror-symmetrically distributed.

[0007] In the embodiment of the present application, the step of performing software calibration on the two third front view images to make the two groups of preset calibration points mirror-symmetrically distributed comprises: acquiring the preset calibration points of the working device in the two third front view images respectively; mirror mapping the preset calibration points in one of the third front view images in the other third front view image to obtain the position difference corresponding to the two groups of preset calibration points; and adjusting the acquisition direction of the image acquisition unit according to the position difference until the position difference is zero.

[0008] In the embodiment of the present application, the step of generating the outer contour corresponding to the working device according to the two groups of coordinate points comprises: mirror mapping the coordinate points in one of the first front view images in the other first front view image to obtain a fourth front view image; and taking the contour composed of the coincident points of the two groups of coordinate points in the fourth front view image as the outer contour.

[0009] In the embodiment of the present application, the front view image acquisition method further comprises the following steps: acquiring the contour attribute corresponding to the working device; and performing noise removal processing on the outer contour according to the contour attribute to obtain a precise contour after noise removal.

[0010] In the embodiment of the present application, after mirror mapping the coordinate points in one of the first front view images in the other first front view image to obtain a fourth front view image, and before taking the contour composed of the coincident points of the two groups of coordinate points in the fourth front view image as the outer contour, the method further comprises the following steps: acquiring the display range of the working device in the first front view image; and removing the coordinate points outside the display range and on the boundary of the display range, and all coordinate points in the contour corresponding to the boundary coordinate points, to obtain the coordinate points in the display range.

[0011] In the embodiment of the present application, the step of performing image processing on the first front view images to obtain two second front view images with the outer contour removed comprises: performing dilation processing on the outer contour to obtain a dilated outer contour after dilation processing; mapping the dilated outer contour in the two first front view images respectively; and removing the dilated outer contour in the two first front view images respectively to obtain two second front view images.

[0012] The second aspect of the present application provides a front view image acquisition system of a working machine, comprising: a memory configured to store instructions; and a processor configured to call the instructions from the memory and enable the above-mentioned front view image acquisition method of the working machine when executing the instructions.

[0013] The third aspect of the present application provides a working machine, comprising: the above-mentioned front view image acquisition system of the working machine; a vehicle body; a working device provided at the front end of the vehicle body and comprising an arm frame and an implement provided on the arm frame; and two image acquisition units provided on the vehicle body and located on both sides of the working device, both of which are used to acquire front view images of the working machine.

[0014] The fourth aspect of the present application provides a machine-readable storage medium having instructions stored thereon for causing a machine to perform the above-mentioned front view image acquisition method of the working machine.

[0015] In the above technical solution, a front view image acquisition method of a working machine is provided, the working machine comprising a vehicle body, a working device provided at the front end of the vehicle body, and two image acquisition units provided on the vehicle body and located on both sides of the working device, both of which are used to acquire front view images of the working machine. The front view image acquisition method for the working machine comprises the following steps: controlling the two image acquisition units to respectively acquire front view images of the working machine and obtaining two first front view images. Both of the two first front view images include an environmental image in front of the working machine and an image on one side of the working device. The two image acquisition units are positionally calibrated so that preset calibration points of the working device in the two first front view images are mirror-symmetrically distributed. When the preset calibration points are mirror-symmetrically distributed, the angles and positions at which the two image acquisition units acquire the first front view images are relatively accurate, which can ensure the accuracy of subsequent image processing steps. Then, two groups of coordinate points of the working device in the two first front view images are obtained, and an outer contour corresponding to the working device is generated according to the two groups of coordinate points. The working device will cause occlusion to the front view images. The front view image acquisition method of the present application can perform image processing on the two first front view images after identifying the outer contour, so as to obtain two second front view images from which the outer contour is removed. Finally, the two second front view images are fitted and spliced to obtain an unoccluded front view image of the working machine. By using the above-mentioned front view image acquisition method, the problem of visual field blind area caused by the occlusion of the working device during the operation of the working machine can be effectively avoided, and the operation safety and working efficiency of the working machine are improved.

[0016] Other features and advantages of the embodiments of the present application will be described in detail in the following specific implementation part. BRIEF DESCRIPTION OF DRAWINGS

[0017] The accompanying drawings are included to provide a further understanding of embodiments of the application, and are incorporated in and constitute a part of the specification, illustrate embodiments of the application and serve to explain the principles of the application, but are not intended to limit the application. In the drawings: Figure 1 A flowchart schematically shows a front-view image acquisition method of a working machine according to an embodiment of the application; Figure 2 A structural diagram of a working machine according to an embodiment of the application is schematically shown; Figure 3 A process of external contour identification and removal according to an embodiment of the application is schematically shown; Figure 4 Image diagrams of two first front-view images according to an embodiment of the application are schematically shown; Figure 5 Image diagrams of two first front-view images according to another embodiment of the application are schematically shown; Figure 6 An internal structure diagram of a computer device according to an embodiment of the application is schematically shown. DETAILED DESCRIPTION

[0018] In order to make the objects, technical solutions and advantages of the embodiments of the application clearer, the technical solutions in the embodiments of the application will be described clearly and completely below with reference to the drawings in the embodiments of the application. It should be understood that the specific embodiments described herein are merely used to explain and illustrate the embodiments of the application, and are not used to limit the embodiments of the application. Based on the embodiments of the application, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the application.

[0019] It should be noted that the acquisition, transmission, storage, use, processing and the like of data in the technical solutions of the application comply with relevant provisions of laws and regulations. In the embodiments of the application, some existing industry solutions such as software, components, models and the like may be mentioned, which should be considered as exemplary, and the purpose is only to illustrate the feasibility of the implementation of the technical solutions of the application, but does not mean that the applicant has or will necessarily use the solutions.

[0020] It should be noted that if the embodiments of the application involve directional indications (such as up, down, left, right, front, back, etc.), the directional indications are only used to explain the relative positional relationship, movement condition and the like between components in a certain posture (as shown in the drawings), and if the certain posture changes, the directional indications also change accordingly.

[0021] In addition, if the description of "first", "second" and the like is involved in the embodiments of the present application, the description of "first", "second" and the like is only for the purpose of description, and cannot be understood as indicating or implying the relative importance of the indicated technical features or implicitly indicating the number of the indicated technical features. Therefore, the features limited by "first", "second" can be explicitly or implicitly included at least one of the features. In addition, the technical solutions of various embodiments can be combined with each other, but it must be based on the realization of ordinary skilled in the art, when the combination of technical solutions appears contradictory or unachievable, it should be considered that the combination of technical solutions does not exist, nor in the protection scope required by the present application.

[0022] Figure 1 The flowchart of the front view image acquisition method of the working machine according to the embodiment of the present application is schematically shown. As shown in Figure 2 The structural diagram of the working machine provided by the embodiment of the present application is shown. The embodiment of the present application provides a front view image acquisition method of a working machine, which includes a vehicle body, a working device arranged at the front end of the vehicle body, and two image acquisition devices arranged on the vehicle body and located on both sides of the working device. The two image acquisition devices are used to acquire the front view image of the working machine. The front view image acquisition method can include the following steps: S101, controlling the two image acquisition devices to acquire the front view image of the working machine respectively and obtaining two first front view images; S102, calibrating the positions of the two image acquisition devices, so that the preset calibration points of the working device in the two first front view images are mirror-symmetrically distributed; S103, obtaining two groups of coordinate points of the working device in the two first front view images; S104, generating the outer contour corresponding to the working device according to the two groups of coordinate points; S105, performing image processing on the two first front view images to obtain two second front view images without the outer contour; S106, fitting and splicing the two second front view images to obtain the unobstructed front view image of the working machine.

[0023] When driving the working machine such as excavator and skid steer loader, the front view of the driver is often blocked by the boom and accessory, resulting in partial view blind area and safety hazard. In the prior art, there is a scheme of displaying the view of the other side through the center control screen, but this requires the driver to combine the view in front of the window with the view of the center control screen, which is not only easy to misjudge, but also leads to poor driving experience.

[0024] The present invention provides a forward-view image acquisition method for a work machine to address the aforementioned issues. First, a processor of the work machine controls two image acquisition components to capture forward-view images of the work machine, respectively, to obtain two first forward-view images. The two image acquisition components are positioned on either side of the work machine's body and on either side of the working device, ensuring a comprehensive view of the area in front of the work machine. Both first forward-view images contain images of the environment in front of the work machine and images of the working device. After obtaining the two first forward-view images, the two image acquisition components are positionally calibrated so that the preset calibration points of the working device in the two first forward-view images are mirror-symmetrically distributed. This step ensures the relative accuracy of the angles and positions of the two image acquisition components, providing a foundation for subsequent image processing steps. Position calibration can be achieved by adjusting the position and acquisition direction of the image acquisition components until the preset calibration points in the two first forward-view images are mirror-symmetrically distributed. Next, the processor obtains two sets of coordinate points of the working device in the two first forward-view images. These coordinate points represent the position of the working device in the image and serve as the basis for generating the corresponding outer contour of the working device. Based on these two sets of coordinate points, the corresponding outer contour of the working device can be generated. This outer contour represents the occlusion range of the working device in the two first front-view images.

[0025] Figure 3 This schematic diagram illustrates the outline recognition and removal process provided by an embodiment of the present invention. After identifying the outline, the processor performs image processing on the two first front view images to remove the outline, thereby generating two second front view images without the outline removed. Finally, the processor performs fitting and splicing on the two second front view images to obtain an unobstructed front view image of the working machine.

[0026] The above-mentioned front view image acquisition method can obtain an unobstructed front view image of the operating machinery. This unobstructed front view image provides the driver with a comprehensive front view, avoids the blind spot problem caused by obstruction of the working device, improves the operational safety and efficiency of the operating machinery, and optimizes the driver's driving experience.

[0027] In one embodiment, the step of calibrating the positions of the two image acquisition components so that the preset calibration points of the working device in the two first front view images are distributed in a mirror-symmetrical manner includes: adjusting the positions of the two image acquisition components so that the two image acquisition components are at the same height and are arranged symmetrically relative to the working device; adjusting the acquisition directions of the two image acquisition components according to the first front view image to obtain two third front view images, wherein the vertical coordinate axis of the first front view image is perpendicular to the horizontal direction; and performing software calibration on the two third front view images so that the two sets of preset calibration points are distributed in a mirror-symmetrical manner.

[0028] The process of position calibration of the two image acquisition devices further comprises physical calibration and software calibration. The image acquisition devices can be physically calibrated first, and then the collection directions of the two image acquisition devices are fine-tuned through software calibration. Specifically, the positions of the two image acquisition devices are first adjusted so that the two image acquisition devices are arranged symmetrically relative to the working device at the same height, ensuring that they can capture a comprehensive view in front of the working machine and obtain images from two opposite positions, providing a basis for subsequent image processing steps. As shown in Figure 2 , after the two image acquisition devices are arranged symmetrically relative to the working device, , wherein and are the distances of the two image acquisition devices relative to the working device. Subsequently, the collection directions of the two image acquisition devices are further adjusted according to the first front view image to obtain two third front view images. After this process is completed, the vertical coordinate axis of the third front view image is perpendicular to the horizontal direction, which helps to improve the accuracy and efficiency of subsequent image processing. Finally, the two third front view images are software calibrated, and the collection directions of the two image acquisition devices are adjusted through an algorithm until two sets of preset calibration points are symmetrically distributed in the image. This step ensures that the angles and positions of the two image acquisition devices are relatively accurate, providing a reliable basis for subsequent generation of unobstructed front view images. By combining physical calibration and software calibration, the front view image acquisition method of the present application can achieve accurate calibration of the image acquisition devices, so that subsequent image processing steps can generate an outer contour according to the coordinate points in the third front view image.

[0029] In one specific embodiment, the vertical coordinate axis of the third front view image is determined to be perpendicular to the horizontal direction when the coordinates of the two preset calibration points satisfy the following formula (1).

[0030] (1) wherein a point and b point are two preset calibration points with the same height, and are the longitudinal coordinates of a point and b point in the left third front view image, and a point and b point can be selected as the two top vertices of the bucket.

[0031] In one embodiment, as Figure 4As shown, the image schematic diagram of two first front view images provided by the embodiment of the present application, wherein each of the two first front view images comprises a preset calibration point, and the first front view image on the right side further comprises a group of mirror mapped preset calibration points. The step of software calibration of the two third front view images to make the two groups of preset calibration points mirror symmetrically distributed comprises: acquiring the preset calibration points of the working device in the two third front view images respectively; mirror mapping the preset calibration points in one of the third front view images in the other third front view image to obtain the position difference corresponding to the two groups of preset calibration points; and adjusting the collection direction of the image collection member according to the position difference until the position difference is zero. In order to make the working devices in the two images radially symmetrically distributed, the preset calibration points of the working devices in the two third front view images need to be acquired respectively, and the preset calibration points can be a plurality of vertices on the implement or the boom. For example, when the working machine is an excavator, the preset calibration points can be the four vertices of the opening of the bucket. Then the preset calibration points in one of the third front view images are mirror mapped in the other third front view image to obtain the mirror mapped preset calibration points. By calculating the position difference between the two groups of preset calibration points, it can be evaluated whether the relative position and angle between the two image collection members are accurate. If the position difference is not zero, it means that the position or angle of the two image collection members needs to be adjusted. According to the calculated position difference, the collection direction of the image collection member is adjusted accordingly until the position difference is zero, that is, the preset calibration points in the two third front view images are completely mirror symmetrically distributed. This software calibration method can accurately adjust the position and angle of the image collection member, and ensure the accuracy and efficiency of subsequent image processing.

[0032] In a specific embodiment, the position difference is determined to be zero in the case that the coordinates of the preset calibration points in the third front view image satisfy the following formula (2).

[0033]

[0034] wherein the points a and b are two preset calibration points with the same height, and are the horizontal coordinate and the vertical coordinate of the first preset calibration point in the left third front view image, and are the horizontal coordinate and the vertical coordinate of the second preset calibration point in the left third front view image, and are the horizontal coordinate and the vertical coordinate of the first preset calibration point in the right third front view image, and are the horizontal coordinate and the vertical coordinate of the second preset calibration point in the right third front view image. Specifically, the points a and b can be selected as two top vertices of the bucket.

[0035] In one embodiment, the step of generating the outer contour corresponding to the working device based on two sets of coordinate points includes: mirroring the coordinate points in one of the first front view images onto the other first front view image to obtain a fourth front view image; and using the outline formed by the overlapping points of the two sets of coordinate points in the fourth front view image as the outer contour. After the image acquisition component is calibrated, the working devices in the two first front view images are roughly mirror-symmetrically distributed, and the coordinate points corresponding to the working devices are also roughly symmetrically distributed. Therefore, by mirroring the coordinate points in one of the first front view images onto the other first front view image, a virtual fourth front view image can be obtained. Subsequently, in the fourth front view image, the overlapping points of the two sets of coordinate points are found, and the outline formed by these overlapping points is the outer contour corresponding to the working device. This outer contour represents the common occlusion range of the working device in the two first front view images and is the key basis for subsequent image processing to remove occlusion. Through this method, the outer contour of the working device can be accurately identified and generated, providing a reliable foundation for subsequent image processing steps.

[0036] In one embodiment, the front view image acquisition method further includes the following steps: obtaining contour attributes corresponding to the working device; and performing noise removal on the outer contour according to the contour attributes to obtain an accurate contour after noise removal. Contour attributes are the perimeter, area, mean, histogram distribution of contour features, etc. of the contour obtained after the processor calculates the geometric center position of the contour. The above-mentioned contour attributes can be used to match the contour with the outer contour, and then perform noise removal on the outer contour to exclude coordinate points that obviously do not belong to the working device, thereby obtaining an accurate contour after noise removal. The above-mentioned method can further reduce errors caused by image noise or misidentification, and improve the accuracy and efficiency of subsequent image processing.

[0037] In one embodiment, Figure 5 , which is a schematic diagram of two first front-view images provided according to another embodiment of the present invention, wherein: Figure 5The four points in the first front view image correspond to preset calibration points, and the shaded part corresponds to the display range of the working device in the first front view image. After mirroring and mapping the coordinate points in one of the first front view images to another first front view image to obtain a fourth front view image, the contour formed by the coincident points of the two groups of coordinate points in the fourth front view image is taken as the outer contour, and before this, the following steps are further included: obtaining the display range of the working device in the first front view image; and removing the coordinate points outside the display range and on the boundary of the display range to obtain the coordinate points within the display range. During the operation of the working device, the display range will not be exceeded. The display range is preset according to the size and operation trajectory of the working device of the working machine. Removing the coordinate points outside the display range and on the boundary of the display range can exclude the interference coordinate points caused by environmental factors, and ensure that the subsequent generated outer contour is more accurate. This step helps to reduce errors in the image processing process and improve the accuracy and clarity of the finally obtained unobstructed front view image. In addition, the above method can also improve the calculation speed of the processor and simplify the calculation process.

[0038] In one embodiment, the step of performing image processing on the first front view image to obtain two second front view images with the outer contour removed includes: performing dilation processing on the outer contour to obtain a dilated outer contour after the dilation processing; mapping the dilated outer contour to the two first front view images respectively; and removing the dilated outer contour in the two first front view images respectively to obtain the two second front view images. Dilation processing is an image processing technique that expands the edges of the outer contour to ensure that all possible obstructed coordinate points are included. The purpose of this step is to ensure that when the outer contour is removed, the working device and its surrounding possible obstructed parts can be completely removed, avoiding the existence of a visual blind area in the spliced and fitted image. The dilated outer contour after the dilation processing is mapped back to the two first front view images respectively, and then the processor removes the dilated outer contour in the two first front view images respectively, thereby obtaining the two second front view images with the outer contour removed. In the two second front view images, the environmental image in front of the working machine is completely presented without obstruction of the working device, providing a clear and comprehensive front view for the driver. Finally, the processor splices and fits the two second front view images, seamlessly combines the two images through an image splicing algorithm, and obtains an unobstructed front view image of the working machine. This unobstructed front view image not only improves the operation safety of the working machine, but also optimizes the driving experience of the driver, enabling them to focus more on the work process and improve work efficiency.

[0039] In one embodiment, a front view image acquisition system of a working machine is provided, including: a memory configured to store instructions; and a processor configured to call the instructions from the memory and capable of implementing the above-mentioned front view image acquisition method of the working machine when executing the instructions.

[0040] In one embodiment, a work machine is provided, comprising: the front view image acquisition system of the work machine as described above; a vehicle body; a work device provided at a front end of the vehicle body and comprising an arm frame and an implement provided on the arm frame; and two image acquisition units provided on the vehicle body and located at two sides of the work device, the two image acquisition units each being configured to acquire a front view image of the work machine.

[0041] In one embodiment, a machine readable storage medium having stored thereon instructions for causing a machine to perform the front view image acquisition method of the work machine as described above is provided.

[0042] In one embodiment, a computer device, which can be a server, is provided, and an internal structure diagram of the computer device can be as shown in Figure 6 The computer device comprises a processor, a network interface, a memory (not shown in the figure) and a database (not shown in the figure) connected through a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises an internal memory and a non-volatile storage medium. The non-volatile storage medium stores an operating system, a computer program and a database (not shown in the figure). The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The network interface of the computer device is configured to communicate with an external terminal through a network connection. The computer program, when executed by the processor, can implement the front view image acquisition method of the work machine.

[0043] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0044] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one or more flows and / or blocks in the flowcharts and / or block diagrams can be implemented as an apparatus. Figure 1 The functions specified in one or more flows and / or blocks in the flowcharts and / or block diagrams can be implemented as an apparatus.

[0045] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the Figure 1 function specified in the flow or flows and / or blocks Figure 1 of the block or blocks.

[0046] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions that are executed on the computer or other programmable apparatus provide steps for implementing the Figure 1 function specified in the flow or flows and / or blocks Figure 1 of the block or blocks.

[0047] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0048] The memory can include non-persistent memory and / or volatile memory, such as random access memory (RAM) and / or cache memory. The memory can also include non-volatile memory, such as read only memory (ROM), EPROM, and / or flash memory. The memory can be a memory of a computer readable medium.

[0049] Computer readable media includes permanent and non-permanent, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital versatile disc (DVD), or other optical storage, magnetic cassettes, magnetic disks storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer readable media does not include transitory media, such as modulated data signals and carrier waves.

[0050] It is also to be noted that the terms "comprising", "including", and any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises a... " does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0051] The above embodiments are only used to illustrate the present application, but not to limit it. Instead of the above, various modifications and changes can be made to the application by those skilled in the art. Any modification, equivalent replacement, improvement, and the like made within the spirit and principle of the application shall fall into the scope of claims of the application.

Claims

1. A method for collecting front view images of an operating machine, characterized in that: The working machine includes a vehicle body, a working device provided at the front end of the vehicle body, and two image acquisition components provided on the vehicle body and located on both sides of the working device. The two image acquisition components are both used to acquire a front view image of the working machine. The front view image acquisition method includes: Controlling the two image acquisition components to respectively acquire front view images of the operating machine and obtain two first front view images; Calibrate the positions of the two image acquisition components so that the preset calibration points of the working device in the two first front-view images are distributed in a mirror-symmetrical manner; Acquire two sets of coordinate points of the working device in the two first front-view images; generating an outer contour corresponding to the working device according to the two groups of coordinate points; performing image processing on the two first front-view images to obtain two second front-view images with the outer contour removed; The two second front view images are fitted and spliced ​​to obtain an unobstructed front view image of the working machine.

2. The front view image acquisition method of a working machine according to claim 1, characterized in that: The step of calibrating the positions of the two image acquisition components so that the preset calibration points of the working device in the two first front-view images are distributed in a mirror-symmetrical manner comprises: Adjusting the positions of the two image acquisition components so that the two image acquisition components are at the same height and are symmetrically arranged relative to the working device; adjusting the acquisition directions of the two image acquisition components according to the first front view image to obtain two third front view images, wherein the vertical coordinate axis of the first front view image is perpendicular to the horizontal direction; The two third front-view images are calibrated by software so that the two groups of preset calibration points are distributed in a mirror-symmetrical manner.

3. The front view image acquisition method of a working machine according to claim 2, characterized in that: The step of performing software calibration on the two third front-view images so that the two sets of preset calibration points are distributed in a mirror-symmetrical manner comprises: respectively obtaining preset calibration points of the working device in the two third front-view images; Mirror mapping the preset calibration points in one of the third front-view images to the other third front-view image to obtain a position difference between the two sets of preset calibration points; According to the position difference, the acquisition direction of the image acquisition component is adjusted until the position difference is zero.

4. The front view image acquisition method of a working machine according to claim 1, characterized in that: The step of generating the outer contour corresponding to the working device according to the two groups of coordinate points includes: Mirror mapping the coordinate point in one of the first front-view images to another of the first front-view images to obtain a fourth front-view image; The contour formed by the coincident points of the two groups of coordinate points in the fourth front view image is used as the outer contour.

5. The front view image acquisition method of a working machine according to claim 4, characterized in that: The forward-looking image acquisition method further comprises the following steps: Obtaining the contour attributes corresponding to the working device; The outer contour is subjected to denoising according to the contour attributes to obtain an accurate contour after denoising.

6. The front view image acquisition method of a working machine according to claim 4, characterized in that: After mirroring the coordinate points in one of the first front-view images to another of the first front-view images to obtain a fourth front-view image, and before using the contour formed by the overlapping points of the two sets of coordinate points in the fourth front-view image as the outer contour, the method further includes the following steps: obtaining a display range of the working device in the first front view image; The coordinate points outside the display range and on the boundary of the display range are removed to obtain the coordinate points within the display range.

7. The front view image acquisition method for a working machine according to any one of claims 1 to 6, characterized in that: The step of performing image processing on the first front view image to obtain two second front view images without the outer contour comprises: Performing expansion processing on the outer contour to obtain an expanded outer contour after the expansion processing; mapping the expanded outer contour into the two first front-view images respectively; The expanded outer contours in the two first front-view images are removed respectively to obtain two second front-view images.

8. A front view image acquisition system for an operating machine, characterized in that: include: a memory configured to store instructions; as well as The processor is configured to call the instructions from the memory and implement the forward-view image acquisition method for a working machine according to any one of claims 1 to 7 when executing the instructions.

9. A working machine, characterized in that: include: The forward-looking image acquisition system for a working machine according to claim 8; body; A working device is provided at the front end of the vehicle body and includes an arm and accessories provided on the arm; Two image acquisition components are provided on the vehicle body and located on both sides of the working device. Both of the image acquisition components are used to acquire the front view image of the working machine.

10. A machine-readable storage medium, characterized in that The machine-readable storage medium stores instructions for enabling a machine to execute the forward-view image acquisition method for a working machine according to any one of claims 1 to 7.

Images