Binocular imaging method, binocular imaging device, electronic equipment and storage medium

Abstract

The invention provides a binocular imaging method, a binocular imaging device, electronic equipment and a storage medium. The method obtains a first image and a second image. Determining a first tray hole region identifier corresponding to a first tray hole of the target tray in the first image and a second tray hole region identifier corresponding to a second tray hole of the target tray, and determining a third tray hole region identifier corresponding to the first tray hole and a fourth tray hole region identifier corresponding to the second tray hole in the second image; and controlling a first lens of the intelligent glasses to at least display the first image, the first tray hole area identifier and the second tray hole area identifier, and controlling a second lens of the intelligent glasses to at least display the second image, the third tray hole area identifier and the fourth tray hole area identifier. The method is beneficial to improving the operation efficiency.

Description

Technical Field

[0001] This invention relates to the field of industrial image processing technology, and more specifically to a binocular imaging method, a binocular imaging device, an electronic device, a storage medium, and a computer program product. Background Technology

[0002] In stacker truck operations, the forks of the stacker truck are typically used to transport pallets and the objects on them. The bottom of the pallet usually has two pallet holes to accommodate two forks. Users need to rely on their experience to control the stacker truck's movement so that the two forks are positioned into the two pallet holes. This makes the operation relatively difficult for users.

[0003] In the above example, the user needs to consider the relative position of the stacker truck and the pallet to avoid collisions between other areas of the pallet and the forks. Collisions can easily damage the forks, pallet, and goods on the pallet. If the forks are not aligned with the pallet holes (or if the extension line of the forks away from the stacker truck does not intersect the pallet hole space), the stacker truck may need to be moved and adjusted multiple times, which is detrimental to improving its operational efficiency. Summary of the Invention

[0004] The present invention was proposed in view of the above-mentioned problems. The present invention provides a binocular imaging method, a binocular imaging device, an electronic device, a storage medium, and a computer program product.

[0005] According to one aspect of the present invention, a binocular imaging method is provided. The method includes: acquiring a first image and a second image, wherein the first image is acquired through a first lens in a binocular camera, and the second image is acquired through a second lens in a binocular camera; the first image includes a first fork of a stacker truck, and the second image includes a second fork of a stacker truck; determining a first pallet hole region identifier corresponding to a first pallet hole of a target pallet and a second pallet hole region identifier corresponding to a second pallet hole of the target pallet in the first image, and determining a third pallet hole region identifier corresponding to a first pallet hole and a fourth pallet hole region identifier corresponding to a second pallet hole in the second image, wherein the first pallet hole is used to accommodate the first fork, and the second pallet hole is used to accommodate the second fork; controlling a first lens of smart glasses to display at least the first image, the first pallet hole region identifier, and the second pallet hole region identifier, and controlling a second lens of smart glasses to display at least the second image, the third pallet hole region identifier, and the fourth pallet hole region identifier.

[0006] For example, the first lens controlling the smart glasses displays at least a first image, a first tray hole area identifier, and a second tray hole area identifier, including:

[0007] The first lens is controlled to display at least a first image, a first pallet hole area identifier, a second pallet hole area identifier, and a first fork area identifier, wherein the first fork area identifier is used to mark the area corresponding to the first fork in the first image;

[0008] The second lens controlling the smart glasses displays at least a second image, a third tray hole area identifier, and a fourth tray hole area identifier, including:

[0009] The second lens is controlled to display at least the second image, the third pallet hole area identifier, the fourth pallet hole area identifier, and the second fork area identifier, wherein the second fork area identifier is used to mark the area corresponding to the second fork in the second image.

[0010] For example, controlling the first lens to display at least a first image, a first pallet hole area identifier, a second pallet hole area identifier, and a first fork area identifier includes:

[0011] The first lens is controlled to display at least a first image, a first pallet hole area identifier, a second pallet hole area identifier, a first fork area identifier, and a first extension line, wherein the first extension line is used to indicate the extension line of the first fork away from the stacker truck.

[0012] Controlling the second lens to display at least the second image, the third pallet hole area marking, the fourth pallet hole area marking, and the second fork area marking, including:

[0013] The second lens is controlled to display at least a second image, a third pallet hole area identifier, a fourth pallet hole area identifier, a second fork area identifier, and a second extension line, wherein the second extension line is used to indicate the extension line of the second fork away from the side of the stacker truck.

[0014] For example, the above method further includes:

[0015] When the first extended line intersects with the first tray hole area mark and the second extended line intersects with the fourth tray hole area mark, the first color displayed by the first extended line in the first lens and the second color displayed by the second extended line in the second lens are both adjusted to the target color, wherein the first color and the second color are different from the target color.

[0016] For example, the above method further includes:

[0017] Based on the region size of at least one of the first tray hole region identifier, the second tray hole region identifier, the third tray hole region identifier, and the fourth tray hole region identifier, a first transparency of the first image displayed in the first lens and a second transparency of the second image displayed in the second lens are determined, wherein the transparency of both the first image and the second image is positively correlated with the region size.

[0018] For example, the above method further includes:

[0019] Based on the target travel distance of the stacker truck, the target focal length is adjusted. The target travel distance is the travel distance of the stacker truck from the target time when the first pallet hole area marker and the second pallet hole area marker are determined to the current time. The target focal length is the focal length used by the first lens and the second lens.

[0020] For example, the above method also includes any one of the following:

[0021] Based on the area dimensions of the first pallet hole area marker and the fourth pallet hole area marker, a first orientation indicator is determined. When the area dimension of the first pallet hole area marker is larger than that of the fourth pallet hole area marker, the first orientation indicator is used to indicate that the traveling direction of the stacker truck is toward the second pallet hole and away from the first pallet hole. When the area dimension of the first pallet hole area marker is smaller than that of the fourth pallet hole area marker, the first orientation indicator is used to indicate that the traveling direction is toward the first pallet hole and away from the second pallet hole.

[0022] A second orientation indicator is determined based on the area dimensions of both the second tray hole area identifier and the third tray hole area identifier. When the area dimension of the second tray hole area identifier is greater than that of the third tray hole area identifier, the second orientation indicator is used to indicate the direction of travel toward the second tray hole and away from the first tray hole. When the area dimension of the second tray hole area identifier is less than that of the third tray hole area identifier, the second orientation indicator is used to indicate the direction of travel toward the first tray hole and away from the second tray hole.

[0023] For example, the above method includes:

[0024] Based on the area dimensions of the first tray hole area identifier and the fourth tray hole area identifier, a first orientation indicator is determined, and based on the area dimensions of the second tray hole area identifier and the third tray hole area identifier, a second orientation indicator is determined.

[0025] For example, the above method further includes:

[0026] Based on the area dimensions of the first tray hole area identifier and the fourth tray hole area identifier, a first weight corresponding to the first orientation indicator is determined, wherein the difference between the area dimensions of the first tray hole area identifier and the fourth tray hole area identifier is positively correlated with the first weight.

[0027] Based on the area dimensions of the second tray hole area identifier and the third tray hole area identifier, a second weight corresponding to the second orientation indicator is determined, wherein the difference between the area dimensions of the second tray hole area identifier and the third tray hole area identifier is positively correlated with the second weight;

[0028] Based on the first orientation indicator, the second orientation indicator, the first weight, and the second weight, a final orientation indicator is determined, wherein the final orientation indicator is displayed in the first lens and / or the second lens.

[0029] According to another aspect of the present invention, a binocular imaging device is also provided, the binocular imaging device comprising:

[0030] The image acquisition module is used to acquire a first image and a second image, wherein the first image is acquired through a first lens in a binocular camera, and the second image is acquired through a second lens in a binocular camera. The first image includes the first fork of the stacker truck, and the second image includes the second fork of the stacker truck.

[0031] The pallet hole area identification determination module is used to determine the first pallet hole area identification corresponding to the first pallet hole of the target pallet in the first image and the second pallet hole area identification corresponding to the second pallet hole of the target pallet, and to determine the third pallet hole area identification corresponding to the first pallet hole and the fourth pallet hole area identification corresponding to the second pallet hole in the second image, wherein the first pallet hole is used to accommodate the first fork and the second pallet hole is used to accommodate the second fork.

[0032] The image display module is used to control the first lens of the smart glasses to display at least a first image, a first tray hole area identifier, and a second tray hole area identifier, and to control the second lens of the smart glasses to display at least a second image, a third tray hole area identifier, and a fourth tray hole area identifier.

[0033] According to another aspect of the present invention, an electronic device is also provided, comprising a memory and a processor, wherein: the memory is used to store a computer program; and the processor is used to execute the computer program to implement the above-described binocular imaging method.

[0034] According to another aspect of the present invention, a storage medium is also provided, which stores computer program instructions that, when executed, are used to perform the above-described binocular imaging method.

[0035] According to another aspect of the present invention, a computer program product is also provided, comprising computer program instructions which, when executed by a processor, are used to perform the above-described binocular imaging method.

[0036] According to the above-described scheme of the embodiments of the present invention, a first image and a second image can be acquired. Then, the first pallet hole area identifier corresponding to the first pallet hole of the target pallet and the second pallet hole area identifier corresponding to the second pallet hole of the target pallet are determined in the first image, and the third pallet hole area identifier corresponding to the first pallet hole and the fourth pallet hole area identifier corresponding to the second pallet hole are determined in the second image. Finally, the first lens of the smart glasses is controlled to display at least the first image, the first pallet hole area identifier, and the second pallet hole area identifier, and the second lens of the smart glasses is controlled to display at least the second image, the third pallet hole area identifier, and the fourth pallet hole area identifier. Considering the actual structure of the stacker truck, the user's viewing angle is high, making it difficult for the user to directly observe the pallet holes. The user needs to move the stacker truck based on operating experience. Therefore, the operation of the stacker truck is greatly affected by the user's operating experience. However, in the above-described scheme of the embodiments of the present invention, the first pallet hole area identifier, the third pallet hole area identifier, the second pallet hole area identifier, and the fourth pallet hole area identifier of the second pallet hole can be effectively displayed through the first and second images, thereby increasing the salience of the first and second pallet holes in the first and second images, which is beneficial for assisting the user in controlling the movement of the stacker truck. Furthermore, allowing users to view the first and second images through the first and second lenses respectively helps them understand the positional relationship between the target pallet and the stacker truck in three-dimensional space, thus assisting them in controlling the movement of the stacker truck. Since the above solution can intuitively and clearly display the first and second pallet holes in both the first and second images, it reduces the need for user experience during operations, thereby improving operational efficiency. Attached Figure Description

[0037] The above and other objects, features, and advantages of the present invention will become more apparent from the more detailed description of the embodiments of the invention in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same parts or steps.

[0038] Figure 1 A schematic flowchart of a binocular imaging method according to an embodiment of the present invention is shown;

[0039] Figure 2 A schematic diagram showing the contents displayed by a first lens and a second lens according to an embodiment of the present invention is shown;

[0040] Figure 3 A schematic block diagram of a binocular imaging device according to an embodiment of the present invention is shown;

[0041] Figure 4A schematic block diagram of an electronic device according to an embodiment of the present invention is shown. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of the present invention more apparent, exemplary embodiments according to the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely a part of the embodiments of the present invention, and not all of the embodiments of the present invention. It should be understood that the present invention is not limited to the exemplary embodiments described herein. Based on the embodiments of the present invention described herein, all other embodiments obtained by those skilled in the art without inventive effort should fall within the protection scope of the present invention.

[0043] In existing technologies, inserting the forks into the pallet holes relies heavily on the user's operational experience.

[0044] To at least partially solve the above problems, embodiments of the present invention provide a binocular imaging method. Figure 1 A schematic flowchart of a binocular imaging method according to an embodiment of the present invention is shown. Figure 1 As shown, the method may include the following steps S110 to S130.

[0045] In step S110, the first image and the second image are acquired.

[0046] The first image is captured through a first lens of a binocular camera, and the second image is captured through a second lens of the binocular camera. The binocular camera can be mounted on the mast of the stacker truck. The height of the binocular camera can be below the eye level of the user when seated in the stacker truck. In one example, the binocular camera can be located at or near the horizontal plane of the first and second forks of the stacker truck, such that the first and second images from the binocular camera can include the first and second pallet holes of the target pallet. The target pallet can be a pallet to be moved by the user.

[0047] The first pallet hole can be used to accommodate the first fork, and the second pallet hole can be used to accommodate the second fork.

[0048] The first fork can be the fork located on the left side of the user's body when the user is seated in the stacker truck (hereinafter referred to as the left side). Therefore, the first pallet hole can be the left-side pallet hole, and the first lens can be the left-side lens of the binocular camera. The second fork can be the fork located on the right side of the user's body when the user is seated in the stacker truck (hereinafter referred to as the right side). Therefore, the second pallet hole can be the right-side pallet hole, and the second lens can be the right-side lens of the binocular camera. It can be understood that the first fork can also be the right-side fork, the first pallet hole can be the right-side pallet hole, and the first lens can be the right-side lens of the binocular camera. The second fork can be the left-side fork, the second pallet hole can be the left-side pallet hole, and the second lens can be the left-side lens of the binocular camera. The specific configuration depends on the actual needs of the developers, and this embodiment of the invention does not impose any limitations.

[0049] The first image may include at least the first fork, and the second image may include at least the second fork. In one example, both the first and second images may include both the first and second forks, depending on the shooting angle or placement of the binocular camera.

[0050] In step S120, the first tray hole region identifier corresponding to the first tray hole of the target tray in the first image and the second tray hole region identifier corresponding to the second tray hole of the target tray are determined, and the third tray hole region identifier corresponding to the first tray hole and the fourth tray hole region identifier corresponding to the second tray hole in the second image are determined.

[0051] A first tray hole area identifier can be used to mark the area where the first tray hole is located in the first image. A second tray hole area identifier can be used to mark the area where the second tray hole is located in the first image. A third tray hole area identifier can be used to mark the area where the first tray hole is located in the second image. A fourth tray hole area identifier can be used to mark the area where the second tray hole is located in the second image.

[0052] In one example, an image recognition algorithm can be used to determine the first tray hole region and the second tray hole region in the first image, and the third tray hole region and the fourth tray hole region in the second image. For example, a tray hole reference image can be used to match the first image and the second image respectively, so that the matched region is taken as the target tray hole region. The tray hole reference image includes the image corresponding to the first tray hole or the second tray hole. Specifically, for example, the first image may include two target tray hole regions that match the tray hole reference image. The first tray hole and the second tray hole in the target tray holes can be determined based on the positional relationship between the target tray hole regions. If the first tray hole is the left tray hole mentioned above, then the left-leaning target tray hole region can be taken as the first tray hole region, and the right-leaning target tray hole region can be taken as the second tray hole region. Specifically, for example, the second image may include two target tray hole regions that match the tray hole reference image. The first tray hole and the second tray hole in the target tray holes can be determined based on the positional relationship between the target tray hole regions. If the first tray hole is the left tray hole mentioned above, then the left-leaning target tray hole region can be taken as the third tray hole region, and the right-leaning target tray hole region can be taken as the fourth tray hole region.

[0053] In another example, a trained tray hole detection model can be used to determine the first and second tray hole regions in the first image, and the third and fourth tray hole regions in the second image. The tray hole detection model can be a machine learning model, and the specific model architecture is not limited in this embodiment. This embodiment provides a training process for a tray hole detection model for reference. A first sample image can be input into the tray hole detection model to be trained to obtain the predicted tray hole regions (represented by a binary image or a set of coordinate values). The pre-labeled actual tray hole regions (represented by a binary image or a set of coordinate values) in the first sample image can be compared with the predicted tray hole regions, and the difference between the two can be used as part of the loss value. The training objective of the trained tray hole detection model may include reducing this loss value. Specifically, the trained tray hole detection model can adjust its model parameters based on this loss value to reduce the difference between the predicted tray hole regions and the actual tray hole regions. By iteratively training multiple first sample images, each corresponding to the actual tray hole region, a trained tray hole detection model can be obtained, whose output tray hole prediction region closely approximates the actual tray hole region. It is understood that the tray hole detection model can also be adjusted based on other loss values, depending on the developer's actual needs. After determining the tray hole prediction region, the method used in the image recognition algorithm examples can also be used to determine the tray hole prediction region as one of the first, second, third, and fourth tray hole regions; this will not be elaborated upon in this embodiment.

[0054] In step S130, the first lens of the smart glasses is controlled to display at least the first image, the first tray hole area identifier, and the second tray hole area identifier, and the second lens of the smart glasses is controlled to display at least the second image, the third tray hole area identifier, and the fourth tray hole area identifier.

[0055] The smart glasses may include at least a first lens with image display function, a second lens with image display function, and a frame for supporting the first and second lenses. The binocular imaging method of this invention can be applied to a processing terminal, which can be connected to a binocular camera to receive a first image and a second image. After determining the first tray hole area identifier and the second tray hole area identifier, the processing terminal can superimpose them onto the first image. After determining the third tray hole area identifier and the fourth tray hole area identifier, it can superimpose them onto the second image. In one example, the processing terminal can send the superimposed first image to the first lens and the superimposed second image to the second lens. In this example, the control module of the smart glasses may be the aforementioned processing terminal, which may be located within the smart glasses or connected to the smart glasses via wired or wireless connection. In another example, the processing terminal may also send the superimposed first image and the superimposed second image to the control module of the smart glasses, so that the control module controls the first lens to display the superimposed first image and controls the second lens to display the superimposed second image. This invention does not limit the specific hardware structure of the smart glasses and can be adjusted according to the actual needs of developers or users.

[0056] In one example, taking the first lens as the left-hand lens, the first lens element could be the left-hand lens in the smart glasses when the user wears them. The second lens element could be the right-hand lens in the smart glasses when the user wears them. In another example, taking the first lens as the right-hand lens, the first lens element could be the right-hand lens in the smart glasses when the user wears them. The second lens element could be the left-hand lens in the smart glasses when the user wears them.

[0057] Taking the left-hand lens as the first lens as an example, and considering a real-world scenario, when a user wears smart glasses, the left-hand first lens can display the first image captured by the left-hand first lens, the first tray hole area marker, and the second tray hole area marker. The right-hand second lens can display the second image captured by the right-hand second lens, the third tray hole area marker, and the fourth tray hole area marker.

[0058] Because the first and second lenses shoot from different angles, the first image is displayed on the first lens and the second image is displayed on the second lens. This creates a strong sense of stereoscopic vision for users wearing smart glasses, making it easier for them to understand the positional relationship between the pallet hole and the fork in three-dimensional space.

[0059] According to the above-described scheme of the embodiments of the present invention, a first image and a second image can be acquired. Then, the first pallet hole area identifier corresponding to the first pallet hole of the target pallet and the second pallet hole area identifier corresponding to the second pallet hole of the target pallet are determined in the first image, and the third pallet hole area identifier corresponding to the first pallet hole and the fourth pallet hole area identifier corresponding to the second pallet hole are determined in the second image. Finally, the first lens of the smart glasses is controlled to display at least the first image, the first pallet hole area identifier, and the second pallet hole area identifier, and the second lens of the smart glasses is controlled to display at least the second image, the third pallet hole area identifier, and the fourth pallet hole area identifier. Considering the actual structure of the stacker truck, the user's viewing angle is high, making it difficult for the user to directly observe the pallet holes. The user needs to move the stacker truck based on operating experience. Therefore, the operation of the stacker truck is greatly affected by the user's operating experience. However, in the above-described scheme of the embodiments of the present invention, the first pallet hole area identifier, the third pallet hole area identifier, the second pallet hole area identifier, and the fourth pallet hole area identifier of the second pallet hole can be effectively displayed through the first and second images, thereby increasing the salience of the first and second pallet holes in the first and second images, which is beneficial for assisting the user in controlling the movement of the stacker truck. Furthermore, allowing users to view the first and second images through the first and second lenses respectively helps them understand the positional relationship between the target pallet and the stacker truck in three-dimensional space, thus assisting them in controlling the movement of the stacker truck. Since the above solution can intuitively and clearly display the first and second pallet holes in both the first and second images, it reduces the need for user experience during operations, thereby improving operational efficiency.

[0060] For example, in step S130, controlling the first lens of the smart glasses to display at least the first image, the first tray hole area identifier, and the second tray hole area identifier may include step S131a.

[0061] In step S131a, the first lens is controlled to display at least the first image, the first pallet hole area identifier, the second pallet hole area identifier, and the first fork area identifier.

[0062] The first fork area marker can be used to mark the area (or first fork region) corresponding to the first fork in the first image. Users can refer to the first fork area marker, the first pallet hole area marker, and the third pallet hole area marker to move the stacker truck. Specifically, the first pallet hole area marker and the third pallet hole area marker construct a three-dimensional visual result of the first pallet hole in the user's eyes. The first fork area marker can be considered as the area where the first fork is located; therefore, if the first fork area marker intersects with the three-dimensional visual result of the first pallet hole, it can be considered that the first fork has been fully or partially placed into the first pallet hole in physical space, thus assisting the user in moving the stacker truck.

[0063] In step S130, controlling the second lens of the smart glasses to display at least the second image, the third tray hole area identifier, and the fourth tray hole area identifier may include step S131b.

[0064] In step S131b, the second lens is controlled to display at least the second image, the third pallet hole area identifier, the fourth pallet hole area identifier, and the second fork area identifier.

[0065] The second fork area marker can be used to mark the area (or second fork region) corresponding to the second fork in the second image. Users can refer to the second fork area marker, the second pallet hole area marker, and the fourth pallet hole area marker to move the stacker truck. Specifically, the second pallet hole area marker and the fourth pallet hole area marker construct a three-dimensional visual result of the second pallet hole in the user's eyes. The second fork area marker can be considered as the area where the second fork is located; therefore, if the second fork area marker intersects with the three-dimensional visual result of the second pallet hole, it can be considered that the second fork has been fully or partially placed into the second pallet hole in physical space, thus assisting the user in moving the stacker truck.

[0066] In one example, an image recognition algorithm can be used to determine the first fork in the first image and the second fork in the second image. For example, a reference image of the first or second fork can be used to match the first and second images respectively, so that the matched area in the first image is taken as the first fork area and the matched area in the second image is taken as the second fork area. It is understood that due to the position of the binocular camera, both the first and second forks may exist in the first and second images. In other words, the first image may contain an area that matches the first fork and an area that matches the second fork. If the first fork is the left fork and the first image is taken by the left-hand lens, then the area slightly to the left of these two areas can be taken as the aforementioned first fork area. In this example, the second image may also contain an area that matches the first fork and an area that matches the second fork. The area slightly to the right of these two areas can be taken as the aforementioned second fork area. If the first fork is the right fork and the first image is taken by the right-hand lens, adjustments can be made to the above example, and the embodiments of the present invention will not be elaborated here.

[0067] In another example, a first forklift region in a first image and a second forklift region in a second image can be determined using a trained forklift detection model. The forklift detection model can be a machine learning model, and the specific model architecture is not limited in this embodiment. This embodiment provides a training process for a forklift detection model for reference. A second sample image can be input into the forklift detection model to be trained to obtain a predicted forklift region (represented by a binary image or a set of coordinate values). The pre-labeled actual forklift region in the second sample image (represented by a binary image or a set of coordinate values) can be compared with the predicted forklift region, and the difference between the two can be used as part of the loss value. The training objective of the forklift detection model to be trained may include reducing this loss value. Specifically, the forklift detection model to be trained can adjust its model parameters based on this loss value to reduce the difference between the predicted forklift region and the actual forklift region. By iteratively training using multiple second sample images and their corresponding actual forklift regions, a trained forklift detection model can be obtained, where the output predicted forklift region is close to the actual forklift region. It is understood that the fork detection model can also be adjusted based on other loss values, depending on the actual needs of the developers. After determining the fork prediction region, the method in the image recognition algorithm example can also be used to determine the fork prediction region as one of the first fork region and the second fork region, which will not be elaborated here in the embodiments of the present invention.

[0068] In another example, if the position of the binocular camera remains unchanged, the relative positions between the binocular camera and the first and second forks generally also remain unchanged. Therefore, the first fork region identifier in the first image can be determined using the image recognition algorithm and fork detection model described above, and used as the preset fork region identifier. In other words, the position of the first fork region identifier can be the same in different first images. Similarly, the position of the second fork region identifier can also be the same in different second images. This avoids the need to determine the first fork region identifier for each first image and the second fork region identifier for each second image, improving the real-time performance of the binocular imaging method and reducing computational requirements. It is also understandable that the determination of the first and second fork region identifiers can be performed only once after each time the stacker truck is opened, to avoid movement of the binocular camera or shift in the shooting angle due to external forces when the stacker truck is stationary.

[0069] Based on the first image, the first pallet hole area identifier, the second pallet hole area identifier, the first fork area identifier, the second image, the third pallet hole area identifier, the fourth pallet hole area identifier, and the second fork area identifier, the user can be assisted in placing the first fork into the first pallet hole and the second fork into the second pallet hole.

[0070] According to the above-described solution of the present invention, the first lens can be controlled to display at least a first image, a first pallet hole area identifier, a second pallet hole area identifier, and a first fork area identifier, and the second lens can be controlled to display at least a second image, a third pallet hole area identifier, a fourth pallet hole area identifier, and a second fork area identifier. In the above solution, the first fork area identifier and the second fork area identifier can be displayed to show the position of the first fork in the first lens and the position of the second fork in the second lens. By combining the first pallet hole area identifier, the second pallet hole area identifier, the third pallet hole area identifier, and the fourth pallet hole area identifier, the user can accurately place the first fork into the first pallet hole and the second fork into the second pallet hole. Specifically, the user can obtain real-time feedback on the relative positions of the first fork and the first pallet hole, and the relative positions of the second fork and the second pallet hole after the mobile stacker truck has moved, thereby reducing the need for user experience during operation and improving work efficiency.

[0071] For example, step S131a controls the first lens to display at least the first image, the first pallet hole area identifier, the second pallet hole area identifier, and the first fork area identifier, including: controlling the first lens to display at least the first image, the first pallet hole area identifier, the second pallet hole area identifier, the first fork area identifier, and the first extension line.

[0072] The first extension line can be used to represent the extension line of the first fork away from the stacker truck. The first extension line can extend from the first fork area marker in the direction indicated by the first fork area marker, away from the stacker truck. For example, it can be the extension line of the centerline of the first fork area marker that meets the above conditions.

[0073] Users can refer to the first fork area marker, the first extension line, the first pallet hole area marker, and the third pallet hole area marker to move the stacker truck. Specifically, the first pallet hole area marker and the third pallet hole area marker construct a three-dimensional visual result of the first pallet hole in the user's eyes. If the first extension line intersects with this three-dimensional visual result, it can be considered that moving the stacker truck along the direction of the extension line will allow the first fork to be placed into the first pallet hole. The first fork area marker can be regarded as the area where the first fork is located. Therefore, if the first fork area marker intersects with the three-dimensional visual result of the first pallet hole, it can be considered that the first fork has been fully or partially placed into the first pallet hole in physical space, thereby assisting the user in moving the stacker truck.

[0074] Step S131b, controlling the second lens to display at least the second image, the third pallet hole area identifier, the fourth pallet hole area identifier, and the second fork area identifier, may include: controlling the second lens to display at least the second image, the third pallet hole area identifier, the fourth pallet hole area identifier, the second fork area identifier, and the second extension line.

[0075] The second extension line is used to indicate the extension line of the second fork away from the stacker truck. The second extension line may extend from the second fork area marker in the direction indicated by the second fork area marker, away from the stacker truck. For example, it may be the extension line of the centerline of the second fork area marker that meets the above conditions.

[0076] Users can refer to the second fork area marker, the second extension line, the second pallet hole area marker, and the fourth pallet hole area marker to move the stacker truck. Specifically, the second and fourth pallet hole area markers construct a three-dimensional visual result of the second pallet hole in the user's eyes. If the second extension line intersects with this three-dimensional visual result, it can be considered that moving the stacker truck along the direction of the extension line will allow the second fork to be placed into the second pallet hole. The second fork area marker can be regarded as the area where the second fork is located. Therefore, if the second fork area marker intersects with the three-dimensional visual result of the second pallet hole, it can be considered that the second fork has been fully or partially placed into the second pallet hole in physical space, thus assisting the user in moving the stacker truck.

[0077] Based on the first and second extension lines, users can comprehensively consider the travel direction of the stacker truck, thereby assisting them in moving the stacker truck. (See also...) Figure 2 , Figure 2A schematic diagram illustrating the contents displayed by a first lens and a second lens according to an embodiment of the present invention is shown. The contents displayed by the first lens may include a first image, on which a first fork area identifier, a first extension line, a first pallet hole area identifier, and a second pallet hole area identifier are displayed. The contents displayed by the second lens may include a second image, on which a second fork area identifier, a second extension line, a third pallet hole area identifier, and a fourth pallet hole area identifier are displayed. It is understood that the first image may also display a third fork area identifier (used to identify the position of the second fork in the first image) and a third extension line (used to indicate the extension line of the second fork in the first image). The second image may also display a fourth fork area identifier (used to identify the position of the first fork in the second image) and a fourth extension line (used to indicate the extension line of the first fork in the second image).

[0078] According to the above-described solution of the present invention, the first lens can be controlled to display at least a first image, a first pallet hole area marker, a second pallet hole area marker, a first fork area marker, and a first extension line, and the second lens can be controlled to display at least a second image, a third pallet hole area marker, a fourth pallet hole area marker, a second fork area marker, and a second extension line. The above solution, by displaying the first and second extension lines, assists the user in controlling the movement of the stacker truck by combining the first to fourth pallet hole area markers, the first fork area marker, and the second fork area marker. Specifically, the user can obtain real-time feedback on the relative positions between the first fork and the first pallet hole, and between the second fork and the second pallet hole after moving the stacker truck through the first and second lenses, thereby reducing the need for user experience during operation and improving operational efficiency.

[0079] For example, when the first extension line intersects with the first tray hole area mark and the second extension line intersects with the fourth tray hole area mark, the first color displayed by the first extension line in the first lens and the second color displayed by the second extension line in the second lens are both adjusted to the target color.

[0080] Both the first and second colors are different from the target color. The first color may be the same as or different from the second color. For example, the first and second colors may both be red, and the target color may be green. If the user sees red first and second extension lines through the first and second lenses, it can be assumed that the forks of the stacker truck are not aligned with the pallet holes, and the steering wheel rotation angle of the stacker truck needs to be adjusted. If the user sees green first and second extension lines, it can be assumed that the forks of the stacker truck are aligned with the pallet holes, and the user can reset the existing steering wheel angle to zero to control the stacker truck to move straight to the pallet, thus placing the first fork into the first pallet hole and the second fork into the second pallet hole.

[0081] The specific colors of the first color, the second color, and the target color are not limited in this embodiment of the invention and can be set by the developers or users.

[0082] When the first extended line intersects the first pallet hole area marker, and the second extended line intersects the fourth pallet hole area marker, it can be considered that the user has aligned the first fork with the first pallet hole and the second fork with the second pallet hole. Therefore, the first color of the first extended line and the second color of the second extended line can be changed to the target color, thereby informing the user to zero the existing steering wheel angle to control the stacker truck to travel straight to the pallet. It is understood that the first and second extended lines can also bend with the steering wheel rotation angle to assist the user in determining the steering wheel rotation angle. For example, if the steering wheel is turned to the left, the first and second extended lines can bend to the left. It is understood that if the first extended line does not intersect the first pallet hole area marker, and the second extended line does not intersect the fourth pallet hole area marker, it may also be that the stacker truck is too far from the pallet.

[0083] According to the above-described solution of the present invention, when the first extended line intersects with the first pallet hole area marker and the second extended line intersects with the fourth pallet hole area marker, the first color displayed in the first lens and the second color displayed in the second lens of the second extended line can both be adjusted to the target color. This solution can assist the user in controlling the stacker truck to control its movement. The user can obtain real-time feedback on the relative positions between the first forks and the first pallet hole, and between the second forks and the second pallet hole after moving the stacker truck through the first and second lenses, thereby reducing the need for user experience during operation and improving work efficiency.

[0084] For example, the above method further includes: determining a first transparency of a first image displayed in a first lens and a second transparency of a second image displayed in a second lens based on the region size of at least one of a first tray hole region identifier, a second tray hole region identifier, a third tray hole region identifier, and a fourth tray hole region identifier.

[0085] The transparency of both the first and second images is positively correlated with the size of the area. In a real-world scenario, as the user moves the stacker truck towards the target pallet, the binocular camera gradually approaches the pallet, causing the sizes of the first, second, third, and fourth pallet hole area markers to increase. During this process, because the user is wearing smart glasses, the approaching target pallet may create a visually oppressive feeling, causing psychological stress. Therefore, as the area size increases, the first transparency of the first image and the second transparency of both the second image can be increased. In other words, the closer the stacker truck gets to the target pallet, the more transparent the first and second images become. This reduces the aforementioned visual oppression and alleviates the user's psychological stress. The values ​​of the first and second transparency can be the same or different; this embodiment of the invention does not impose any limitations.

[0086] It is understandable that although the transparency of the first and second images has been adjusted, the transparency of the first pallet hole area marker, the second pallet hole area marker, the first fork area marker, the first extension line, the third pallet hole area marker, the fourth pallet hole area marker, the second fork area marker, and the second extension line mentioned above do not need to be adjusted, so as to maintain the auxiliary effect on the user's movement of the stacker truck.

[0087] According to the above-described solution of the present invention, a first transparency of the first image displayed in the first lens and a second transparency of the second image displayed in the second lens can be determined based on the area size of at least one of the first tray hole area markers, the second tray hole area markers, the third tray hole area markers, and the fourth tray hole area markers. This solution maintains the assistive effect for users moving stacker trucks. Furthermore, this solution can reduce the visual pressure on users from the approaching target tray, thus reducing their psychological stress.

[0088] For example, the above method may also include adjusting the target focal length based on the target travel distance of the stacker truck.

[0089] The target travel distance is the distance traveled by the stacker truck from the target time to the current time when the first pallet hole area marker and the second pallet hole area marker are determined. In one example, the target travel distance can be determined based on the stacker truck's moving speed combined with the travel time. In another example, it can also be determined based on the change in the area size of any one of the first to fourth pallet hole area markers from the target time to the current time. Considering the actual scenario, as the stacker truck gradually approaches the target pallet, the area size will gradually increase. The correspondence between the change in area size and the baseline travel distance can be predetermined. Subsequently, based on the change in size from the target time to the current time and this correspondence, the baseline travel distance corresponding to this change can be determined as the target travel distance.

[0090] The target focal length refers to the focal length used by the first lens and the second lens. Specifically, the target focal length can be the focal length used by the first lens to capture the first image and the focal length used by the second lens to capture the second image. In practical scenarios, if the focal lengths of the first and second lenses remain unchanged, the first and second images acquired in real-time may gradually become blurry as the stacker truck moves. Therefore, as the stacker truck approaches the target pallet, the target focal length can be reduced so that the first and second lenses can focus on the target pallet. For example, the target focal length used by the first and second lenses at a target time can be subtracted from the focal length adjustment amount corresponding to the target travel distance (which can be pre-calibrated) to obtain the target focal length used at the current time. It is understood that developers or users can also adjust the relationship between the target focal length and the target travel distance, and this embodiment of the invention does not impose limitations on this.

[0091] According to the above-described solution of the present invention, the target focal length can be adjusted based on the target travel distance of the stacker truck. This solution ensures that the clarity of the first image captured by the first lens and the clarity of the second image captured by the second lens are consistently high. This improves the accuracy of the first pallet hole area markers to the fourth pallet hole area markers, the first fork area markers, the second fork area markers, the first extension line, and the second extension line, thus facilitating accurate user control of the stacker truck's movement.

[0092] For example, the above method may further include: determining a first orientation indicator based on the area dimensions of both the first tray hole area identifier and the fourth tray hole area identifier.

[0093] The first orientation indicator can be represented by arrows, text, etc., as long as it indicates the direction of travel. For example, the first orientation indicator can be used to indicate whether a stacker truck is moving to the left or right. In one example, the first orientation indicator may be displayed in a first lens and / or a second lens.

[0094] When the area size of the first pallet hole region marker is larger than the area size of the fourth pallet hole region marker, the first orientation indicator is used to indicate that the stacker truck's travel direction is towards the second pallet hole and away from the first pallet hole. Taking the pallet hole region marker on the left as an example, since the area size of the first pallet hole region marker is larger than the fourth pallet hole region marker, it can be considered that the stacker truck's forks are not currently aligned with the pallet holes. Specifically, when the stacker truck's current travel direction is the reference travel direction (the travel direction that allows the first fork to enter the first pallet hole and the second fork to enter the second pallet hole), the area size of the first pallet hole region marker in the first mirror on the left is similar to the area size of the fourth pallet hole region marker in the second mirror on the right. Since the area size of the first pallet hole region marker in the first mirror is larger than the fourth pallet hole region marker, it can be considered that the current travel direction of the stacker truck is to the left of the reference travel direction. Therefore, the user needs to adjust the direction towards the second pallet hole on the right to make the current travel direction closer to the reference travel direction.

[0095] In one example, if the area size of the first pallet hole area identifier is larger than the area size of the fourth pallet hole area identifier, and the difference between their area sizes is greater than a difference threshold, then the first orientation indicator can be used to indicate that the stacker truck is traveling towards the second pallet hole and away from the first pallet hole. In another example, if the area size of the first pallet hole area identifier is larger than the area size of the fourth pallet hole area identifier, and the difference between their area sizes is less than a difference threshold, then the first orientation indicator may not need to be displayed. The specific value of the aforementioned difference threshold is not limited in this embodiment of the invention and can be determined according to the actual needs of the developers or users.

[0096] When the area size of the first pallet hole region marker is smaller than the area size of the fourth pallet hole region marker, the first orientation indicator is used to indicate the direction of travel towards the first pallet hole and away from the second pallet hole. Taking the pallet hole region marker on the left as an example, since the area size of the first pallet hole region marker is smaller than the fourth pallet hole region marker, it can be considered that the forks of the stacker truck are not currently aligned with the pallet hole. Specifically, when the current direction of travel of the stacker truck is the reference direction of travel, the area size of the first pallet hole region marker in the first mirror on the left is similar to the area size of the fourth pallet hole region marker in the second mirror on the right. Since the area size of the first pallet hole region marker is smaller than the fourth pallet hole marker, it can be considered that the current direction of travel of the stacker truck is biased to the right compared to the reference direction of travel. Therefore, the user needs to adjust the orientation towards the first pallet hole on the left to make the current direction of travel closer to the reference direction of travel.

[0097] In one example, if the area size of the first tray hole region identifier is smaller than the area size of the fourth tray hole region identifier, and the difference between their area sizes is greater than a difference threshold, then the first orientation indicator can be used to indicate the direction of travel towards the first tray hole and away from the second tray hole. In another example, if the area size of the first tray hole region identifier is smaller than the area size of the fourth tray hole region identifier, and the difference between their area sizes is less than a difference threshold, then the first orientation indicator may not need to be displayed. The specific value of this difference threshold is not limited in this embodiment of the invention and can be determined according to the actual needs of the developer or user.

[0098] According to the above-described solution of the present invention, the indication content of the first direction indicator can be determined based on the size relationship between the area size of the first pallet hole area indicator and the area size of the fourth pallet hole area indicator. This can instruct the user to adjust the travel direction of the stacker truck according to the first direction indicator, reducing the need for user experience during operation and improving work efficiency.

[0099] For example, the above method may further include: determining a second orientation indicator based on the area dimensions of both the second tray hole area identifier and the third tray hole area identifier.

[0100] The second orientation indicator can be represented by arrows, text, etc., and can indicate the direction of travel. For example, the second orientation indicator can be used to indicate whether the stacker truck is moving to the left or to the right. In one example, the second orientation indicator may be displayed in the first lens and / or the second lens.

[0101] When the area size of the second pallet hole area marker is larger than the area size of the third pallet hole area marker, the second orientation indicator is used to indicate the direction of travel towards the second pallet hole and away from the first pallet hole. Taking the pallet hole area marker of the first pallet hole on the left as an example, since the area size of the second pallet hole area marker is larger than the third pallet hole area marker, it can be considered that the forks of the stacker truck are not currently aligned with the pallet hole. Specifically, when the current direction of travel of the stacker truck is the reference direction of travel, the area size of the second pallet hole area marker in the first mirror on the left is similar to the area size of the third pallet hole area marker in the second mirror on the right. Since the area size of the second pallet hole area marker is larger than the third pallet hole marker, it can be considered that the current direction of travel of the stacker truck is to the left compared to the reference direction of travel. Therefore, the user needs to adjust the direction towards the second pallet hole on the right to make the current direction of travel closer to the reference direction of travel.

[0102] In one example, if the area size of the second pallet hole region identifier is larger than the area size of the third pallet hole region identifier, and the difference between their area sizes is greater than a difference threshold, then the second orientation indicator can be used to indicate that the stacker truck's travel direction is toward the second pallet hole and away from the first pallet hole. In another example, if the area size of the second pallet hole region identifier is larger than the area size of the third pallet hole region identifier, and the difference between their area sizes is less than a difference threshold, then the second orientation indicator may not need to be displayed. The specific value of the aforementioned difference threshold is not limited in this embodiment of the invention and can be determined according to the actual needs of the developers or users.

[0103] When the area size of the second pallet hole area marker is smaller than the area size of the third pallet hole area marker, the second orientation indicator is used to indicate the direction of travel towards the first pallet hole and away from the second pallet hole. Taking the pallet hole area marker of the first pallet hole on the left as an example, since the area size of the second pallet hole area marker is smaller than the third pallet hole area marker, it can be considered that the forks of the stacker truck are not currently aligned with the pallet hole. Specifically, when the current direction of travel of the stacker truck is the reference direction of travel, the area size of the second pallet hole area marker in the first mirror on the left is similar to the area size of the third pallet hole area marker in the second mirror on the right. Since the area size of the second pallet hole area marker is smaller than the third pallet hole marker, it can be considered that the current direction of travel of the stacker truck is biased to the right compared to the reference direction of travel. Therefore, the user needs to adjust the orientation towards the first pallet hole on the left to make the current direction of travel closer to the reference direction of travel.

[0104] In one example, if the area size of the second pallet hole area identifier is smaller than the area size of the third pallet hole area identifier, and the difference between their area sizes is greater than a difference threshold, then the second orientation indicator can be used to indicate that the stacker truck's travel direction is toward the first pallet hole and away from the second pallet hole. In another example, if the area size of the second pallet hole area identifier is smaller than the area size of the third pallet hole area identifier, and the difference between their area sizes is less than a difference threshold, then the second orientation indicator may not need to be displayed. The specific value of the aforementioned difference threshold is not limited in this embodiment of the invention and can be determined according to the actual needs of the developers or users.

[0105] According to the above-described solution of the present invention, the indication content of the second direction indicator can be determined based on the size relationship between the area size of the second pallet hole area indicator and the area size of the third pallet hole area indicator. This can instruct the user to adjust the travel direction of the stacker truck according to the second direction indicator, reducing the need for user experience during operation and improving work efficiency.

[0106] For example, the above method includes determining a first orientation indicator based on the area dimensions of both the first tray hole area identifier and the fourth tray hole area identifier, as described above, and determining a second orientation indicator based on the area dimensions of both the second tray hole area identifier and the third tray hole area identifier.

[0107] In this case, the above method further includes steps S210 to S230.

[0108] In step S210, based on the area dimensions of the first tray hole area identifier and the fourth tray hole area identifier, the first weight corresponding to the first orientation indicator is determined.

[0109] The difference in area size between the first tray hole area identifier and the fourth tray hole area identifier is positively correlated with the first weight. The greater the difference in area size between the first tray hole area identifier and the fourth tray hole area identifier, the lower the probability of misjudgment of the first tray hole area identifier, and the higher the representativeness of the first orientation indicator, i.e., the higher the first weight can be.

[0110] In step S220, based on the area dimensions of the second tray hole area identifier and the third tray hole area identifier, the second weight corresponding to the second orientation indicator is determined.

[0111] The difference in area size between the second tray hole area marker and the third tray hole area marker is positively correlated with the second weight. The greater the difference in area size between the second tray hole area marker and the third tray hole area marker, the lower the probability of misjudgment of the second tray hole area marker, and the higher the representativeness of the second orientation indicator, i.e., the higher the second weight can be.

[0112] In step S230, the final orientation indicator is determined based on the first orientation indicator, the second orientation indicator, the first weight, and the second weight.

[0113] The final orientation indicator can be represented by arrows, text, etc., as long as it indicates the direction of travel. For example, the final orientation indicator can be used to indicate whether a stacker truck is moving to the left or right. The final orientation indicator is displayed in the first and / or second mirror.

[0114] In one example, if the first orientation indicator and the second orientation indicator indicate the same direction of movement for the stacker truck (e.g., both indicating the stacker truck to move to the right), then either the first or second orientation indicator can be used as the final orientation indicator. In another example, if the first and second orientation indicators indicate different directions of movement for the stacker truck, and the first weight is greater than the second weight, then the first orientation indicator can be used as the final orientation indicator. In yet another example, if the first and second orientation indicators indicate different directions of movement for the stacker truck, and the second weight is greater than the first weight, then the second orientation indicator can be used as the final orientation indicator.

[0115] According to the above-described scheme of the present invention, a first weight corresponding to the first orientation indicator can be determined based on the area dimensions of both the first pallet hole area identifier and the fourth pallet hole area identifier. Then, a second weight corresponding to the second orientation indicator can be determined based on the area dimensions of both the second pallet hole area identifier and the third pallet hole area identifier. Finally, a final orientation indicator can be determined based on the first orientation indicator, the second orientation indicator, the first weight, and the second weight. The final orientation indicator determined by the above scheme has higher accuracy. This allows the user to accurately adjust the travel direction of the stacker truck according to the final orientation indicator, reducing the need for user experience during operation and improving work efficiency.

[0116] This invention also provides a binocular imaging device. Figure 3 A schematic block diagram of a binocular imaging device 300 according to an embodiment of the present invention is shown. (In conjunction with...) Figure 3 As shown, the processing device 300 may include an image acquisition module 310, a tray hole area determination module 320, and an image display module 330.

[0117] The image acquisition module 310 is used to acquire a first image and a second image, wherein the first image is acquired through the first lens of the binocular camera and the second image is acquired through the second lens of the binocular camera.

[0118] The pallet hole area determination module 320 is used to determine the first pallet hole area corresponding to the first pallet hole of the target pallet in the first image and the second pallet hole area corresponding to the second pallet hole of the target pallet, and to determine the third pallet hole area corresponding to the first pallet hole and the fourth pallet hole area corresponding to the second pallet hole in the second image, wherein the first pallet hole is used to accommodate the first fork of the stacker truck and the second pallet hole is used to accommodate the second fork of the stacker truck.

[0119] The image display module 330 is used to control the first lens of the smart glasses to display at least the first image, the first tray hole area and the second tray hole area, and to control the second lens of the smart glasses to display at least the second image, the third tray hole area and the fourth tray hole area.

[0120] For example, the image display module 330 includes a first image display submodule and a second image display submodule.

[0121] The first image display submodule is used to control the first lens to display at least the first image, the first pallet hole area identifier, the second pallet hole area identifier, and the first fork area identifier, wherein the first fork area identifier is used to mark the area corresponding to the first fork in the first image.

[0122] The second image display submodule is used to control the second lens to display at least the second image, the third pallet hole area identifier, the fourth pallet hole area identifier, and the second fork area identifier, wherein the second fork area identifier is used to mark the area corresponding to the second fork in the second image.

[0123] For example, the first image display submodule includes a first display unit. The second image display submodule includes a second display unit.

[0124] The first display unit is used to control the first lens to display at least a first image, a first pallet hole area identifier, a second pallet hole area identifier, a first fork area identifier, and a first extension line, wherein the first extension line is used to represent the extension line of the first fork away from the side of the stacker truck.

[0125] The second display unit is used to control the second lens to display at least the second image, the third pallet hole area mark, the fourth pallet hole area mark, the second fork area mark, and the second extension line, wherein the second extension line is used to indicate the extension line of the second fork away from the side of the stacker truck.

[0126] For example, the binocular imaging device 300 also includes a target color display module.

[0127] The target color display module is used to adjust the first color displayed in the first lens by the first extension line and the second color displayed in the second lens by the second extension line to the target color when the first extension line intersects with the first tray hole area mark and the second extension line intersects with the fourth tray hole area mark. The first color and the second color are different from the target color.

[0128] For example, the binocular imaging device 300 also includes a transparency determination module.

[0129] The transparency determination module is used to determine the first transparency of the first image displayed in the first lens and the second transparency of the second image displayed in the second lens based on the area size of at least one of the first tray hole area identifier, the second tray hole area identifier, the third tray hole area identifier, and the fourth tray hole area identifier, wherein the transparency of both the first image and the second image is positively correlated with the area size.

[0130] For example, the binocular imaging device 300 also includes a target focal length determination module.

[0131] The target focal length determination module is used to adjust the target focal length based on the target travel distance of the stacker truck. The target travel distance is the travel distance of the stacker truck from the target time when the first pallet hole area identifier and the second pallet hole area identifier are determined to the current time. The target focal length is the focal length used by the first lens and the second lens.

[0132] For example, the binocular imaging device 300 further includes a first orientation indicator determination module.

[0133] The first orientation indicator determination module is used to determine the first orientation indicator based on the area size of both the first pallet hole area indicator and the fourth pallet hole area indicator. When the area size of the first pallet hole area indicator is greater than the area size of the fourth pallet hole area indicator, the first orientation indicator is used to indicate that the traveling direction of the stacker truck is toward the second pallet hole and away from the first pallet hole. When the area size of the first pallet hole area indicator is less than the area size of the fourth pallet hole area indicator, the first orientation indicator is used to indicate that the traveling direction is toward the first pallet hole and away from the second pallet hole.

[0134] For example, the binocular imaging device 300 further includes a second orientation indicator determination module.

[0135] The second orientation indicator determination module is used to determine the second orientation indicator based on the area size of both the second tray hole area indicator and the third tray hole area indicator. When the area size of the second tray hole area indicator is greater than the area size of the third tray hole area indicator, the second orientation indicator is used to indicate the direction of travel toward the second tray hole and away from the first tray hole. When the area size of the second tray hole area indicator is less than the area size of the third tray hole area indicator, the second orientation indicator is used to indicate the direction of travel toward the first tray hole and away from the second tray hole.

[0136] For example, the binocular imaging device 300 further includes the aforementioned first orientation indicator determination module and the aforementioned second orientation indicator determination module. The specific details of the first orientation indicator determination module and the second orientation indicator determination module can be found above.

[0137] For example, when the binocular imaging device 300 includes a first orientation indicator determination module and the aforementioned second orientation indicator determination module, the binocular imaging device 300 further includes a first weight determination module, a second weight determination module, and a final orientation indicator determination module.

[0138] The first weight determination module is used to determine the first weight corresponding to the first orientation indicator based on the area size of the first tray hole area identifier and the fourth tray hole area identifier, wherein the difference between the area size of the first tray hole area identifier and the fourth tray hole area identifier is positively correlated with the first weight.

[0139] The second weight determination module is used to determine the second weight corresponding to the second orientation indicator based on the area size of the second tray hole area identifier and the third tray hole area identifier, wherein the difference between the area size of the second tray hole area identifier and the third tray hole area identifier is positively correlated with the second weight.

[0140] The final orientation indicator determination module is used to determine the final orientation indicator based on the first orientation indicator, the second orientation indicator, the first weight, and the second weight, wherein the final orientation indicator is displayed in the first lens and / or the second lens.

[0141] According to another aspect of the present invention, an electronic device is also provided, the electronic device including a memory and a processor, wherein: the memory is used to store a computer program; and the processor is used to execute the computer program to implement the above-described binocular imaging method. Figure 4 A schematic block diagram of an electronic device 400 according to an embodiment of the present invention is shown. Figure 4 As shown, the electronic device 400 includes a processor 410 and a memory 420. The memory 420 stores a computer program, and the computer program instructions are executed by the processor 410 to perform the above-described binocular imaging method.

[0142] According to another aspect of the present invention, a computer program product is also provided, including computer program instructions, which, when run by a processor, are used to execute the above-described binocular imaging method.

[0143] Furthermore, according to another aspect of the present invention, a storage medium is provided, on which program instructions are stored. When the program instructions are executed by a computer or processor, the computer or processor performs corresponding steps of the binocular imaging method described in the embodiments of the present invention, and is used to implement corresponding modules in the binocular imaging device or the electronic device described in the embodiments of the present invention. The storage medium may, for example, include a memory card of a smartphone, a storage component of a tablet computer, a hard disk of a personal computer, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disc read-only memory (CD-ROM), a USB memory, or any combination of the above storage media. A computer-readable storage medium may be any combination of one or more computer-readable storage media. According to yet another aspect of the present invention, a computer program product is also provided, including computer program instructions. When the computer program instructions are executed by a computer or processor, the computer or processor performs corresponding steps of the binocular imaging method described above.

[0144] Those skilled in the art can understand the specific implementation scheme of the above-mentioned electronic device and storage medium by reading the relevant description of the binocular imaging method, and for the sake of brevity, they will not be described in detail here.

[0145] Although exemplary embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above exemplary embodiments are merely illustrative and are not intended to limit the scope of the invention. Various changes and modifications can be made therein by those skilled in the art without departing from the scope and spirit of the invention. All such changes and modifications are intended to be included within the scope of the invention as claimed in the appended claims.

[0146] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0147] In the several embodiments provided by this invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed.

[0148] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0149] Similarly, it should be understood that, in order to streamline the invention and aid in understanding one or more of the various aspects of the invention, features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof in the description of exemplary embodiments of the invention. However, this approach should not be construed as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the corresponding claims, its inventive point lies in solving the corresponding technical problem with fewer features than all of those in a single disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of the invention.

[0150] Those skilled in the art will understand that, apart from the mutual exclusion of features, all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or apparatus so disclosed can be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0151] Furthermore, those skilled in the art will understand that although some embodiments herein include certain features included in other embodiments but not others, combinations of features from different embodiments are intended to be within the scope of the invention and form different embodiments. For example, in the claims, any of the claimed embodiments can be used in any combination.

[0152] The various component embodiments of the present invention can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some modules in the binocular imaging device according to embodiments of the present invention. The present invention can also be implemented as an apparatus program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such programs implementing the present invention can be stored on a computer-readable medium or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0153] It should be noted that the above embodiments are illustrative of the invention and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0154] The above are merely specific embodiments or descriptions of the present invention, and the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. The scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A binocular imaging method, characterized in that, The method includes: Acquire a first image and a second image, wherein the first image is acquired through a first lens in a binocular camera, and the second image is acquired through a second lens in the binocular camera; the first image includes the first fork of a stacker truck, and the second image includes the second fork of the stacker truck. The first pallet hole region identifier corresponding to the first pallet hole of the target pallet in the first image and the second pallet hole region identifier corresponding to the second pallet hole of the target pallet are determined, and the third pallet hole region identifier corresponding to the first pallet hole and the fourth pallet hole region identifier corresponding to the second pallet hole in the second image are determined, wherein the first pallet hole is used to accommodate the first fork and the second pallet hole is used to accommodate the second fork. The first lens of the smart glasses is controlled to display at least the first image, the first tray hole area identifier, and the second tray hole area identifier, and the second lens of the smart glasses is controlled to display at least the second image, the third tray hole area identifier, and the fourth tray hole area identifier.

2. The method as described in claim 1, characterized in that, The first lens of the smart glasses displays at least the first image, the first tray hole area identifier, and the second tray hole area identifier, including: The first lens is controlled to display at least the first image, the first tray hole area identifier, the second tray hole area identifier, and the first fork area identifier, wherein the first fork area identifier is used to mark the area corresponding to the first fork in the first image; The second lens controlling the smart glasses displays at least the second image, the third tray hole area identifier, and the fourth tray hole area identifier, including: The second lens is controlled to display at least the second image, the third pallet hole area identifier, the fourth pallet hole area identifier, and the second fork area identifier, wherein the second fork area identifier is used to mark the area corresponding to the second fork in the second image.

3. The method as described in claim 2, characterized in that, The control of the first lens to display at least the first image, the first pallet hole area identifier, the second pallet hole area identifier, and the first fork area identifier includes: The first lens is controlled to display at least the first image, the first pallet hole area identifier, the second pallet hole area identifier, the first fork area identifier, and the first extension line, wherein the first extension line is used to represent the extension line of the first fork away from the stacker truck; The control of the second lens to display at least the second image, the third pallet hole area identifier, the fourth pallet hole area identifier, and the second fork area identifier includes: The second lens is controlled to display at least the second image, the third pallet hole area identifier, the fourth pallet hole area identifier, the second fork area identifier, and the second extension line, wherein the second extension line is used to indicate the extension line of the second fork away from the side of the stacker truck.

4. The method as described in claim 3, characterized in that, The method further includes: When the first extended line intersects with the first tray hole area mark and the second extended line intersects with the fourth tray hole area mark, the first color displayed by the first extended line in the first lens and the second color displayed by the second extended line in the second lens are both adjusted to the target color, wherein the first color and the second color are different from the target color.

5. The method as described in claim 1, characterized in that, The method further includes: Based on the region size of at least one of the first tray hole region identifier, the second tray hole region identifier, the third tray hole region identifier, and the fourth tray hole region identifier, a first transparency of the first image displayed in the first lens and a second transparency of the second image displayed in the second lens are determined, wherein the transparency of both the first image and the second image is positively correlated with the region size.

6. The method as described in claim 1, characterized in that, The method further includes: Based on the target travel distance of the stacker truck, the target focal length is adjusted, wherein the target travel distance is the travel distance of the stacker truck from the target time when the first pallet hole area identifier and the second pallet hole area identifier were determined to the current time, and the target focal length is the focal length used by the first lens and the second lens.

7. The method as described in claim 1, characterized in that, The method further includes any one of the following: Based on the area dimensions of the first pallet hole area marker and the fourth pallet hole area marker, a first orientation indicator is determined. Wherein, if the area dimension of the first pallet hole area marker is larger than the area dimension of the fourth pallet hole area marker, the first orientation indicator is used to indicate that the traveling direction of the stacker truck is toward the second pallet hole and away from the first pallet hole. If the area dimension of the first pallet hole area marker is smaller than the area dimension of the fourth pallet hole area marker, the first orientation indicator is used to indicate that the traveling direction is toward the first pallet hole and away from the second pallet hole. A second orientation indicator is determined based on the area dimensions of both the second tray hole area identifier and the third tray hole area identifier. When the area dimension of the second tray hole area identifier is greater than that of the third tray hole area identifier, the second orientation indicator indicates that the travel direction is toward the second tray hole and away from the first tray hole. When the area dimension of the second tray hole area identifier is smaller than that of the third tray hole area identifier, the second orientation indicator indicates that the travel direction is toward the first tray hole and away from the second tray hole.

8. The method as described in claim 7, characterized in that, The method includes: Based on the area dimensions of the first tray hole area identifier and the fourth tray hole area identifier, a first orientation indicator is determined, and based on the area dimensions of the second tray hole area identifier and the third tray hole area identifier, a second orientation indicator is determined. The method further includes: Based on the area sizes of the first tray hole area identifier and the fourth tray hole area identifier, a first weight corresponding to the first orientation indicator is determined, wherein the difference between the area sizes of the first tray hole area identifier and the fourth tray hole area identifier is positively correlated with the first weight; Based on the area dimensions of the second tray hole area identifier and the third tray hole area identifier, a second weight corresponding to the second orientation indicator is determined, wherein the difference between the area dimensions of the second tray hole area identifier and the third tray hole area identifier is positively correlated with the second weight; A final orientation indicator is determined based on the first orientation indicator, the second orientation indicator, the first weight, and the second weight, wherein the final orientation indicator is displayed in the first lens and / or the second lens.

9. A binocular imaging device, characterized in that, The binocular imaging device includes: An image acquisition module is used to acquire a first image and a second image, wherein the first image is acquired through a first lens in a binocular camera, and the second image is acquired through a second lens in the binocular camera; the first image includes the first fork of a stacker truck, and the second image includes the second fork of the stacker truck. The pallet hole area identification determination module is used to determine the first pallet hole area identification corresponding to the first pallet hole of the target pallet in the first image and the second pallet hole area identification corresponding to the second pallet hole of the target pallet, and to determine the third pallet hole area identification corresponding to the first pallet hole and the fourth pallet hole area identification corresponding to the second pallet hole in the second image, wherein the first pallet hole is used to accommodate the first fork and the second pallet hole is used to accommodate the second fork. An image display module is used to control the first lens of the smart glasses to display at least the first image, the first tray hole area identifier, and the second tray hole area identifier, and to control the second lens of the smart glasses to display at least the second image, the third tray hole area identifier, and the fourth tray hole area identifier.

10. An electronic device, characterized in that, It includes a memory and a processor, wherein: the memory is used to store a computer program; and the processor is used to execute the computer program to implement the binocular imaging method as described in any one of claims 1-8.

11. A storage medium storing computer program instructions, characterized in that, The computer program instructions, when executed, are used to perform the binocular imaging method as described in any one of claims 1-8.

12. A computer program product comprising computer program instructions, characterized in that, The computer program instructions, when executed by a processor, are used to perform the binocular imaging method as described in any one of claims 1-8.

Images