Image splicing method and device and hearing electronic equipment

Abstract

The invention relates to an image splicing method and device and hearing electronic equipment, and the method comprises the steps: obtaining a first image collected by a first camera and a second image collected by a second camera, the first camera is disposed on a left headset, and the second camera is disposed on a right headset; determining a left-side head contour of the user according to the first image, and determining a right-side head contour of the user according to the second image; cutting off an image surrounded by the left head contour of the user from the first image to obtain a first cut image, and cutting off an image surrounded by the right head contour of the user from the second image to obtain a second cut image; obtaining a first key point pair according to the left-side head contour of the user and the right-side head contour of the user; according to the first cutting image and the second cutting image, matching to obtain a second key point pair; and splicing the first image and the second image according to the first key point pair and the second key point pair to obtain a spliced image.

Description

Technical Field

[0001] The present disclosure relates to a hearing device image processing technology, and more specifically, to an image stitching method, device and hearing electronic device. Background Art

[0002] Hearing electronic devices have become very popular electronic products on the market. They can realize audio playback and meet people's life and learning needs, especially TWS (True Wireless Stereo) headphones, which realize wireless audio transmission through Bluetooth technology, providing a convenient wearing experience and rich functions (such as listening to music, making calls, active noise reduction, spatial sound effects, etc.).

[0003] At present, the functions of hearing electronic devices are mainly focused on audio, and other functions, such as shooting functions, are also beginning to be involved. After the user wears the headphones, the camera set on the left earphone and the camera set on the right earphone can shoot. Then the images or videos taken by the left and right cameras are spliced ​​together to form a seamless image or video.

[0004] Existing stitching algorithms are prone to stitching errors when processing images or video streams captured by left and right cameras, resulting in discontinuous video images or ghosting.

[0005] Therefore, it is necessary to provide a new technical solution to improve the coherence of the spliced ​​image pictures.

[0006] Public Content

[0007] One object of the present disclosure is to provide a new technical solution for an image stitching method.

[0008] According to a first aspect of the present disclosure, there is provided an image stitching method, comprising:

[0009] Acquire a first image captured by a first camera and a second image captured by a second camera, wherein the first camera is arranged on a left earphone and the second camera is arranged on a right earphone;

[0010] Determine a left side head contour of the user according to the first image, and determine a right side head contour of the user according to the second image;

[0011] cropping the image surrounded by the left side of the user's head outline from the first image to obtain a first cropped image, and cropping the image surrounded by the right side of the user's head outline from the second image to obtain a second cropped image;

[0012] Obtaining a first key point pair according to a left head contour of the user and a right head contour of the user;

[0013] According to the first cropped image and the second cropped image, matching is performed to obtain a second key point pair;

[0014] The first image and the second image are spliced ​​according to the first key point pair and the second key point pair to obtain a spliced ​​image.

[0015] Optionally, the performing stitching processing on the first image and the second image according to the first key point pair and the second key point pair to obtain a stitched image includes:

[0016] Determine a homography matrix between the first image and the second image according to the first key point pair and the second key point pair;

[0017] Aligning the first image and the second image according to the homography matrix to obtain an aligned first image and a second image;

[0018] The aligned first image and the second image are spliced ​​to obtain a spliced ​​image.

[0019] Optionally, obtaining a first key point pair according to a left head contour of the user and a right head contour of the user includes:

[0020] Identifying a first specific location point from a left side head contour of the user, and identifying a second specific location point from a right side head contour of the user;

[0021] A first key point pair is obtained according to the first specific position point and the second specific position point.

[0022] Optionally, matching the first cropped image and the second cropped image to obtain a second key point pair includes:

[0023] A feature point matching algorithm is used to match the first cropped image and the second cropped image to obtain a second key point pair.

[0024] Optionally, the method further includes:

[0025] A first recognition result of the occluder is determined based on the first cropped image, and a second recognition result of the occluder is determined based on the second cropped image, wherein the first recognition result and the second recognition result both include whether the occluder exists and the category of the occluder.

[0026] Optionally, the first recognition result and the second recognition result also include the location of the obstruction.

[0027] Optionally, the method further includes:

[0028] When the first recognition result and / or the second recognition result indicates that an obstruction exists, a prompt indicating the existence of the obstruction is sent to the user.

[0029] Optionally, determining a first recognition result of the occluder according to the first cropped image, and determining a second recognition result of the occluder according to the second cropped image includes:

[0030] The first cropped image is input into a trained deep learning model to obtain a first recognition result of the occluder, and the second cropped image is input into the trained deep learning model to obtain a second recognition result of the occluder.

[0031] According to a second aspect of the present disclosure, there is provided an image stitching device, comprising:

[0032] An acquisition module, configured to acquire a first image captured by a first camera and a second image captured by a second camera, wherein the first camera is arranged on a left earphone and the second camera is arranged on a right earphone;

[0033] A facial contour determination module, configured to determine a left side head contour of a user based on the first image, and to determine a right side head contour of the user based on the second image;

[0034] an image cropping module, configured to crop the image surrounded by the left side of the user's head outline from the first image to obtain a first cropped image, and to crop the image surrounded by the right side of the user's head outline from the second image to obtain a second cropped image;

[0035] A first key point pair determination module, configured to obtain a first key point pair according to a left head contour of the user and a right head contour of the user;

[0036] A second key point pair determination module, configured to obtain a second key point pair by matching the first cropped image and the second cropped image;

[0037] A stitching module is used to stitch the first image and the second image according to the first key point pair and the second key point pair to obtain a stitched image.

[0038] According to a third aspect of the present disclosure, a hearing electronic device is provided, comprising a memory and a processor, wherein the memory stores a computer program, and the computer program is used to control the processor to operate so as to execute the method according to any one of the first aspects of the present disclosure.

[0039] The image stitching method provided by the present invention obtains a first key point pair according to the left head contour and the right head contour of a user, that is, the first key point pair is obtained by utilizing the left-right symmetry of the head, so that the obtained first key point pair is more accurate, and then a second key point pair is obtained from the first cropped image and the second cropped image, and the first image and the second image are stitched together in combination with the first key point pair and the second key point pair to obtain a stitched image. Since the first key point pair and the second key point pair are determined from different image areas, a more accurate basis is provided for the image stitching process, so that the picture presented by the stitched image is more coherent and more accurate.

[0040] Features and advantages of the embodiments of the present specification will become apparent from the following detailed description of exemplary embodiments of the present specification with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the specification and, together with the description, serve to explain the principles of the embodiments of the specification.

[0042] Figure 1 is a processing flow chart of an image stitching method according to an embodiment of the present invention.

[0043] Figure 2 4 is a principle block diagram of an image stitching device according to an embodiment of the present invention.

[0044] Figure 3 FIG. 4 is a hardware structure block diagram of a hearing electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0045] Various exemplary embodiments of the present specification will now be described in detail with reference to the accompanying drawings.

[0046] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the embodiments of the present specification and its application or uses.

[0047] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0048] In one embodiment of the present invention, an image stitching method is provided. Figure 1 As shown, the image stitching method of this embodiment may include the following steps S110 to S160.

[0049] Step S110, acquiring a first image captured by a first camera and a second image captured by a second camera, wherein the first camera is arranged on the left earphone and the second camera is arranged on the right earphone.

[0050] The first image is a static image captured by the first camera alone, or a frame of image extracted from a video captured by the first camera. The second image is a static image captured by the second camera alone, or a frame of image extracted from a video captured by the second camera.

[0051] It should be noted that after the user wears the left earphone and the right earphone, the first camera set on the left earphone and the second camera set on the right earphone are close to the user's head, so that the shooting angle of the first camera is blocked by the left side of the user's head, and the first image collected contains the image corresponding to the left side of the user's head, and the shooting angle of the second camera is blocked by the right side of the user's head, and the second image collected contains the image corresponding to the right side of the user's head.

[0052] Step S120: determining a left side head contour of the user according to the first image, and determining a right side head contour of the user according to the second image.

[0053] In one embodiment, a head detection algorithm is used to determine the left face outline and / or left ear outline of the user from the first image, and a head detection algorithm is used to determine the right face outline and / or right ear outline of the user from the second image.

[0054] The head detection algorithm can be an MTCNN (Multi-task convolutional neural network) algorithm, an SSD (Single Shot MultiBox Detector) algorithm, or the like.

[0055] In one embodiment, before executing step S120, the method further includes: performing denoising and grayscale processing on the image captured by the camera to obtain a processed image. The processed image is used in subsequent execution steps, which can reduce useless interference information and the influence of noise in the image. In addition, the grayscale processing can change the intensity distribution of image pixels, making the difference between the bright and dark parts of the image more obvious, which is helpful for determining the key point pairs.

[0056] Step S130 , cropping the image surrounded by the left side of the user's head outline from the first image to obtain a first cropped image, and cropping the image surrounded by the right side of the user's head outline from the second image to obtain a second cropped image.

[0057] The first cropped image is a picture taken by the first camera and not blocked by the user's head. The second cropped image is a picture taken by the second camera and not blocked by the user's head.

[0058] Step S140: obtaining a first key point pair according to the left head contour of the user and the right head contour of the user.

[0059] Since the left and right sides of a human face have a high degree of symmetry, the first key point pair can be obtained by using the left-right symmetry of the human face. In one embodiment, step S140 specifically includes: identifying a first specific position point from the left side head contour of the user, the first specific position point includes at least one of a position point corresponding to the left corner of the eye, a position point corresponding to the highest point of the left cheekbone, and a position point corresponding to the left ear; identifying a second specific position point from the right side head contour of the user, the second specific position point includes at least one of a position point corresponding to the right corner of the eye, a position point corresponding to the highest point of the right cheekbone, and a position point corresponding to the right ear; and obtaining a first key point pair based on the first specific position point and the second specific position point.

[0060] When the first specific position point includes the position point corresponding to the left corner of the eye, the position point corresponding to the highest point of the left cheekbone, and the position point corresponding to the left ear, and the second specific position point includes the position point corresponding to the right corner of the eye, the position point corresponding to the highest point of the right cheekbone, and the position point corresponding to the right ear, the first key point pair is three pairs. One pair is the position point corresponding to the left corner of the eye and the position point corresponding to the right corner of the eye, another pair is the position point corresponding to the highest point of the left cheekbone and the position point corresponding to the highest point of the right cheekbone, and another pair is the position point corresponding to the left ear and the position point corresponding to the right ear.

[0061] Step S150: obtaining a second key point pair by matching the first cropped image and the second cropped image.

[0062] In one embodiment, a feature point matching algorithm is used to match the first cropped image and the second cropped image to obtain a second key point pair. Specifically, an image corresponding to the same object is determined from the first cropped image and the second cropped image; and a second key point pair is matched from the image corresponding to the same object. The image corresponding to the same object includes a picture corresponding to the object determined from the first cropped image and a picture corresponding to the object determined from the second cropped image.

[0063] For example, when the same object determined from the first cropped image and the second cropped image is a high-rise building, the second key point pair obtained by matching may be at least one pair of feature points determined from the corresponding lines at the highest point of the high-rise building.

[0064] For example, when the same object determined from the first cropped image and the second cropped image is a circular object, the second key point pair obtained by matching may be a pair of feature points corresponding to the center of the object.

[0065] The feature point matching algorithm is a SIFT (Scale-Invariant Feature Transform) algorithm or a SURF (Speeded-Up Robust Features) algorithm.

[0066] Step S160: splicing the first image and the second image according to the first key point pair and the second key point pair to obtain a spliced ​​image.

[0067] In one embodiment, step S160 specifically includes: determining the homography matrix between the first image and the second image based on the first key point pair and the second key point pair; aligning the first image and the second image according to the homography matrix to obtain aligned first image and second image; and stitching the aligned first image and second image to obtain a stitched image.

[0068] Specifically, based on the RANSAC (Random Sample Consensus) algorithm, the homography matrix between the first image and the second image is determined using the first key point pair and the second key point pair. The homography matrix is ​​used to describe the transformation relationship between the first image and the second image, that is, the homography matrix describes the mapping relationship between each key point pair. According to the homography matrix, the first image is transformed to obtain the transformed first image so that the transformed first image is aligned with the second image, or, according to the homography matrix, the second image is transformed to obtain the transformed second image so that the transformed second image is aligned with the first image. The aligned first image and second image are spliced ​​to obtain a spliced ​​image.

[0069] The image stitching method provided by the embodiment of the present invention obtains a first key point pair according to the left head contour and the right head contour of the user, that is, the first key point pair is obtained by utilizing the left-right symmetry of the head, so that the obtained first key point pair is more accurate, and then the second key point pair is obtained from the first cropped image and the second cropped image. The first image and the second image are stitched together in combination with the first key point pair and the second key point pair to obtain a stitched image. Since the first key point pair and the second key point pair are determined from different image areas, a more accurate basis is provided for the image stitching process, so that the picture presented by the stitched image is more coherent and more accurate.

[0070] In one embodiment, the method further includes: determining a first recognition result of the occluder based on the first cropped image, and determining a second recognition result of the occluder based on the second cropped image, wherein both the first recognition result and the second recognition result include whether the occluder exists and the category of the occluder.

[0071] Based on any one of the first recognition result and the second recognition result, when the result of the item "whether there is an obstruction" in the recognition result is yes, the result of the item "category of the obstruction" in the recognition result is a specific category. When the result of the item "whether there is an obstruction" in the recognition result is no, the result of the item "category of the obstruction" in the recognition result is no.

[0072] In one embodiment, the first recognition result and the second recognition result also include the position of the obstruction. The position of the obstruction is a position relative to the headset. For example, the position of the obstruction is located in front of the headset camera, upper left, upper right, lower left or lower right.

[0073] In one embodiment, the method further includes: when the first recognition result and / or the second recognition result indicates that an obstruction exists, a prompt is issued to the user that an obstruction exists. In this way, when a temporary obstruction exists, the user can be promptly prompted so that the user can promptly eliminate the influence of the obstruction on the camera shooting.

[0074] In one embodiment, the headset includes a speaker. When the first recognition result and / or the second recognition result indicates that an obstruction exists, the speaker is controlled to emit a prompt voice to prompt the user that an obstruction exists.

[0075] When only the first recognition result is that there is an obstruction, the control speaker issues the following prompt voice: there is a temporary obstruction on the left earphone.

[0076] When only the second recognition result is that there is an obstruction, the control speaker issues the following prompt voice: there is a temporary obstruction in the right earphone.

[0077] When both the first recognition result and the second recognition result show that there is an obstruction, the control speaker issues the following prompt voice: both the left earphone and the right earphone are temporarily obstructed.

[0078] In one embodiment, the headset includes a vibration motor. When the recognition result shows that an obstruction exists, the vibration motor is controlled to vibrate to prompt the user that the obstruction exists.

[0079] In one embodiment, the left earphone is provided with a vibration motor and the right earphone is provided with a vibration motor.

[0080] When only the first recognition result is that an obstruction exists, the vibration motor provided on the left earphone is controlled to generate vibration.

[0081] When only the second recognition result is that an obstruction exists, the vibration motor provided on the right earphone is controlled to generate vibration.

[0082] When both the first recognition result and the second recognition result indicate that an obstruction exists, the vibration motor disposed on the left earphone is controlled to generate vibration, and the vibration motor disposed on the right earphone is controlled to generate vibration.

[0083] In one embodiment, the training process of the trained deep learning model is as follows: obtain a training sample set and a test sample set, each sample includes an image after cropping the face area and the annotation result of the occlusion, and the annotation result of the occlusion includes whether there is an occlusion and the category of the occlusion; use the training sample set to train the deep learning model to obtain the trained deep learning model; use the test sample set to verify the trained deep learning model to obtain the verification result; when the verification result is that the accuracy of the output result of the trained deep learning model reaches a preset threshold, the trained deep learning model is used as the trained deep learning model. The training process of the deep learning model is the optimization process of the parameters in the deep learning model. The image after cropping the face area in each sample is cropped based on the image captured by the camera set on the headset. The specific method of determining the face area can refer to the above embodiment, and no further details are given here.

[0084] When the verification result is that the accuracy of the output results of the trained deep learning model does not reach the preset threshold, continue to train the deep learning model using the training sample set until the accuracy of the output results of the trained deep learning model reaches the preset threshold using the test sample set, and then stop training the deep learning model.

[0085] The annotation results of occlusions are based on manual annotation. The annotation results of occlusions can be hair, glasses, hands, hats, etc.

[0086] The deep learning model can be a MobileNet model, a YOLO model, or an SSD (Single Shot MultiBoxDetector) model. The MobileNet model uses a Depthwise Separable Convolution algorithm.

[0087] The method for identifying occlusions provided in an embodiment of the present invention crops the image corresponding to the face area from the image captured by the camera, and then uses the trained deep learning model to obtain the recognition result of the occlusion. On the one hand, the recognition result of the occlusion is obtained by using the cropped image, which reduces the computational complexity of the trained deep learning model. On the other hand, the influence of the fixed occlusion of the face part is eliminated. When combined with the trained deep learning model, the accuracy of identifying temporary occlusions can be improved.

[0088] The present invention also provides an image stitching device for implementing any of the above method embodiments. Figure 2 FIG. 4 shows a structural block diagram of an image stitching device according to an embodiment of the present invention. Figure 2 As shown, the image stitching device may include an external sound acquisition module 210 , a facial contour determination module 220 , an image cropping module 230 , a first key point pair determination module 240 , a second key point pair determination module 250 and a stitching module 260 .

[0089] The acquisition module 210 is used to acquire a first image captured by a first camera and a second image captured by a second camera, where the first camera is arranged on the left earphone and the second camera is arranged on the right earphone.

[0090] The face contour determining module 220 is used to determine the left side head contour of the user according to the first image, and to determine the right side head contour of the user according to the second image.

[0091] The image cropping module 230 is used to crop the image surrounded by the left side of the user's head outline from the first image to obtain a first cropped image, and to crop the image surrounded by the right side of the user's head outline from the second image to obtain a second cropped image.

[0092] The first key point pair determining module 240 is used to obtain a first key point pair according to the left head contour of the user and the right head contour of the user.

[0093] The second key point pair determination module 250 is used to obtain a second key point pair by matching the first cropped image and the second cropped image.

[0094] The stitching module 260 is used to stitch the first image and the second image according to the first key point pair and the second key point pair to obtain a stitched image.

[0095] In one embodiment, the stitching module 260 is used to determine the homography matrix between the first image and the second image based on the first key point pair and the second key point pair; align the first image and the second image according to the homography matrix to obtain the aligned first image and the second image; and stitch the aligned first image and the second image to obtain a stitched image.

[0096] In one embodiment, the first key point pair determination module 240 is used to identify a first specific position point from the left side of the user's head contour, and to identify a second specific position point from the right side of the user's head contour; a first key point pair is obtained based on the first specific position point and the second specific position point.

[0097] In one embodiment, the second key point pair determination module 250 is used to obtain the second key point pair by matching the first cropped image and the second cropped image using a feature point matching algorithm.

[0098] In one embodiment, the device further includes an occluder recognition module. The occluder recognition module is used to determine a first recognition result of the occluder based on the first cropped image, and to determine a second recognition result of the occluder based on the second cropped image, wherein both the first recognition result and the second recognition result include whether there is an occluder and the category of the occluder.

[0099] In one embodiment, the first recognition result and the second recognition result further include the location of the obstruction.

[0100] In one embodiment, the device further comprises a prompt module. The prompt module is used to issue a prompt to the user that an obstruction exists when the first recognition result and / or the second recognition result indicates that an obstruction exists.

[0101] In one embodiment, the occluder recognition module is used to input the first cropped image into a trained deep learning model to obtain a first recognition result of the occluder, and to input the second cropped image into the trained deep learning model to obtain a second recognition result of the occluder.

[0102] The present invention also provides a hearing electronic device for implementing any of the above method embodiments. The hearing electronic device may be a TWS (True Wireless Stereo) earphone, a headphone, an open earphone, an audio glasses, and the like.

[0103] Figure 3 FIG. 4 shows a hardware structure block diagram of a hearing electronic device 300 according to an embodiment of the present invention. Figure 3 As shown, the hearing electronic device 300 includes a processor 310 and a memory 320 for storing instructions executable by the processor 310. The processor 310 is configured to implement the image stitching method according to any embodiment of the present disclosure when executing the instructions stored in the memory 320.

[0104] The processor 310 is used to execute computer instructions, which can be written in an instruction set of an architecture such as x86, Arm, RISC, MIPS, SSE, etc. The memory 320 includes, for example, ROM (read-only memory), RAM (random access memory), a non-volatile memory such as a hard disk, etc., which are not limited here.

[0105] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. For the device embodiment, its related parts can be referred to the partial description of the method embodiment.

[0106] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0107] The embodiments of the present specification may be systems, methods and / or computer program products. The computer program product may include a computer-readable storage medium carrying computer instructions for causing a processor to implement various aspects of the embodiments of the present specification.

[0108] A computer-readable storage medium may be a tangible device that can hold and store computer instructions used by a computer instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which computer instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0109] The computer instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device through a network layer, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network layer may include copper transmission cables, optical fiber transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. The network layer adapter card or network layer interface in each computing / processing device receives the computer instructions from the network layer and forwards the computer instructions for storage in a computer-readable storage medium in each computing / processing device.

[0110] The flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to the multiple embodiments of this specification. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a computer instruction, and a part of a module, a program segment or a computer instruction contains one or more executable computer instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of the boxes in the block diagram and / or the flowchart can be implemented by a dedicated hardware-based system that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that it is equivalent to implement it by hardware, implement it by software, and implement it by combining software and hardware.

[0111] The embodiments of the present specification have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations are obvious to those of ordinary skill in the art without departing from the scope of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. An image stitching method, characterized in that: include: Acquire a first image captured by a first camera and a second image captured by a second camera, wherein the first camera is arranged on a left earphone and the second camera is arranged on a right earphone; Determine a left side head contour of the user according to the first image, and determine a right side head contour of the user according to the second image; cropping the image surrounded by the left side of the user's head outline from the first image to obtain a first cropped image, and cropping the image surrounded by the right side of the user's head outline from the second image to obtain a second cropped image; Obtaining a first key point pair according to a left head contour of the user and a right head contour of the user; According to the first cropped image and the second cropped image, matching is performed to obtain a second key point pair; The first image and the second image are spliced ​​according to the first key point pair and the second key point pair to obtain a spliced ​​image.

2. The method according to claim 1, characterized in that The step of performing stitching processing on the first image and the second image according to the first key point pair and the second key point pair to obtain a stitched image includes: Determine a homography matrix between the first image and the second image according to the first key point pair and the second key point pair; Aligning the first image and the second image according to the homography matrix to obtain an aligned first image and a second image; The aligned first image and the second image are spliced ​​to obtain a spliced ​​image.

3. The method according to claim 1, characterized in that The obtaining a first key point pair according to the left head contour of the user and the right head contour of the user comprises: Identifying a first specific location point from a left side head contour of the user, and identifying a second specific location point from a right side head contour of the user; A first key point pair is obtained according to the first specific position point and the second specific position point.

4. The method according to claim 1, characterized in that: The matching according to the first cropped image and the second cropped image to obtain a second key point pair includes: A feature point matching algorithm is used to match the first cropped image and the second cropped image to obtain a second key point pair.

5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: A first recognition result of the occluder is determined based on the first cropped image, and a second recognition result of the occluder is determined based on the second cropped image, wherein the first recognition result and the second recognition result both include whether the occluder exists and the category of the occluder.

6. The method according to claim 5, characterized in that The first recognition result and the second recognition result also include the location of the obstruction.

7. The method according to claim 5, characterized in that The method further comprises: When the first recognition result and / or the second recognition result indicates that an obstruction exists, a prompt indicating the existence of the obstruction is sent to the user.

8. The method according to claim 5, characterized in that The determining a first recognition result of the occluder according to the first cropped image, and determining a second recognition result of the occluder according to the second cropped image, comprises: The first cropped image is input into a trained deep learning model to obtain a first recognition result of the occluder, and the second cropped image is input into the trained deep learning model to obtain a second recognition result of the occluder.

9. An image stitching device, characterized in that: include: An acquisition module, configured to acquire a first image captured by a first camera and a second image captured by a second camera, wherein the first camera is arranged on a left earphone and the second camera is arranged on a right earphone; A facial contour determination module, configured to determine a left side head contour of a user based on the first image, and to determine a right side head contour of the user based on the second image; an image cropping module, configured to crop the image surrounded by the left side of the user's head outline from the first image to obtain a first cropped image, and to crop the image surrounded by the right side of the user's head outline from the second image to obtain a second cropped image; A first key point pair determination module, configured to obtain a first key point pair according to a left head contour of the user and a right head contour of the user; A second key point pair determination module, configured to obtain a second key point pair by matching the first cropped image and the second cropped image; A stitching module is used to stitch the first image and the second image according to the first key point pair and the second key point pair to obtain a stitched image.

10. A hearing electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and the computer program is used to control the processor to operate so as to execute the method according to any one of claims 1 to 8.

Images