Vehicle sound area determination method and device, equipment and medium
By performing image detection and posture recognition on the target image in the vehicle, the user's area is determined as the voice area of the voice wake-up command, which solves the problem that the voice command cannot be recognized by the prior art midrange area being blocked, and improves the accuracy of voice control.
Patent Information
- Application Number
- CN202311359766.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-10-19
- Publication Date
- 2025-04-22
AI Technical Summary
In the voice control scenario of the vehicle cockpit, when the prior art determines the sound area through sound source positioning, the user may probe to other seats to wake up voice, and then sit back to his own seat, resulting in the sound area of his own seat being blocked and the voice command cannot be recognized.
By responding to the voice wake-up command, image detection is performed on multiple target images collected in the vehicle, key points information of the character is obtained, and preset correspondence relationships are queried according to the position change trend to determine the user's changing posture. If the user's changing posture is a preset changing posture, the target image is recognized and the area where the user is located is determined as the voice area of the voice wake-up command.
It realizes the sound area correction using visual information, accurately determines the sound area of voice wake-up, solves the problem of unable to recognize voice commands, improves the accuracy of voice control, and reduces erroneous operations.
Smart Images
Figure CN119864023A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of vehicle technology, and in particular to a method, device, equipment and medium for determining a sound zone of a vehicle. Background Art
[0002] With the increasing intelligence of vehicles, voice control has become a common function in vehicles, and users can control the vehicle through voice.
[0003] At present, in the voice control scenario inside the vehicle cabin, there is a sound zone isolation function. The sound zone isolation function means that if the sound zone in a certain seat area is awakened, only the sound in the seat area will be processed later, and the sound in other areas will be shielded. For example, for windows and doors with voice control functions, when the sound collection device obtains the voice wake-up command, it determines the area where the voice wake-up command is issued by locating the sound source.
[0004] However, since the effect of sound collection is related to the distance, there is a situation where the user probes to other seats for voice wake-up, and then sits back in his own seat for voice control. In this case, the sound zone is determined by sound source positioning. Since the device is in the sound zone of other seats when it is woken up, the sound zone of other seats will be woken up, while the sound zone of the own seat will not be woken up. Therefore, the sound zone of the own seat is blocked and the voice command cannot be recognized. Summary of the invention
[0005] In order to solve the above technical problems, the present disclosure provides a method, device, equipment and storage medium for determining a sound zone of a vehicle.
[0006] In a first aspect, an embodiment of the present disclosure provides a method for determining a sound zone of a vehicle, comprising:
[0007] In response to receiving a voice wake-up command, performing image detection on a plurality of target images to obtain key point information of a person in each target image; wherein the plurality of target images are collected in the vehicle within a time period including the moment when the voice wake-up command is received;
[0008] According to the position change trend of the key point information of the character in the multiple target images, querying a preset corresponding relationship to obtain the user's changed posture; the corresponding relationship includes the relationship between the position change trend and the user's changed posture;
[0009] If the user's changing posture is a preset changing posture, the target image is identified to determine the area where the user is located, and the area where the user is located is used as the sound zone of the voice wake-up command; wherein the preset changing posture includes the user's changing posture from his own seat to other seats.
[0010] In a second aspect, an embodiment of the present disclosure provides a device for determining a sound zone of a vehicle, comprising:
[0011] A detection module, configured to perform image detection on a plurality of target images in response to receiving a voice wake-up command, and obtain key point information of a person in each target image; wherein the plurality of target images are collected in the vehicle within a time period including the moment when the voice wake-up command is received;
[0012] A query module, configured to query a preset corresponding relationship to obtain a user's changed posture according to a position change trend of the character key point information in the multiple target images; the corresponding relationship includes a relationship between the position change trend and the user's changed posture;
[0013] A determination module is used to identify the target image to determine the area where the user is located if the user's change posture is a preset change posture, and use the area where the user is located as the sound zone of the voice wake-up instruction; wherein the preset change posture includes the change posture of the user from his own seat to other seats.
[0014] In a third aspect, an embodiment of the present disclosure provides an electronic device, comprising: a processor; a memory for storing executable instructions of the processor; the processor is used to read the executable instructions from the memory and execute the instructions to implement the method for determining the sound zone of a vehicle described in the first aspect above.
[0015] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the method for determining a sound zone of a vehicle described in the first aspect above is implemented.
[0016] In a fifth aspect, an embodiment of the present disclosure provides a vehicle, comprising the sound zone determination device for the vehicle as described in the second aspect above, or the electronic device as described in the third aspect above.
[0017] The technical solution provided by the embodiment of the present disclosure has the following advantages over the prior art: According to the technical solution of the embodiment of the present disclosure, in response to receiving a voice wake-up command, image detection is performed on multiple target images collected in the vehicle to obtain character key point information in each target image, and according to the position change trend of the character key point information in the multiple target images, a preset corresponding relationship is queried to obtain the user's changed posture. Therefore, compared with the method of determining the sound zone by sound source positioning, the sound zone is assisted in judging the sound zone based on the visual information in the cockpit. When it is detected that the user's changed posture is a preset changed posture, the target image is identified to determine the area where the user is located, and the area where the user is located is used as the sound zone of the voice wake-up command, so as to realize sound zone correction using visual information, accurately determine the sound zone of voice wake-up, and then accurately identify the voice command in the wake-up sound zone, thereby solving the technical problem in the prior art that the voice command cannot be recognized due to the sound zone of the seat being shielded, thereby improving the accuracy of voice control and reducing misoperation. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.
[0019] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0020] Figure 1 A schematic flow chart of a method for determining a sound zone of a vehicle provided by an embodiment of the present disclosure;
[0021] Figure 2 A schematic flow chart of another method for determining a sound zone of a vehicle provided by an embodiment of the present disclosure;
[0022] Figure 3 A schematic diagram of the structure of a device for determining a sound zone of a vehicle provided in an embodiment of the present disclosure. DETAILED DESCRIPTION
[0023] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.
[0024] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.
[0025] Figure 1 This is a flow chart of a method for determining a sound zone of a vehicle provided in an embodiment of the present disclosure. The method provided in an embodiment of the present disclosure can be executed by a sound zone determination device of a vehicle, which can be implemented in software and / or hardware and can be integrated in any electronic device with computing capabilities, such as a vehicle-mounted terminal or a cloud server.
[0026] like Figure 1 As shown, the method for determining the sound zone of a vehicle provided by the embodiment of the present disclosure may include:
[0027] Step 101, in response to receiving a voice wake-up instruction, performing image detection on multiple target images to obtain key point information of a person in each target image.
[0028] The method of the disclosed embodiment is applied to the scenario of voice wake-up in a vehicle cabin to determine the sound zone for voice wake-up. After the sound zone for voice wake-up is determined, only the sound in the sound zone is subsequently processed, and the sound in other areas is shielded.
[0029] In this embodiment, a sound collection device is provided in the vehicle for receiving voice wake-up instructions and voice control instructions. The voice wake-up instruction may be a preset wake-up word. After receiving the voice wake-up instruction, the vehicle-mounted terminal controls the vehicle according to the received voice control instruction, including but not limited to controlling the opening and closing of windows, controlling the opening and closing of doors, controlling the vehicle to navigate, and playing audio and video resources.
[0030] A camera is provided in the vehicle, which is used to capture target images in the vehicle. In response to the sound collection device receiving a voice wake-up command, the on-board terminal obtains multiple target images captured in the vehicle. Optionally, the multiple target images can be images obtained by capturing images in the vehicle within a preset time period, or can be a preset number of images selected from several images obtained by capturing images in the vehicle. The preset time period can be a time period including the moment when the voice wake-up command is received. For example, the preset time period is a time period five seconds before and five seconds after the voice wake-up command is received. The preset time period can also be set according to actual needs. The preset number of images includes images captured before receiving the voice wake-up command and images captured after receiving the voice wake-up command. The preset number can be determined according to needs and is not limited here.
[0031] The vehicle is provided with a front camera and a rear camera. The front camera continuously collects images of the front row in the vehicle, and the rear camera continuously collects images of the rear row in the vehicle. As an example, taking the front row in the vehicle as an example, when the sound collection device in the vehicle receives a voice wake-up command, multiple target images collected by the front camera within a preset time period are obtained, and there is timing information between the multiple target images, and the timing information represents the time sequence between the multiple target images. As another example, taking the rear row in the vehicle as an example, from the images continuously collected by the rear camera, 11 frames of images before receiving the voice wake-up command and 11 frames of images after receiving the voice wake-up command are selected as 22 frames of target images, and the 22 frames of target images are continuous and have timing information between them.
[0032] In this embodiment, the image detection technology in the field of artificial intelligence can be used to perform image detection on each target image to obtain the key point information of the person in each target image. The key point information of the person includes the key points of the human body and the key points of the face.
[0033] The image detection process is described below.
[0034] In one embodiment of the present disclosure, key point detection is performed on the human area in each target image by a pre-trained human key point detection network to obtain the human key points in each target image, and key point detection is performed on the facial area in each target image by a pre-trained facial key point detection network to obtain the facial key points in each target image.
[0035] Among them, the human key point detection network is a deep learning network built for image detection scenarios. The human key point detection network consists of a backbone network, a shoulder network, and a head network. It is used to detect whether there are human key points in the image and return the detection results. The detection results are, for example, the coordinates of the human key points. The training process of the human key point detection network includes, for example: obtaining a training set, the training set includes multiple sample images containing people, and the person area in each sample image is annotated with human key points. The above training set is used to train the human key point detection network model so that the human key point detection network can detect the human key points in the image.
[0036] The facial key point detection network is a deep learning network built for image detection scenarios. It consists of a backbone network and a face network. It is used to detect whether there are faces and facial key points in an image, and return the detection results, such as the coordinates of the face detection frame and the facial key points. The training process of the facial key point detection network includes, for example: obtaining a training set, the training set includes multiple sample images containing faces, and the facial area in each sample image is annotated with facial key points. The above training set is used to train the model of the facial key point detection network so that the facial key point detection network can detect facial key points in the image.
[0037] Among them, the key points of the human body include but are not limited to the key points of the head, the key points of the hands, the key points of the shoulders, the key points of the hips, etc., and the key points of the face include but are not limited to the key points of the eyes, the key points of the mouth, etc.
[0038] Step 102 , according to the position change trend of the key point information of the person in multiple target images, query the preset corresponding relationship to obtain the user's changed posture.
[0039] In this embodiment, the user's changing posture indicates the change in the user's human posture, and the human posture includes postures such as the human body sitting normally in the current seat and the human body probing to another seat. Optionally, a corresponding relationship is pre-set, which describes the relationship between the user's changing posture and the position change trend of the character's key point information. For example, the vertical height difference between the head key point and the hip key point first increases and then decreases, and first decreases and then increases, respectively corresponding to different changing postures. The user's changing posture can be determined by comparing whether the position change trend of the character's key point information in multiple target images is consistent with the position change trend set by the aforementioned corresponding relationship. Among them, the user's changing posture can include a preset changing posture and a non-preset changing posture. When the user's changing posture is a preset changing posture, the sound zone of the voice wake-up instruction is subsequently determined based on the target image.
[0040] As an example, taking the mouth key point as an example, the area where the mouth key point is located in each target image is determined, and if the area where the mouth key point is located changes from a certain area to another area, and then changes to the certain area again, the user's change posture is determined to be a preset change posture. The preset change posture includes the change posture of the user probing from his own seat to other seats.
[0041] Step 103: If the user's changed posture is a preset changed posture, the target image is recognized to determine the area where the user is located, and the area where the user is located is used as the sound zone of the voice wake-up command.
[0042] In this embodiment, when the sound collection device receives a voice wake-up instruction, it determines the area where the voice wake-up instruction is issued by means of sound source positioning. If the user's changed posture is not a preset changed posture, the area determined by the sound source positioning method is used as the sound zone of the voice wake-up instruction. If the user's changed posture is a preset changed posture, the area where the user is located is determined based on the target image, so that the area where the user is located is used as the sound zone of the voice wake-up instruction. After the sound zone of the voice wake-up instruction is determined, only the voice control instructions within the sound zone are subsequently processed, and the sounds in other areas are shielded. Among them, the preset changed posture can be a situation where the human body posture changes from the human body sitting normally in the current seat to the human body probing to another seat, and the human body sitting normally in the current seat, or it can be a situation where the human body posture changes from the human body sitting normally in the current seat to the human body probing to another seat. It is specifically set according to the needs of the actual voice wake-up scenario and is not limited here.
[0043] The following is an example of determining the area where the user is located based on the target image.
[0044] As an example, obtain the human body key points obtained by image detection in the aforementioned steps, or perform image detection on the target image to obtain the human body key points in the target image, and then determine the coordinates of the hip key points in the target image from the human body key points, and convert them according to the coordinates of the hip key points in the target image to determine the area where the hip key points are located in the vehicle as the area where the user is located.
[0045] According to the technical solution of the embodiment of the present disclosure, in response to receiving a voice wake-up instruction, image detection is performed on each target image to obtain key point information of a person in each target image, and according to the position change trend of the key point information of the person in the multiple target images, a preset corresponding relationship is queried to obtain the user's change posture. Therefore, compared with the method of determining the sound zone by sound source localization, the sound zone is assisted in judging the sound zone based on the visual information in the cabin. When it is detected that the user's change posture is a preset change posture, the target image is identified to determine the area where the user is located, and the area where the user is located is used as the sound zone of the voice wake-up instruction, so as to achieve sound zone correction using visual information, accurately determine the sound zone of voice wake-up, and then accurately identify the voice instruction in the wake-up sound zone, solve the technical problem in the prior art that the voice instruction cannot be recognized due to the shielding of the sound zone of the own seat, improve the accuracy of voice control, and solve the technical problems of false awakening and missed awakening of the sound zone isolation function in the voice awakening scenario, and low voice awakening accuracy, reduce false awakening and missed awakening, and improve the success rate of sound awakening. For example, when the user probes to other seats for voice awakening and then sits upright in his own seat, the sound zone of voice awakening can be accurately determined.
[0046] Based on the above embodiments, Figure 2A flow chart of another method for determining a sound zone of a vehicle provided by an embodiment of the present disclosure is shown as follows: Figure 2 As shown, the method includes:
[0047] Step 201 , in response to receiving a voice wake-up command, acquiring a plurality of target images captured inside a vehicle.
[0048] Step 202: perform key point detection on each target image to obtain human body key points and face key points in each target image.
[0049] The above explanation of step 101 is also applicable to steps 201 and 202 and will not be repeated here.
[0050] Step 203, according to the position change trends of the key points of the head, hips, and mouth of the user in the multiple target images, query the preset corresponding relationship to obtain the user's changed posture.
[0051] In this embodiment, the user's changing posture indicates the change in the user's body posture, which includes postures such as the body sitting normally in the current seat and the body probing to another seat. The preset changing posture can be the body posture changing from the body sitting normally in the current seat to the body probing to another seat and the body sitting normally in the current seat.
[0052] In one embodiment of the present disclosure, for each target image, the vertical height difference between the head key point and the hip key point is calculated, and the area where the mouth key point is located is determined. Then, if the vertical height difference corresponding to multiple target images first decreases and then increases, and the area where the mouth key point is located changes, it is determined that the user's change posture is a preset change posture.
[0053] As an example, the head key point coordinates (x0, y0) and the hip key point coordinates (x1, y1) are determined from the detection results of the human body key point detection network, and the mouth key point coordinates (x2, y2) are determined from the detection results of the face key point detection network, the vertical height difference y = |y1-y0| between the head key point and the hip key point is calculated, and the mouth key point coordinates (x2, y2) are used to determine the area where the mouth key point is located in the vehicle. If the vertical height difference y first decreases and then increases, and the area where the mouth key point is located changes from one area to another area and then changes to the one area again, then the user's change posture is determined to be the preset change posture.
[0054] In one embodiment of the present disclosure, for each target image, the angle between the line connecting the head key point and the hip key point and the horizontal direction is calculated, and the area where the mouth key point is located is determined. Then, if the angle corresponding to the multiple target images decreases first and then increases, and the area where the mouth key point is located changes, the user's change posture is determined to be a preset change posture. Among them, the horizontal direction refers to the horizontal direction in the horizontal plane, for example, the horizontal direction can be horizontal to the right (from the main driver's seat to the co-driver's seat), or horizontal to the back (from the front seat to the back seat), which is not limited here.
[0055] As an example, the head key point coordinates (x0, y0) and the hip key point coordinates (x1, y1) are determined from the detection results of the human body key point detection network, and the mouth key point coordinates (x2, y2) are determined from the detection results of the face key point detection network. By determining the angle α between the line connecting (x0, y0) and (x1, y1) and the horizontal direction, if the angle α first decreases and then increases, and the area where the mouth key point is located changes from one area to another area and then changes to the one area again, it is determined that the user's change posture is the preset change posture.
[0056] In one embodiment of the present disclosure, for each target image, the angle between the line connecting the head key point and the hip key point and the vertical direction is calculated, and the area where the mouth key point is located is determined. Then, if the angles corresponding to multiple target images first increase and then decrease, and the area where the mouth key point is located changes, it is determined that the user's change posture is a preset change posture.
[0057] As an example, the head key point coordinates (x0, y0) and the hip key point coordinates (x1, y1) are determined from the detection results of the human body key point detection network, and the mouth key point coordinates (x2, y2) are determined from the detection results of the face key point detection network. By determining the angle β between the line connecting (x0, y0) and (x1, y1) and the horizontal direction, if the angle β first increases and then decreases, and the area where the mouth key point is located changes from one area to another area and then changes to the one area again, it is determined that the user's change posture is the preset change posture.
[0058] Step 204: If the user's changed posture is a preset changed posture, the target image is recognized to determine the area where the user is located, and the area where the user is located is used as the sound zone of the voice wake-up command.
[0059] As an example, the facial key points obtained by image detection in the aforementioned step are obtained, or image detection is performed on the target image to obtain the facial key points in the target image. In this step, if the preset change posture represents the change posture of the user probing from his own seat to other seats and then returning to his own seat, the facial key points can adopt the target image with the latest acquisition time among multiple target images, and then, determine the coordinates of the mouth key points in the target image with the latest acquisition time from the facial key points, transform the coordinates of the mouth key points in the target image, and obtain the area where the mouth key points are located in the vehicle as the sound area of the voice wake-up command.
[0060] The following is a description of how to determine the area where the mouth key points are located based on the target image.
[0061] In this embodiment, the target image is divided into multiple image intervals, and the coordinates of the mouth key point are matched with the multiple pre-divided image intervals to determine the target image interval to which the mouth key point belongs, and the target seat area corresponding to the target image interval is determined as the area where the mouth key point is located. Among them, the image interval corresponds to the seat area in the vehicle one by one. As an example, for the front row image and the back row image in the vehicle, different numbers of image intervals can be divided. If the target image is the front row image in the vehicle, the coordinates of the mouth key point are matched with the preset first interval and the second interval. If the coordinates of the mouth key point are in the first interval, the area where the mouth key point is located is determined to be the first area of the front row in the vehicle. If the coordinates of the mouth key point are in the second interval, the area where the mouth key point is located is determined to be the second area of the front row in the vehicle. If the target image is an image of the rear row of the vehicle, the coordinates of the mouth key point are matched with the preset third interval, fourth interval and fifth interval. If the coordinates of the mouth key point are located in the third interval, the area where the mouth key point is located is determined to be the third area of the rear row of the vehicle. If the coordinates of the mouth key point are located in the fourth interval, the area where the mouth key point is located is determined to be the fourth area of the rear row of the vehicle. If the coordinates of the mouth key point are located in the fifth interval, the area where the mouth key point is located is determined to be the fifth area of the rear row of the vehicle.
[0062] In this example, the sound zone in the vehicle is divided into the main driver's area, the co-driver's area, the rear right area, the rear middle area and the rear left area. The resolution of the front camera is w0*h0, and the resolution of the rear camera is w1*h1. For the front image, if the key point coordinates are in the interval [0,w0 / 2], the key point is determined to be in the co-driver's area. If the key point coordinates are in the interval [w0 / 2,w0], the key point is determined to be in the main driver's area. For the rear image, if the key point coordinates are in the interval [0,w1 / 3], the key point is determined to be in the rear right area. If the key point coordinates are in the interval [w1 / 3,w1*2 / 3], the key point is determined to be in the rear middle area. If the key point coordinates are in the interval [w1*2 / 3,w1], the key point is determined to be in the rear left area.
[0063] According to the technical solution of the embodiment of the present disclosure, the user's changed posture is determined by combining the head key points, hip key points and mouth key points, which can more accurately determine the changes in human body posture. On this basis, when the user's changed posture is a preset changed posture, visual information is used to correct the sound zone, which further improves the accuracy of the sound zone determination and further improves the accuracy of subsequent voice control.
[0064] The disclosed embodiment also provides a device for determining a sound zone of a vehicle.
[0065] Figure 3 A schematic diagram of the structure of a vehicle sound zone determination device provided by an embodiment of the present disclosure, such as Figure 3 As shown, the device for determining the sound zone of a vehicle includes: a detection module 31 , a query module 32 , and a determination module 33 .
[0066] The detection module 31 is used for performing image detection on a plurality of target images in response to receiving a voice wake-up instruction, and obtaining key point information of a person in each target image; wherein the plurality of target images are collected in the vehicle within a time period including the moment when the voice wake-up instruction is received;
[0067] A query module 32, configured to query a preset corresponding relationship to obtain a user's changed posture according to a position change trend of the character key point information in the multiple target images; the corresponding relationship includes a relationship between the position change trend and the user's changed posture;
[0068] The determination module 33 is used to identify the target image to determine the area where the user is located if the user's change posture is a preset change posture, and use the area where the user is located as the sound zone of the voice wake-up instruction; wherein the preset change posture includes the change posture of the user from his own seat to other seats.
[0069] In one embodiment of the present disclosure, the detection module 31 is specifically used to: perform key point detection on the person area in each target image through a pre-trained human key point detection network to obtain the human key points in each target image; perform key point detection on the face area in each target image through a pre-trained face key point detection network to obtain the face key points in each target image.
[0070] In one embodiment of the present disclosure, the human body key points include head key points and hip key points, the face key points include mouth key points, and the query module 32 is specifically used to: for each target image, calculate the vertical height difference between the head key points and the hip key points, and determine the area where the mouth key points are located; if the vertical height difference corresponding to the multiple target images first decreases and then increases, and the area where the mouth key points are located changes, then determine that the user's change posture is a preset change posture.
[0071] In one embodiment of the present disclosure, the human body key points include head key points and hip key points, the face key points include mouth key points, and the query module 32 is specifically used to: for each target image, calculate the angle between the line connecting the head key points and the hip key points and the horizontal direction, and determine the area where the mouth key points are located; if the angle corresponding to the multiple target images first decreases and then increases, and the area where the mouth key points are located changes, then determine that the user's change posture is a preset change posture.
[0072] In one embodiment of the present disclosure, the determination module 33 is specifically used for: if the preset change posture represents the change posture of the user probing from his own seat to other seats and then returning to his own seat, performing image detection on the target image with the latest acquisition time among the multiple target images to obtain the mouth key points in the target image with the latest acquisition time; determining the area where the mouth key points are located, and using the area where the mouth key points are located as the sound area of the voice wake-up command.
[0073] In one embodiment of the present disclosure, the device also includes: a position determination module, used to match the coordinates of the mouth key point with multiple pre-divided image intervals to determine the target image interval to which the mouth key point belongs; wherein the image intervals correspond one-to-one to the seating areas in the vehicle; and the target seating area corresponding to the target image interval is determined as the area where the mouth key point is located.
[0074] The vehicle sound zone determination device provided in the embodiment of the present disclosure can execute any vehicle sound zone determination method provided in the embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method. The contents not fully described in the embodiment of the device of the present disclosure can refer to the description in any method embodiment of the present disclosure.
[0075] An embodiment of the present disclosure also provides an electronic device, which includes one or more processors and a memory.
[0076] The processor may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.
[0077] The memory may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on a computer-readable storage medium, and a processor may run the program instructions to implement the methods of the embodiments of the present disclosure above and / or other desired functions. Various contents such as input signals, signal components, noise components, etc. may also be stored in the computer-readable storage medium.
[0078] In one example, the electronic device may further include: an input device and an output device, and these components are interconnected through a bus system and / or other forms of connection mechanisms. In addition, the input device may also include, for example, a keyboard, a mouse, etc. The output device may output various information to the outside, including determined distance information, direction information, etc. The output device may include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, etc.
[0079] Of course, for simplicity, only some of the components in the electronic device related to the present disclosure are shown, omitting components such as a bus, an input / output interface, etc. In addition, the electronic device may further include any other appropriate components according to specific application scenarios.
[0080] In addition to the above methods and devices, the embodiments of the present disclosure may also be a computer program product, which includes computer program instructions. When the computer program instructions are executed by a processor, the processor executes any method provided by the embodiments of the present disclosure.
[0081] The computer program product may be written in any combination of one or more programming languages to write program code for performing the operations of the disclosed embodiments, including object-oriented programming languages such as Java, C++, etc., and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.
[0082] In addition, the embodiments of the present disclosure may also be a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes any method provided by the embodiments of the present disclosure.
[0083] Computer readable storage media can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
[0084] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0085] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A method for determining a sound zone of a vehicle, characterized in that: include: In response to receiving a voice wake-up command, performing image detection on a plurality of target images to obtain key point information of a person in each target image; wherein the plurality of target images are collected in the vehicle within a time period including the moment when the voice wake-up command is received; According to the position change trend of the key point information of the character in the multiple target images, querying a preset corresponding relationship to obtain the user's changed posture; the corresponding relationship includes the relationship between the position change trend and the user's changed posture; If the user's changing posture is a preset changing posture, the target image is identified to determine the area where the user is located, and the area where the user is located is used as the sound zone of the voice wake-up command; wherein the preset changing posture includes the user's changing posture from his own seat to other seats.
2. The method according to claim 1, characterized in that The performing of image detection on a plurality of target images to obtain key point information of a person in each target image includes: Performing key point detection on the human region in each target image through a pre-trained human key point detection network to obtain the human key points in each target image; The key points of the face in each target image are detected by using a pre-trained face key point detection network to obtain the face key points in each target image.
3. The method according to claim 2, characterized in that The human body key points include head key points and hip key points, the face key points include mouth key points, and the changing posture of the user is obtained by querying a preset corresponding relationship according to the position change trend of the key point information of the person in the multiple target images, including: For each target image, calculating the vertical height difference between the head key point and the hip key point, and determining the area where the mouth key point is located; If the vertical height difference corresponding to the multiple target images first decreases and then increases, and the area where the mouth key point is located changes, it is determined that the user's changed posture is a preset changed posture.
4. The method according to claim 2, characterized in that The human body key points include head key points and hip key points, the face key points include mouth key points, and the changing posture of the user is obtained by querying a preset corresponding relationship according to the position change trend of the key point information of the person in the multiple target images, including: For each target image, calculate the angle between the line connecting the head key point and the hip key point and the horizontal direction, and determine the area where the mouth key point is located; If the angles corresponding to the multiple target images decrease first and then increase, and the area where the mouth key point is located changes, it is determined that the user's changed posture is a preset changed posture.
5. The method according to claim 3 or 4, characterized in that The step of identifying the target image to determine the area where the user is located, and using the area where the user is located as the sound zone of the voice wake-up instruction includes: If the preset change posture indicates the change posture of the user probing from his own seat to another seat and then returning to his own seat, key point detection is performed on the target image with the latest acquisition time among the multiple target images to obtain the mouth key point in the target image with the latest acquisition time; Determine the area where the key points of the mouth are located, and use the area where the key points of the mouth are located as the sound area of the voice wake-up instruction.
6. The method according to claim 3 or 4, characterized in that Determine the area where the mouth key point is located, including: Matching the coordinates of the mouth key point with a plurality of pre-divided image intervals to determine the target image interval to which the mouth key point belongs; wherein the image intervals correspond one-to-one to the seat areas in the vehicle; The target seat area corresponding to the target image interval is determined as the area where the mouth key point is located.
7. A device for determining a sound zone of a vehicle, characterized in that: include: A detection module, configured to perform image detection on a plurality of target images in response to receiving a voice wake-up command, and obtain key point information of a person in each target image; wherein the plurality of target images are collected in the vehicle within a time period including the moment when the voice wake-up command is received; A query module, configured to query a preset corresponding relationship to obtain a user's changed posture according to a position change trend of the character key point information in the multiple target images; the corresponding relationship includes a relationship between the position change trend and the user's changed posture; A determination module is used to identify the target image to determine the area where the user is located if the user's change posture is a preset change posture, and use the area where the user is located as the sound zone of the voice wake-up instruction; wherein the preset change posture includes the change posture of the user from his own seat to other seats.
8. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the instructions to implement the vehicle sound zone determination method as described in any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method for determining the sound zone of a vehicle as described in any one of claims 1 to 6 is implemented.
10. A vehicle, characterized in that: include: The device as claimed in claim 7, or the electronic device as claimed in claim 8.