Feedback method and device for object searching of vehicle

Abstract

The invention provides a feedback method and device for object searching of a vehicle. The method comprises the following steps: identifying environment information of the vehicle where a target object is located based on an object position of the target object; the vehicle environment information is converted into the voice information, and the environment information of the target object can be recognized based on the object position of the target object and converted into the voice information for feedback, so that the object searching requirement of a user is effectively met, the riding space of the user is not affected, and the user experience is improved.

Description

Technical Field

[0001] This invention relates to the field of vehicle identification, and more specifically to a feedback method and apparatus for vehicle object retrieval. Background Technology

[0002] With the continuous development of the automotive industry and the continuous improvement of people's living standards, vehicles have become commonplace in people's daily lives. When drivers need to find items while operating a vehicle, their attention may be diverted from the road, making it difficult for them to concentrate on driving, or increasing their reaction time in the event of unexpected incidents, thus making them more prone to traffic accidents.

[0003] To address the aforementioned issues, related technologies have proposed using robotic arms to retrieve lost items to assist users in finding them. However, the robotic arm significantly occupies space in the vehicle's cabin, and the process of moving and retrieving the item also encroaches on the user's normal space, severely impacting the user's driving experience.

[0004] Therefore, there is an urgent need to provide a method and device for finding lost items that can meet users' needs without affecting their driving experience, thereby overcoming the above problems.

[0005] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of the present invention, and therefore may contain information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] To address the problems existing in the prior art, the present invention provides a feedback method and device for vehicle item retrieval, which can identify the environmental information of the target object based on the object's location and convert it into voice information for feedback, thereby effectively meeting the user's item retrieval needs without affecting the user's driving space and improving the user experience.

[0007] This invention provides a feedback method for vehicle item locating, comprising:

[0008] Identify the vehicle environment information where the target object is located based on the object's position;

[0009] The vehicle environment information is converted into voice information.

[0010] According to an embodiment of the present invention, the method of identifying the vehicle environment information where the target object is located based on the object's position includes:

[0011] Obtain a semantic image for determining the location of the item;

[0012] Based on the semantics corresponding to the semantic image, the vehicle environment information of the target object is determined.

[0013] According to one embodiment of the present invention, it further includes:

[0014] Acquire vehicle environment images;

[0015] The vehicle environment image is semantically segmented using a semantic segmentation model to obtain semantic images corresponding to each vehicle environment information.

[0016] According to an embodiment of the present invention, the method of identifying the vehicle environment information where the target object is located based on the object's position includes:

[0017] Obtain the environmental reference objects corresponding to the target object, and the positional relationship between the target object and the environmental reference objects;

[0018] Based on the environmental reference points and the positional relationship between the target object and the environmental reference points, the vehicle environment information of the target object is determined.

[0019] According to an embodiment of the present invention, obtaining the environmental reference object corresponding to the target object includes:

[0020] The environmental reference objects that are occluded with the target object are determined based on depth analysis.

[0021] According to an embodiment of the present invention, obtaining the environmental reference object corresponding to the target object includes:

[0022] Determine the pixel region corresponding to the target object;

[0023] A pixel search area is determined based on the pixel area, and any of the environmental reference objects is identified within the pixel search area.

[0024] If the environmental reference is not detected, the range of the pixel search area is expanded according to a preset rule until the environmental reference is detected in the pixel search area.

[0025] According to an embodiment of the present invention, converting the vehicle environment into voice information includes:

[0026] Obtain a text description corresponding to the target object and the vehicle environment information;

[0027] The text description is converted into speech information using speech synthesis technology.

[0028] According to an embodiment of the present invention, obtaining the text description corresponding to the target object and the vehicle environment information includes:

[0029] The text identifiers of the target object, the environmental reference objects, and the text identifiers corresponding to the positional relationship between the target object and the environmental reference objects are obtained respectively.

[0030] The textual description is obtained by semantically fusing the textual identifier and the positional relationship.

[0031] According to one embodiment of the present invention, it further includes:

[0032] Receive a user's voice input request for finding an item, and determine the target item to be found based on the request;

[0033] In response to the item retrieval request, a vehicle environment image is acquired, and the item location of the target object is determined based on the vehicle environment image.

[0034] According to one embodiment of the present invention, the vehicle environment information includes environmental reference objects and / or the positional relationship between the target object and the environmental reference objects; the environmental reference objects include one or more of the following: vehicle center console, driver's seat, passenger seat, rear seat, driver's door, passenger door, rear door, and trunk.

[0035] According to another aspect of the present invention, a vehicle-finding feedback device is also provided, comprising:

[0036] The identification module is used to identify the vehicle environment information where the target object is located based on the object's position.

[0037] The conversion module is used to convert the vehicle environment information into voice information.

[0038] According to another aspect of the present invention, a computer device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the vehicle locator feedback method as described above.

[0039] According to another aspect of the present invention, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the vehicle locator feedback method as described above.

[0040] According to another aspect of the present invention, a computer program product is also provided, comprising computer instructions that, when executed by a processor, implement the steps of the vehicle object locator feedback method as described above.

[0041] The vehicle object finding feedback method of this application embodiment can quickly and accurately find the target object for the driver and passenger, and use voice information to provide feedback on the vehicle environment information where the target object is located. This achieves object finding result feedback without affecting the user's driving status, which not only meets the user's object finding needs, but also effectively avoids driver distraction. In this way, it effectively improves the driving experience of the driver and passenger while effectively reducing the occurrence of driving accidents. Attached Figure Description

[0042] The above and other features of the invention will now be described in detail with reference to specific exemplary embodiments illustrated in the accompanying drawings. These exemplary embodiments are given by way of illustration only and are not intended to limit the invention.

[0043] Figure 1 An exemplary system architecture is shown for a specific implementation of the vehicle-finding feedback method applicable to the present invention.

[0044] Figure 2 A flowchart illustrating a vehicle location feedback method according to an embodiment of the present invention is shown.

[0045] Figure 3 A flowchart illustrating a vehicle location feedback method according to another embodiment of the present invention is shown.

[0046] Figure 4 A flowchart illustrating a vehicle location feedback method according to another embodiment of the present invention is shown.

[0047] Figure 5 A flowchart illustrating a vehicle location feedback method according to another embodiment of the present invention is shown.

[0048] Figure 6 A schematic diagram of a vehicle location feedback method apparatus according to an embodiment of the present invention is shown. Detailed Implementation

[0049] The present invention will be described in detail below through specific embodiments to enable those skilled in the art to easily implement the present invention based on the disclosure herein. The embodiments described below are only a part of the embodiments of the present invention, not all of them. All other embodiments obtained by those skilled in the art based on the embodiments described herein without creative effort are within the scope of protection of the present invention. It should be noted that, unless otherwise specified, the embodiments and features described in this specification can be combined with each other.

[0050] The terminology used herein is for illustrative purposes only and is not intended to limit the invention. Unless the context specifically indicates otherwise, the singular forms “a,” “an,” and “the” used herein also include the plural forms “a plurality,” “multiple,” and “the.” The terms “first,” “second,” etc., used herein are used only to distinguish different features, steps, operations, elements, and / or components, and do not indicate any specific technical meaning or necessary logical order between them. The term “a plurality” as used herein can refer to two or more, and the term “at least one” can refer to one, two, or more. Any feature, step, operation, element, and / or component mentioned herein is generally understood to mean one or more unless the context specifically indicates otherwise. It should also be understood that the terms “comprising” and / or “including” as used herein mean the presence of the stated feature, step, operation, element, and / or component, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, and / or a collection thereof. The term “and / or” as used herein includes any or all combinations of one or more of the associated listed items. The suffixes “module” and “unit” used in this article are for ease of description only, and therefore can be used interchangeably without any distinguishing meaning or function.

[0051] Where prior art related to the description of this invention is obvious to those skilled in the art, its detailed description will be omitted. It should also be understood that the description of the various embodiments in this specification emphasizes the differences between them; similarities or identical aspects between embodiments can be referred to mutually, and for the sake of brevity, these will not be elaborated upon.

[0052] With the development of technology, people's attention is often drawn to a deluge of information, causing them to overlook their personal belongings and lose items. For occupants in vehicles, the limited space and obstructions inside make it difficult to spot lost items with a simple glance. This is especially true when the driver is in motion; searching for lost items can severely distract them and increase the risk of accidents. For example, searching for lost items shifts the driver's attention from the road to the in-car environment, making it difficult to concentrate on driving and increasing the risk of collisions and accidents. Furthermore, this distraction prolongs the driver's reaction time, making it harder to respond promptly to emergencies or changes in the vehicle's movement. If the driver looks down or turns around while searching for lost items, they are more likely to misoperate the steering wheel, leading to vehicle instability and increasing the likelihood of loss of control, especially at high speeds or in complex traffic conditions.

[0053] Based on this, this application proposes a vehicle object finding feedback method that can quickly and accurately locate the target object and use voice information to provide feedback on the vehicle environment where the target object is located without affecting the user's driving status. This satisfies the user's object finding needs and effectively avoids distracting the driver.

[0054] like Figure 1 The illustration shows an exemplary system architecture 100 of a specific implementation of the vehicle-to-object-finding feedback method of the present invention. System architecture 100 may include a vehicle 101, an image acquisition device 102 mounted on the vehicle, a voice acquisition device 103, a voice playback device 104, and a server 105.

[0055] The image acquisition device 102 is installed inside the cabin and / or around the vehicle 101 to acquire image information from inside the cabin and / or around the vehicle, so that the server 105 can determine the location of the target object and the vehicle's environmental information based on the image information from inside the cabin and / or around the vehicle. One or more image acquisition devices 102 can be used, with the aim of acquiring image information from inside the cabin and / or around the vehicle. The voice acquisition device 103 and the voice playback device 104 are installed inside the vehicle 101. The voice acquisition device 103 receives user voice input requests for finding objects, and the voice playback device 104 outputs (plays) the voice information determined by the server 105. One or more voice acquisition devices 103 and voice playback devices 104 can be used, with the aim of acquiring voices from users in various locations within the vehicle cabin and ensuring the playback volume meets the reception needs of users in each location.

[0056] Server 105 is connected to image acquisition device 102, voice acquisition device 103, and voice playback device 104 respectively. Server 105 is used to receive the user's voice input for finding a lost item collected by voice acquisition device 103, determine the target item to be found according to the lost item request, then acquire vehicle environment image through image acquisition device 102, determine the location of the target item according to the vehicle environment image, identify the vehicle environment information where the target item is located, and convert the vehicle environment information into voice information. Finally, server 105 sends the voice information to voice playback device 104, which plays the voice information to provide feedback on the lost item result to the user.

[0057] Optionally, server 105 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0058] The image acquisition device 102, voice acquisition device 103, and voice playback device 104 are directly or indirectly connected to the server 105 via wireless communication. Optionally, the aforementioned wireless network uses standard communication technologies and / or protocols. The network is typically the Internet, but can also be any network, including but not limited to any combination of Local Area Network (LAN), Metropolitan Area Network (MAN), Wide Area Network (WAN), mobile, wired or wireless networks, private networks, or virtual private networks.

[0059] It should be noted that the vehicle-finding feedback method provided in this application embodiment can be executed by the vehicle-mounted terminal, by the server 105, or by the vehicle-mounted terminal and the server 105 working together. Accordingly, the vehicle-finding device can be installed in the vehicle-mounted terminal, in the server 105, or in both the vehicle-mounted terminal and the server 105.

[0060] It is understood that when the vehicle-to-object feedback method provided in the embodiments of this application is executed by the vehicle terminal, the above system architecture 100 may not include the server 105.

[0061] It should be understood that Figure 1 The number and types of image acquisition devices 102, voice acquisition devices 103, and voice playback devices 104 shown in the diagram are merely illustrative. In actual implementation, the diagram may contain any number and type of image acquisition devices 102, voice acquisition devices 103, and voice playback devices 104, as well as a server, depending on actual needs.

[0062] like Figure 2 As shown, the present invention provides a vehicle-finding feedback method 1000. Specifically, method 1000 includes:

[0063] S1100 identifies the vehicle environment information where the target object is located based on the object's position.

[0064] It should be noted that the target object is a lost item that the driver or passenger needs to find. The name information of the target object, such as a mobile phone or keys, can be obtained by extracting the lost item request issued by the driver or passenger. In this embodiment of the application, the driver or passenger can input the lost item request via voice.

[0065] The object's location refers to its specific location information, such as its coordinates in physical space. In this embodiment, the specific location of the object includes, but is not limited to, a location inside the vehicle and / or a location around the vehicle. Optionally, the object's location can be determined by image recognition of images inside and outside the vehicle.

[0066] As a feasible embodiment, the system receives a user's voice input request for finding an item, determines the target item to be found based on the request, and in response to the request, acquires a vehicle environment image and determines the location of the target item based on the vehicle environment image.

[0067] Specifically, the system receives voice data input from drivers and passengers, and processes it using Natural Language Processing (NLP) technology to identify the content carried by the voice data. When the content of the voice data is a request to find something, keywords are further extracted from the voice data to obtain the target object to be found. For example, if the user inputs the voice data "Where is my phone?", the keyword extraction will yield "phone", which will be used as the target object for searching. Simultaneously with determining that the voice data is a request to find something, the system responds by acquiring images of the vehicle's environment, such as images of the vehicle's interior and / or exterior (i.e., images of the vehicle's cabin and / or the vehicle's perimeter). Image recognition is then performed on these images to determine the specific location of the target object, such as the location of the user's phone in the image, its position in the physical coordinate system, or any other information that can determine the location of the target object.

[0068] Optionally, the vehicle is equipped with a voice acquisition device for receiving voice input from drivers and passengers, such as a vehicle-mounted voice system. After the system is started (powered on), the vehicle-mounted voice system initializes and automatically configures necessary audio processing parameters, such as noise suppression level and sound activity threshold detection, to optimize the quality of voice capture. Then, the vehicle-mounted voice system enters a low-power continuous detection mode to detect the sound activity of drivers and passengers in the vehicle in real time, so as to quickly respond to the voice input requests for finding items.

[0069] It should also be noted that the vehicle environment information surrounding the target object is reference information used to effectively locate the target object's position, including environmental reference objects and / or the positional relationship between the target object and environmental reference objects. Environmental reference objects include adjacent objects of the target object, other objects within the same segmented area, other objects within the acquisition area corresponding to the same image acquisition device, and other objects related to the target object; this application does not specifically limit the scope of these reference objects. For example, next to the vehicle's center console, or under the driver's seat, where "vehicle center console" and "driver's seat" are environmental reference objects, and "next to" and "under" represent the positional relationship between the target object and the environmental reference objects. Environmental reference objects include one or more of the following: vehicle center console, driver's seat, passenger seat, rear seats, driver's door, passenger door, rear door, and trunk.

[0070] The S1200 converts vehicle environmental information into voice information.

[0071] In other words, speech synthesis technology can be used to convert the vehicle environment information of the identified target into speech information, so that it can be broadcast by a voice playback device. This allows drivers and passengers to obtain the results of finding the target and pick it up when conditions permit, such as after the driver pulls over to the side of the road.

[0072] In one feasible embodiment, a textual description corresponding to the target object and vehicle environment information can be obtained, and then the textual description can be converted into speech information based on speech synthesis technology.

[0073] Among them, the text description corresponding to the target object and vehicle environment information is a text description obtained by semantic fusion based on the text identifier of the target object, the text identifier of the environmental reference object, and / or the text identifier corresponding to the positional relationship between the target object and the environmental reference object.

[0074] Specifically, textual identifiers of the target object, vehicle environment information, and the positional relationship between the target object and environmental references can be obtained separately. Then, semantic fusion is performed using natural language processing (NLP) technology to obtain a textual description that conforms to natural word order.

[0075] For example, when the target object is a "mobile phone," the vehicle environment information is the "vehicle center console," and the positional relationship between the target object and the environmental reference is "adjacent," semantic fusion can yield the text description "the mobile phone is next to the vehicle center console." Alternatively, when the target object is a "key," the vehicle environment information is the "driver's seat," and the positional relationship between the target object and the environmental reference is "below," semantic fusion can yield the text description "the key is under the driver's seat."

[0076] Furthermore, the text description is converted into speech information using text-to-speech (TTS) technology. For example, text analysis can be used to convert character text into language features, and then acoustic features can be generated using an acoustic model to obtain the final speech information that can be played using a speech playback device.

[0077] Optionally, during speech synthesis, a more user-friendly voice information format can be set. For example, if the text description "the key is under the driver's seat" is determined, the output voice information can be "your key is under the driver's seat" after adjusting the voice information format to improve the experience of drivers and passengers.

[0078] In one feasible embodiment, multiple vehicle environment information of the target object can be identified based on the object's location, such as the mobile phone being above the seat or to the left of the seat cushion. At this time, text descriptions corresponding to the target object and multiple vehicle environment information can be obtained respectively, and then converted into voice information for playback.

[0079] Optionally, multiple vehicle environmental information can be played in the order of recognition or in a preset order.

[0080] Therefore, the vehicle-finding feedback method provided in this application embodiment can quickly and accurately find the target object for the driver and passenger, and use voice information to provide feedback on the vehicle environment information where the target object is located. This achieves the feedback of the finding result without affecting the user's driving status, which not only meets the user's finding needs, but also effectively avoids the driver's attention being distracted. In this way, it effectively improves the driving experience of the driver and passenger while effectively reducing the occurrence of driving accidents.

[0081] In one feasible embodiment, such as Figure 3 As shown, S1100 identifies the vehicle environment information where the target object is located based on the object's position, including:

[0082] S1111, Obtain a semantic image for determining the location of the item.

[0083] S1112, Determine the vehicle environment information of the target object based on the semantics corresponding to the semantic image.

[0084] It should be understood that a semantic image is an image used to represent the location of a target object and its corresponding region, and it contains semantic information about that region. Therefore, when a target object is identified in this image, the vehicle environment information in which the target object is located can be determined based on the semantics of the semantic image itself (i.e., the semantics of the corresponding region).

[0085] In an optional embodiment, since the layout of the vehicle cabin environment is complex, multiple image acquisition devices can be used to acquire images of different semantic regions, and then the semantics of each image can be determined according to the image acquisition device to which it belongs.

[0086] Specifically, based on the area that each image acquisition device can capture, the semantic meaning corresponding to that image acquisition device is determined, so that the image acquired by each image acquisition device is a semantic image corresponding to that semantic area. When a target object appears in a certain semantic image, the semantic meaning corresponding to the object's location is determined according to the semantic meaning corresponding to the image acquisition device acquired in that semantic image. For example, if an image acquisition device is used to acquire an image of the vehicle's center console area, then the semantic meaning corresponding to that image acquisition device is the vehicle's center console. In this case, if a target object is identified in the image acquired by that image acquisition device, then the vehicle environment in which the target object is located is determined to be the vehicle's center console according to the semantic meaning corresponding to the image acquisition device that acquired that image. Or, if another image acquisition device is used to acquire an image of the area outside the driver's side door of the vehicle, then the semantic meaning corresponding to that image acquisition device is the driver's side door. In this case, if a target object is identified in the image acquired by that image acquisition device, then the vehicle environment in which the target object is located is determined to be the driver's side door according to the semantic meaning corresponding to the image acquisition device that acquired that image.

[0087] In another optional embodiment, to ensure that no vehicle environment is missed, multiple image acquisition devices can be used to acquire regional images of corresponding areas respectively. Then, image stitching technology can be used to stitch the multiple regional images into a complete vehicle environment image, such as an image of the vehicle's interior environment and an image of the vehicle's exterior environment. Then, semantic segmentation technology can be used to segment the complete image into multiple semantic images according to the corresponding segmentation semantics.

[0088] Specifically, in response to a locator request, a vehicle environment image is acquired, and then a semantic segmentation model is used to perform semantic segmentation on the vehicle environment image to obtain a semantic image corresponding to each piece of vehicle environment information.

[0089] Image semantic segmentation involves dividing pixels in an image according to their semantic meaning. It's a pixel-level classification process that semantically labels the entire image by classifying each pixel. Specifically, image semantic segmentation categorizes each pixel in an image, determining its semantic category, thus obtaining multiple semantic pixel regions. Pixels within the same semantic pixel region belong to the same semantic category. A semantic category is a linguistic expression based on different semantic expressions. For example, in a vehicle environment image scene, semantic segmentation can be divided into spatial regions corresponding to the image, such as the driver's cabin, passenger cabin, rear seats, and trunk. After semantic segmentation, semantic images of the driver's cabin, passenger cabin, rear seats, and trunk can be obtained.

[0090] In one specific embodiment, in response to a locator request, at least one image acquisition device inside the vehicle cabin acquires corresponding internal region images, and at least one image acquisition device outside the vehicle acquires corresponding external region images. Then, the at least one internal region image and the at least one external region image are stitched together to obtain an internal stitched image and an external stitched image. Segmentation semantics are defined according to the vehicle structure, and the internal and external stitched images are input into an image semantic segmentation model to obtain semantic images corresponding to each segmentation semantic. For example, after segmenting the internal stitched image, semantic images of the driver's cabin, passenger cabin, rear seats, and trunk are obtained; and after segmenting the external stitched image, semantic images of the front of the vehicle, the driver's side door exterior, and the passenger side door exterior are obtained.

[0091] Furthermore, image recognition is performed on the obtained multiple semantic images to identify the target object from any semantic image. When the image recognition model or image recognition algorithm identifies the target object, the segmentation semantics corresponding to the semantic image used to identify the target object are obtained, and the segmentation semantics are used as the vehicle environment information where the target object is located.

[0092] For example, when the target object is a "key", the semantic image of the target object is a segmented image of the driver's side door exterior. That is, the segmented semantics corresponding to the semantic image used to identify the target object is "driver's side door exterior". At this time, the vehicle environment information where the target object "key" is located is determined to be "driver's side door exterior". It should be understood that in this embodiment, "driver's side door" is an environmental reference, and "exterior" is the positional relationship between the target object and the environmental reference.

[0093] It should be understood that, due to the complex layout of the vehicle cabin environment, the segmentation semantics can be refined according to the actual situation. For example, the driver's cabin segmentation image can be further segmented into semantic images above the driver's seat and semantic images below the driver's seat.

[0094] In one feasible embodiment, after at least one image acquisition device acquires the corresponding region images, it can first align the region images, and then use an efficient image stitching algorithm to generate a panoramic image, namely a panoramic image of the vehicle cabin and a panoramic image of the vehicle exterior. Then, steps such as noise reduction and contrast enhancement are performed on the panoramic image to improve its quality, thereby enhancing the accuracy and reliability of target object recognition.

[0095] Therefore, this application uses the technical path of identifying target objects to reverse infer logic. When the location of the target object is identified, the vehicle environment information where the target object is located can be directly determined based on the semantics corresponding to the semantic image used to identify the target object. No additional computing overhead is required. While reducing image computing costs, the vehicle environment information where the target object is located can be obtained quickly and accurately.

[0096] In one feasible embodiment, the stitched panoramic image can also be used directly for target identification. However, since the panoramic image has not undergone semantic segmentation, the vehicle environment information of the target cannot be directly determined through semantic segmentation. Therefore, the vehicle environment information of the target can be determined by identifying environmental references around the target after it has been identified.

[0097] Specifically, such as Figure 4 As shown, S1100 identifies the vehicle environment information where the target object is located based on the object's position, including:

[0098] S1121, Obtain the environmental reference objects corresponding to the target object, and the positional relationship between the target object and the environmental reference objects.

[0099] In one feasible embodiment, environmental reference objects that have an occlusion relationship with the target object are determined based on depth analysis.

[0100] Depth refers to the distance from the surface of an object in the scene to the image acquisition device. Occlusion relationships include, but are not limited to, the target object occluding the environmental reference object and the environmental reference object occluding the target object.

[0101] It should be understood that, in spatial relationships, the occlusion relationship between objects can effectively determine the other object in the relationship and the positional relationship between them. For example, when a mobile phone is under the driver's seat, the driver's seat will usually obscure the phone in the image; if the phone is above the driver's seat, the phone will usually obscure the driver's seat in the image. Therefore, after identifying the target object, depth analysis is performed on the area where the target object is located to identify objects that have an occlusion relationship with the target object. Type identification is then performed on these objects to obtain environmental reference objects, and the positional relationship between the target object and the environmental reference objects is determined based on the depth relationship between them.

[0102] It should also be noted that when using depth analysis to determine occlusion relationships, at least two image acquisition devices are usually required to acquire images of the same area. Then, by using the acquisition positions of at least two image acquisition devices and the position information of the target object and environmental reference objects in the images, the occlusion relationship between the target object and the environmental reference objects can be determined.

[0103] Therefore, this application can quickly locate environmental reference objects that are positionally related to the target object, and determine the positional relationship between the target object and the environmental reference objects through image recognition and depth analysis, which can effectively improve the efficiency of environmental reference object recognition, thereby improving the object finding efficiency.

[0104] In another feasible embodiment, environmental references are determined by searching for adjacent objects of the target.

[0105] Specifically, the pixel region corresponding to the target object is determined, the pixel search region is determined based on the pixel region, any environmental reference object is identified in the pixel search region, and if no environmental reference object is identified, the range of the pixel search region is expanded according to a preset rule until an environmental reference object is identified in the pixel search region.

[0106] The process begins by identifying the outline of the target object. The area enclosed by this outline is then defined as the pixel region corresponding to the target object. A certain range outside the target object's outline is then designated as the pixel search region. Image recognition technology is used to perform image recognition within this region. When a preset environmental reference is detected, the corresponding environmental reference is obtained. If no preset environmental reference is detected in the current pixel search region, the search region is expanded, and recognition is repeated within this expanded region until an environmental reference is found.

[0107] It should be noted that, in some embodiments of this application, the environmental reference object can be any object inside or outside the vehicle that is identified, including but not limited to interior decorations arranged by the vehicle owner, such as cushions on the seats, storage racks behind the front seats and items placed on them, or the ground and tires outside the vehicle. This application does not impose specific limitations. In other embodiments of this application, the environmental reference object is usually preset, such as the driver's seat or the vehicle's center console, so that the environmental reference object used for positioning is a reference object that the driver and passengers can directly determine, avoiding invalid feedback of the search results or triggering a chain of search problems.

[0108] Optionally, the pixel search area directly determined based on the pixel region corresponding to the target object can be the entire region image containing the target object. When expanding the pixel search area, it can be expanded to the region image acquired by the adjacent image acquisition device, or the range of the adjacent region determined during stitching.

[0109] In some feasible embodiments, the expansion of the pixel search area may not stop upon the identification of a reference object. That is, after an environmental reference object is identified, the range of the pixel search area can continue to be expanded until a preset pixel search area range is reached. At least one item identified during the expansion of the search area range is used as an environmental reference object corresponding to the target object.

[0110] In other words, when acquiring environmental reference objects corresponding to a target object, and the positional relationship between the target object and the environmental reference objects, identification can be performed based on the aforementioned stitched image, or it can be performed directly without stitching. That is, the target object can be identified by directly using images acquired separately by multiple image acquisition devices, and then the environmental reference objects can be identified. It should be understood that after the environmental reference objects are identified, the positional relationship between the target object and the environmental reference objects can be identified, thereby obtaining the positional relationship between the target object and the environmental reference objects.

[0111] Therefore, items that are visually adjacent to the target can be identified through the pixel search area, which is the same as the visual experience when drivers and passengers search. This makes it easier for drivers and passengers to observe and locate the target when picking it up, thus improving the user experience.

[0112] In another feasible embodiment, when direct recognition is performed without stitching, object recognition can also be performed directly on the entire image containing the target object, and at least one image recognized in the image can be used as an environmental reference. For example, when a user's mobile phone is left on a seat, object recognition can be performed on the entire image captured by the image acquisition device to obtain at least one environmental reference, such as the seat and the cushion on the seat.

[0113] S1122, Based on environmental reference objects and the positional relationship between the target object and the environmental reference objects, determine the vehicle environment information where the target object is located.

[0114] Specifically, the positional relationship between environmental reference objects and the target object and environmental reference objects is combined to form the vehicle environment information of the target object.

[0115] Therefore, this application proposes two methods for determining the vehicle environment information corresponding to the target object without semantic segmentation processing, which enriches the scenarios and equipment conditions for determining vehicle environment information and improves the generalization ability of the vehicle object finding method proposed in this application.

[0116] In one specific embodiment, such as Figure 5 As shown, the feedback method for vehicle-to-object retrieval may include: initializing and configuring the vehicle's voice system; when a driver or passenger issues a retrieval request, activating the voice acquisition device to receive the request; the vehicle's voice system can extract keywords from the received retrieval request to identify the target object; in response to receiving the retrieval request, activating the vehicle's image acquisition device to acquire images; then determining whether there are multiple vehicle image acquisition devices; if so, stitching together the images of multiple regions acquired by the multiple image acquisition devices and performing semantic segmentation; otherwise, performing semantic segmentation directly without stitching; identifying the target object in the segmented image to determine the object's location; then determining the vehicle's environmental information corresponding to the target object based on its location; converting the vehicle's environmental information corresponding to the target object into voice information; and sending the voice information to the voice playback device for playback.

[0117] It should be understood that the vehicle-finding feedback method provided in this application embodiment can also be applied to other similar environments, such as home and office environments, to help users find daily necessities and office supplies, or in retail and supermarket environments, to help users find goods they need to buy, etc. This application does not make any specific limitations.

[0118] Based on the same inventive concept. Figure 6 A schematic diagram of a vehicle object locator feedback device 3000 according to an embodiment of the present invention is shown. The device 3000 includes: an identification module 3100, configured to identify vehicle environment information where the target object is located based on the object's location. In an embodiment of the present invention, the identification module 3100 can be configured to perform... Figure 2 S1100 Figure 3 S1111-S1112, Figure 4 S1121-S1122 shown and in this specification Figure 2 S1100 Figure 3 S1111-S1112 and, Figure 4 The steps corresponding to S1121-S1122 are shown; the conversion module 3200 is configured to convert the vehicle environment information into voice information. In an embodiment of the present invention, the reminder module 3200 can be configured to perform... Figure 2 S1200 and the components described in this specification Figure 2 The steps corresponding to S1200.

[0119] Understandable. Figure 6 The structure shown is for illustrative purposes only; the device may also include components with a larger diameter. Figure 6 The more or fewer modules or components shown, or having the same Figure 6 The different configurations shown.

[0120] Furthermore, this application also provides a computer device. According to an embodiment of the present invention, the computer device may include a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it can implement the steps of the vehicle locator feedback method described in this specification.

[0121] Furthermore, this application also provides a computer-readable medium, which may be included in the apparatus described in the above embodiments; or it may exist independently and not assembled into the apparatus. The computer-readable medium carries one or more programs that, when executed by the apparatus, enable the apparatus to perform the steps of the vehicle-finding feedback method described in this specification.

[0122] In addition, this application also provides a computer program product, including computer instructions that, when executed by a processor, can implement the steps of the vehicle object finding feedback method described in this specification.

[0123] In particular, the embodiment processes described above with reference to the flowcharts in the accompanying drawings can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts in the drawings, and the methods of this application being executed by a processor.

[0124] It should be noted that the computer-readable medium described in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. Computer-readable storage media can be, for example,, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0125] In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wireline, optical fiber, RF, etc., or any suitable combination thereof.

[0126] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0127] The flowcharts and block diagrams in the accompanying drawings exemplify the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than shown in the drawings. For example, two blocks shown sequentially may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0128] The units or modules involved in the embodiments of this application can be implemented in software or hardware. These units or modules can also be housed in a processor; for example, it can be described as a processor including an identification module and a conversion module, etc. The names of these units or modules do not necessarily constitute a limitation on the unit or module itself in certain circumstances.

[0129] All references mentioned in this specification are incorporated herein by reference as if each reference were incorporated herein by reference in its entirety.

[0130] Furthermore, it should be understood that after reading the above description of the present invention, those skilled in the art can make various alterations or modifications to the present invention, and these equivalent forms also fall within the protection scope of the present invention.

Claims

1. A feedback method for a vehicle search, characterized by, The method comprises: identifying vehicle environment information in which a target object is located based on an object position of the target object; converting the vehicle environment information into voice information.

2. The feedback method for vehicle item search according to claim 1, wherein, The identifying vehicle environment information in which the target object is located based on the object position of the target object comprises: obtaining a semantic image used for determining the object position; determining vehicle environment information in which the target object is located based on semantics corresponding to the semantic image.

3. The feedback method for vehicle item search according to claim 2, wherein, The method further comprises: obtaining a vehicle environment image; performing semantic segmentation on the vehicle environment image by using a semantic segmentation model to obtain a semantic image corresponding to each vehicle environment information.

4. The feedback method for vehicle item search according to claim 1, characterized by, The identifying vehicle environment information in which the target object is located based on the object position of the target object comprises: obtaining an environment reference corresponding to the target object and a positional relationship between the target object and the environment reference; determining vehicle environment information in which the target object is located based on the environment reference and the positional relationship between the target object and the environment reference.

5. The feedback method for vehicle item search according to claim 4, wherein, The obtaining the environment reference corresponding to the target object comprises: determining the environment reference having an occlusion relationship with the target object based on depth analysis.

6. The feedback method for vehicle item search according to claim 4, wherein, The obtaining the environment reference corresponding to the target object comprises: determining a pixel region corresponding to the target object; determining a pixel search region based on the pixel region, and identifying any environment reference in the pixel search region; when the environment reference is not identified, expanding the range of the pixel search region according to a preset rule until the environment reference is identified in the pixel search region.

7. The feedback method for vehicle item search according to claim 1, wherein, The converting the vehicle environment information into voice information comprises: obtaining a textual description corresponding to the target object and the vehicle environment information; converting the textual description into voice information based on voice synthesis technology.

8. The feedback method for vehicle item search according to claim 7, wherein, The obtaining the textual description corresponding to the target object and the vehicle environment information comprises: respectively obtaining a textual identifier of the target object, a textual identifier of the environment reference, and / or a textual identifier corresponding to the positional relationship between the target object and the environment reference; performing semantic fusion on the textual identifiers and the positional relationship to obtain the textual description.

9. The feedback method for vehicle item search according to claim 1, wherein, The method further comprises: receiving a user voice input of a lost object request, and determining the target object to be found according to the lost object request; in response to the lost object request, obtaining a vehicle environment image, and determining an object position of the target object according to the vehicle environment image.

10. The feedback method for vehicle item search according to any one of claims 1 to 9, characterized in that, The vehicle environment information comprises an environment reference and / or a positional relationship reference between the target object and the environment reference.

11. A feedback device for a vehicle search, characterized by, The method comprises: a recognition module configured to identify vehicle environment information in which a target object is located based on an object position of the target object; a conversion module configured to convert the vehicle environment information into voice information.

12. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the steps of the method of any one of claims 1-10.

13. A computer readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the steps of the method of any one of claims 1-10.

14. A computer program product comprising computer instructions, characterized in that, The computer instructions are executed by the processor to implement the steps of the method of any one of claims 1-10.

Images